Fuse LLM completion stream to avoid a panic (#21914)

Cole Miller created 1 year ago

`LanguageModel::stream_completion_text` can poll the `stream_completion`
stream (ultimately a `futures::Unfold`) after it's returned
`Ready(None)`, which leads to a panic; avoid this by fusing the stream.

Release Notes:

- Fixed a panic when streaming language model completions

Change summary

crates/language_model/src/language_model.rs | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Detailed changes

crates/language_model/src/language_model.rs

@@ -147,7 +147,7 @@ pub trait LanguageModel: Send + Sync {
         let events = self.stream_completion(request, cx);
 
         async move {
-            let mut events = events.await?;
+            let mut events = events.await?.fuse();
             let mut message_id = None;
             let mut first_item_text = None;