copilot: Fix rate limit due to Copilot-Vision-Request header (#30989)

Umesh Yadav created 6 months ago

Issues: #30994

I've implemented an important optimisation in response to GitHub
Copilot's recent rate limit on concurrent Vision API calls. Previously,
our system was defaulting to vision header: true for all API calls. To
prevent unnecessary calls and adhere to the new limits, I've updated our
logic: the vision header is now only sent if the current message is a
vision message, specifically when the preceding message includes an
image.

Prompt used to reproduce and verify the fix: `Give me a context for my
agent crate about. Browse my repo.`

Release Notes:

- copilot: Set Copilot-Vision-Request header based on message content

Change summary

crates/copilot/src/copilot_chat.rs | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)

Detailed changes

crates/copilot/src/copilot_chat.rs

@@ -581,6 +581,15 @@ async fn stream_completion(
     api_key: String,
     request: Request,
 ) -> Result<BoxStream<'static, Result<ResponseEvent>>> {
+    let is_vision_request = request.messages.last().map_or(false, |message| match message {
+        ChatMessage::User { content }
+        | ChatMessage::Assistant { content, .. }
+        | ChatMessage::Tool { content, .. } => {
+            matches!(content, ChatMessageContent::Multipart(parts) if parts.iter().any(|part| matches!(part, ChatMessagePart::Image { .. })))
+        }
+        _ => false,
+    });
+
     let request_builder = HttpRequest::builder()
         .method(Method::POST)
         .uri(COPILOT_CHAT_COMPLETION_URL)
@@ -594,7 +603,7 @@ async fn stream_completion(
         .header("Authorization", format!("Bearer {}", api_key))
         .header("Content-Type", "application/json")
         .header("Copilot-Integration-Id", "vscode-chat")
-        .header("Copilot-Vision-Request", "true");
+        .header("Copilot-Vision-Request", is_vision_request.to_string());
 
     let is_streaming = request.stream;