1# Gemini 3 Reasoning Support for Copilot Chat Completions API
2
3## Problem Statement
4
5Gemini 3 models (like `gemini-3-pro-preview`) fail when using tool calls through Copilot with the error:
6
7```
8Unable to submit request because function call `default_api:list_directory` in the 2. content block is missing a `thought_signature`.
9```
10
11The error occurs AFTER the first tool call is executed and we send back the tool results with conversation history. The model requires that we preserve and send back its "reasoning" data.
12
13## Background
14
15### What is `reasoning_opaque`?
16
17When Gemini 3 models perform reasoning before making a tool call, they generate reasoning data that includes:
18- `reasoning_text` - Human-readable reasoning content (optional)
19- `reasoning_opaque` - An encrypted/opaque token that must be preserved and sent back
20
21This is similar to how Anthropic models have "thinking" blocks with signatures that must be preserved.
22
23### API Flow
24
251. **User sends prompt** → Model receives request with tools
262. **Model responds with tool call** → Response includes `reasoning_opaque` in the delta
273. **We execute the tool** → Get the result
284. **We send back conversation history** → **MUST include the `reasoning_opaque`** from step 2
295. **Model continues** → Uses the preserved reasoning context
30
31## What Copilot Sends Us (Response Structure)
32
33From actual Copilot streaming responses with Gemini 3:
34
35```json
36{
37 "choices": [{
38 "index": 0,
39 "finish_reason": "tool_calls",
40 "delta": {
41 "content": null,
42 "role": "assistant",
43 "tool_calls": [{
44 "index": 0,
45 "id": "call_MHxRUnpJbnN2SHV2bFNJZnc3bng",
46 "function": {
47 "name": "list_directory",
48 "arguments": "{\"path\":\"deleteme\"}"
49 }
50 }],
51 "reasoning_opaque": "XLn4be0oRXKamQWgyEcgBYpDximdbf/J/dcDmWIhGjZMFaQvOOmSXTqY/zfnRtDCFmZfvsn4W1AG..."
52 }
53 }]
54}
55```
56
57Key observations:
58- `reasoning_opaque` is at the **delta/message level**, not inside individual tool calls
59- The tool calls themselves do NOT have a `thought_signature` field
60- There may also be `reasoning_text` with human-readable reasoning content
61
62### Important: Message Merging Requirement
63
64Looking at the CodeCompanion implementation (PR #2419), there's a critical insight:
65
66When the model sends reasoning data and then tool calls, they may come as **separate messages** that need to be **merged** into a single message when sending back:
67
68```lua
69-- Check if next message is also from LLM and has tool_calls but no content
70-- This indicates tool calls that should be merged with the previous message
71if i < #result.messages
72 and result.messages[i + 1].role == current.role
73 and result.messages[i + 1].tool_calls
74 and not result.messages[i + 1].content
75then
76 -- Merge tool_calls from next message into current
77 current.tool_calls = result.messages[i + 1].tool_calls
78 i = i + 1 -- Skip the next message since we merged it
79end
80```
81
82## What We Must Send Back (Request Structure)
83
84Based on the CodeCompanion implementation, when sending back the conversation history, the assistant message with tool calls should look like:
85
86```json
87{
88 "role": "assistant",
89 "content": "LLM's response here",
90 "reasoning_text": "Some reasoning here",
91 "reasoning_opaque": "XLn4be0oRXKamQWgyEcgBYpDximdbf...",
92 "tool_calls": [{
93 "id": "call_MHxRUnpJbnN2SHV2bFNJZnc3bng",
94 "type": "function",
95 "function": {
96 "name": "list_directory",
97 "arguments": "{\"path\":\"deleteme\"}"
98 }
99 }]
100}
101```
102
103Key points:
104- `reasoning_opaque` goes at the **message level** (same level as `role`, `content`, `tool_calls`)
105- `reasoning_text` may also be included at the message level
106- `content` can be `null` if there's no text content
107- The `function` object does NOT contain `thought_signature`
108
109## Implementation Plan
110
111### Step 1: Update Response Structures
112
113In `crates/copilot/src/copilot_chat.rs`, add fields to capture reasoning data:
114
115**Update `ResponseDelta`:**
116```rust
117#[derive(Debug, Serialize, Deserialize)]
118pub struct ResponseDelta {
119 pub content: Option<String>,
120 pub role: Option<Role>,
121 #[serde(default)]
122 pub tool_calls: Vec<ToolCallChunk>,
123 // Add these fields:
124 pub reasoning_opaque: Option<String>,
125 pub reasoning_text: Option<String>,
126}
127```
128
129### Step 2: Update Request Structures
130
131**Update `ChatMessage::Assistant`:**
132```rust
133pub enum ChatMessage {
134 Assistant {
135 content: Option<ChatMessageContent>, // Changed to Option for null support
136 #[serde(default, skip_serializing_if = "Vec::is_empty")]
137 tool_calls: Vec<ToolCall>,
138 // Add these fields:
139 #[serde(skip_serializing_if = "Option::is_none")]
140 reasoning_opaque: Option<String>,
141 #[serde(skip_serializing_if = "Option::is_none")]
142 reasoning_text: Option<String>,
143 },
144 // ... other variants
145}
146```
147
148**Important:** The `content` field should be `Option<ChatMessageContent>` so it can serialize to `null` instead of `[]` (empty array) when there's no text content.
149
150### Step 3: Update Internal Event/Message Structures
151
152We need to propagate reasoning data through our internal structures.
153
154**In `crates/language_model/src/language_model.rs`**, `LanguageModelToolUse` already has:
155```rust
156pub struct LanguageModelToolUse {
157 pub id: LanguageModelToolUseId,
158 pub name: Arc<str>,
159 pub raw_input: String,
160 pub input: serde_json::Value,
161 pub is_input_complete: bool,
162 pub thought_signature: Option<String>, // We can repurpose this
163}
164```
165
166However, since reasoning is **message-level** not **tool-level**, we may need a different approach. Consider:
167
1681. Store `reasoning_opaque` and `reasoning_text` in `LanguageModelRequestMessage.reasoning_details` (which already exists as `Option<serde_json::Value>`)
1692. Or create a new dedicated field
170
171**In `crates/language_model/src/request.rs`:**
172```rust
173pub struct LanguageModelRequestMessage {
174 pub role: Role,
175 pub content: Vec<MessageContent>,
176 pub cache: bool,
177 // Use this existing field, or add new specific fields:
178 pub reasoning_details: Option<serde_json::Value>,
179}
180```
181
182### Step 4: Capture Reasoning from Responses
183
184In `crates/language_models/src/provider/copilot_chat.rs`, in the `map_to_language_model_completion_events` function:
185
1861. Capture `reasoning_opaque` and `reasoning_text` from the delta
1872. Store them so they can be associated with tool calls
1883. When emitting `LanguageModelCompletionEvent::ToolUse`, include the reasoning data
189
190```rust
191// Pseudocode for the mapper:
192struct State {
193 events: Pin<Box<dyn Send + Stream<Item = Result<ResponseEvent>>>>,
194 tool_calls_by_index: HashMap<usize, RawToolCall>,
195 reasoning_opaque: Option<String>, // Add this
196 reasoning_text: Option<String>, // Add this
197}
198
199// When processing delta:
200if let Some(opaque) = delta.reasoning_opaque {
201 state.reasoning_opaque = Some(opaque);
202}
203if let Some(text) = delta.reasoning_text {
204 state.reasoning_text = Some(text);
205}
206
207// When emitting tool use events, attach the reasoning
208```
209
210### Step 5: Send Reasoning Back in Requests
211
212In `crates/language_models/src/provider/copilot_chat.rs`, in the `into_copilot_chat` function:
213
214When building `ChatMessage::Assistant` for messages that have tool calls:
215
216```rust
217messages.push(ChatMessage::Assistant {
218 content: if text_content.is_empty() {
219 None // Serializes to null, not []
220 } else {
221 Some(text_content.into())
222 },
223 tool_calls,
224 reasoning_opaque: /* get from message's reasoning_details or tool_use */,
225 reasoning_text: /* get from message's reasoning_details or tool_use */,
226});
227```
228
229### Step 6: Handle Message Merging (If Needed)
230
231If Copilot sends reasoning and tool calls as separate streaming events that result in separate internal messages, we may need to merge them when constructing the request.
232
233Look at the message construction logic and ensure that:
234- If an assistant message has reasoning but no tool calls, AND
235- The next message is also assistant with tool calls but no content
236- Then merge them into a single message
237
238## Files to Modify
239
2401. **`crates/copilot/src/copilot_chat.rs`**
241 - Add `reasoning_opaque` and `reasoning_text` to `ResponseDelta`
242 - Add `reasoning_opaque` and `reasoning_text` to `ChatMessage::Assistant`
243 - Change `content` in `ChatMessage::Assistant` to `Option<ChatMessageContent>`
244 - Update any pattern matches that break due to the Option change
245
2462. **`crates/language_models/src/provider/copilot_chat.rs`**
247 - Update `map_to_language_model_completion_events` to capture reasoning
248 - Update `into_copilot_chat` to include reasoning in requests
249 - Possibly add message merging logic
250
2513. **`crates/language_model/src/request.rs`** (maybe)
252 - Decide how to store reasoning data in `LanguageModelRequestMessage`
253 - Could use existing `reasoning_details` field or add new fields
254
2554. **`crates/language_model/src/language_model.rs`** (maybe)
256 - May need to add a new event type for reasoning, OR
257 - Ensure reasoning can be attached to tool use events
258
259## Testing
260
2611. Test with Gemini 3 Pro Preview through Copilot
2622. Trigger a tool call (e.g., ask "what files are in this directory?")
2633. Verify the first request succeeds and returns with `reasoning_opaque`
2644. Verify the second request (with tool results) includes the `reasoning_opaque`
2655. Verify the model successfully continues and doesn't return a 400 error
266
267## Debug Logging Recommendations
268
269Add `eprintln!` statements to trace:
2701. When `reasoning_opaque` is received from Copilot
2712. When `reasoning_opaque` is stored/attached to tool use
2723. The full JSON of requests being sent (to verify structure)
2734. The full JSON of responses received
274
275## References
276
277- [CodeCompanion PR #2419](https://github.com/olimorris/codecompanion.nvim/pull/2419) - Working implementation in Lua
278- [Original Zed Issue #43024](https://github.com/zed-industries/zed/issues/43024)
279- [Google Thought Signatures Documentation](https://ai.google.dev/gemini-api/docs/thinking#signatures)
280
281## Key Insight from CodeCompanion
282
283The CodeCompanion implementation shows the exact structure:
284
285**Receiving:**
286```lua
287-- In parse_message_meta function:
288if extra.reasoning_text then
289 data.output.reasoning = data.output.reasoning or {}
290 data.output.reasoning.content = extra.reasoning_text
291end
292if extra.reasoning_opaque then
293 data.output.reasoning = data.output.reasoning or {}
294 data.output.reasoning.opaque = extra.reasoning_opaque
295end
296```
297
298**Sending back:**
299```lua
300-- In form_messages function:
301if current.reasoning then
302 if current.reasoning.content then
303 current.reasoning_text = current.reasoning.content
304 end
305 if current.reasoning.opaque then
306 current.reasoning_opaque = current.reasoning.opaque
307 end
308 current.reasoning = nil
309end
310```
311
312The key is that `reasoning_text` and `reasoning_opaque` are **top-level fields** on the assistant message when sent back to the API.