1# Thought Signatures Implementation for Gemini 3 Models
  2
  3## Problem Statement
  4
  5Gemini 3 models (like `gemini-3-pro-preview`) fail when using tool calls through OpenRouter and Copilot with the error:
  6
  7```
  8Unable to submit request because function call `default_api:list_directory` in the 2. content block is missing a `thought_signature`.
  9```
 10
 11The error occurs AFTER the first tool call is executed and we send back the tool results with conversation history.
 12
 13## Background
 14
 15### What are Thought Signatures?
 16
 17Thought signatures are a validation mechanism used by Gemini reasoning models. When the model performs "thinking" (reasoning) before making a tool call, it generates a cryptographic signature of that reasoning. This signature must be preserved and sent back in subsequent requests to maintain the integrity of the conversation flow.
 18
 19### API Formats Involved
 20
 21There are three different API formats in play:
 22
 231. **Google AI Native API** - Uses `Part` objects including `FunctionCallPart` with a `thought_signature` field
 242. **OpenRouter/Copilot Chat Completions API** - OpenAI-compatible format with `tool_calls` array
 253. **Copilot Responses API** - A separate format with streaming `reasoning_details`
 26
 27## Current Architecture
 28
 29### Data Flow
 30
 311. **Model Response** → Contains tool calls with reasoning
 322. **Zed Event Stream** → Emits `LanguageModelCompletionEvent::ToolUse` events
 333. **Agent** → Collects events and constructs `LanguageModelRequestMessage` objects
 344. **Provider** → Converts messages back to provider-specific format
 355. **API Request** → Sent back to the provider with conversation history
 36
 37### Key Data Structures
 38
 39```rust
 40// Core message structure
 41pub struct LanguageModelRequestMessage {
 42    pub role: Role,
 43    pub content: Vec<MessageContent>,
 44    pub cache: bool,
 45    pub reasoning_details: Option<serde_json::Value>, // Added for thought signatures
 46}
 47
 48// Tool use structure
 49pub struct LanguageModelToolUse {
 50    pub id: LanguageModelToolUseId,
 51    pub name: Arc<str>,
 52    pub raw_input: String,
 53    pub input: serde_json::Value,
 54    pub is_input_complete: bool,
 55    pub thought_signature: Option<String>, // NOT USED - wrong approach
 56}
 57```
 58
 59## What We Tried (That Didn't Work)
 60
 61### Attempt 1: `thought_signature` as field on ToolCall
 62We added `thought_signature` as a field on the `ToolCall` structure itself.
 63
 64**Result:** 400 Bad Request - OpenRouter/Copilot don't support this field at the ToolCall level.
 65
 66### Attempt 2: `thought_signature` inside `function` object
 67We moved `thought_signature` inside the `function` object of the tool call.
 68
 69```json
 70{
 71  "function": {
 72    "name": "...",
 73    "arguments": "...",
 74    "thought_signature": "..."
 75  }
 76}
 77```
 78
 79**Result:** 400 Bad Request - Still rejected.
 80
 81### Attempt 3: Using camelCase `thoughtSignature`
 82Tried both snake_case and camelCase variants.
 83
 84**Result:** No difference, still rejected.
 85
 86## The Correct Approach (From OpenRouter Documentation)
 87
 88According to [OpenRouter's documentation](https://openrouter.ai/docs/use-cases/reasoning-tokens#preserving-reasoning-blocks):
 89
 90### Key Insight: `reasoning_details` is a message-level array
 91
 92The thought signature is NOT a property of individual tool calls. Instead, it's part of a `reasoning_details` array that belongs to the entire assistant message:
 93
 94```json
 95{
 96  "role": "assistant",
 97  "content": null,
 98  "tool_calls": [
 99    {
100      "id": "call_123",
101      "type": "function",
102      "function": {
103        "name": "list_directory",
104        "arguments": "{...}"
105      }
106    }
107  ],
108  "reasoning_details": [
109    {
110      "type": "reasoning.text",
111      "text": "Let me think through this step by step...",
112      "signature": "sha256:abc123...",
113      "id": "reasoning-text-1",
114      "format": "anthropic-claude-v1",
115      "index": 0
116    }
117  ]
118}
119```
120
121### `reasoning_details` Structure
122
123The array can contain three types of objects:
124
1251. **reasoning.summary** - High-level summary of reasoning
1262. **reasoning.encrypted** - Encrypted/redacted reasoning data
1273. **reasoning.text** - Raw text reasoning with optional signature
128
129Each object has:
130- `type`: One of the three types above
131- `id`: Unique identifier
132- `format`: Format version (e.g., "anthropic-claude-v1", "openai-responses-v1")
133- `index`: Sequential index
134- `signature`: (for reasoning.text) The cryptographic signature we need to preserve
135
136## What We've Implemented So Far
137
138### 1. Added `reasoning_details` field to core structures
139
140✅ `LanguageModelRequestMessage` now has `reasoning_details: Option<serde_json::Value>`
141
142### 2. Added `reasoning_details` to OpenRouter structs
143
144✅ `RequestMessage::Assistant` has `reasoning_details` field
145✅ `ResponseMessageDelta` has `reasoning_details` field
146
147### 3. Updated `into_open_router` to send `reasoning_details`
148
149✅ When building requests, we now attach `reasoning_details` from the message to the Assistant message
150
151### 4. Added mapper to capture `reasoning_details` from responses
152
153✅ `OpenRouterEventMapper` now has a `reasoning_details` field
154✅ We capture it from `choice.delta.reasoning_details`
155
156### 5. Added debugging
157
158✅ `eprintln!` statements in both OpenRouter and Copilot to log requests and responses
159
160## What's Still Missing
161
162### The Critical Gap: Event → Message Flow
163
164The problem is in how events become messages. Our current flow:
165
1661. ✅ We capture `reasoning_details` from the API response
1672. ❌ We store it in `OpenRouterEventMapper` but never emit it
1683. ❌ The agent constructs messages from events, but has no way to get the `reasoning_details`
1694. ❌ When sending the next request, `message.reasoning_details` is `None`
170
171### What We Need to Do
172
173#### Option A: Add a new event type
174
175Add a `LanguageModelCompletionEvent::ReasoningDetails(serde_json::Value)` event that gets emitted when we receive reasoning details. The agent would need to:
176
1771. Collect this event along with tool use events
1782. When constructing the assistant message, attach the reasoning_details to it
179
180#### Option B: Store reasoning_details with tool use events
181
182Modify the flow so that when we emit tool use events, we somehow associate the `reasoning_details` with them. This is tricky because:
183- `reasoning_details` is per-message, not per-tool
184- Multiple tools can be in one message
185- We emit events one at a time
186
187#### Option C: Store at a higher level
188
189Have the agent or provider layer handle this separately from the event stream. For example:
190- The provider keeps track of reasoning_details for messages it processes
191- When building the next request, it looks up the reasoning_details for assistant messages that had tool calls
192
193## Current Status
194
195### What Works
196- ✅ Code compiles
197- ✅ `reasoning_details` field exists throughout the stack
198- ✅ We capture `reasoning_details` from responses
199- ✅ We send `reasoning_details` in requests (if present)
200
201### What Doesn't Work
202- ❌ `reasoning_details` never makes it from the response to the request
203- ❌ The error still occurs because we're sending `null` for `reasoning_details`
204
205### Evidence from Error Message
206
207The error says:
208```
209function call `default_api:list_directory` in the 2. content block is missing a `thought_signature`
210```
211
212This means:
2131. We're successfully making the first request (works)
2142. The model responds with tool calls including reasoning_details (works)
2153. We execute the tools (works)
2164. We send back the conversation history (works)
2175. BUT the assistant message in that history is missing the reasoning_details (broken)
2186. Google/Vertex validates the message and rejects it (error)
219
220## Next Steps
221
2221. **Choose an approach** - Decide between Option A, B, or C above
2232. **Implement the data flow** - Ensure `reasoning_details` flows from response → events → message → request
2243. **Test with debugging** - Use the `eprintln!` statements to verify:
225   - That we receive `reasoning_details` in the response
226   - That we include it in the next request
2274. **Apply to Copilot** - Once working for OpenRouter, apply the same pattern to Copilot
2285. **Handle edge cases**:
229   - What if there are multiple tool calls in one message?
230   - What if reasoning_details is empty/null?
231   - What about other providers (Anthropic, etc.)?
232
233## Files Modified
234
235- `crates/language_model/src/request.rs` - Added `reasoning_details` to `LanguageModelRequestMessage`
236- `crates/open_router/src/open_router.rs` - Added `reasoning_details` to request/response structs
237- `crates/language_models/src/provider/open_router.rs` - Added capture and send logic
238- `crates/copilot/src/copilot_responses.rs` - Already had `thought_signature` support
239- Various test files - Added `reasoning_details: None` to fix compilation
240
241## SOLUTION: Copilot Chat Completions API Implementation
242
243### Discovery: Gemini 3 Uses Chat Completions API, Not Responses API
244
245Initial plan assumed routing Gemini 3 to Responses API would work, but testing revealed:
246- **Gemini 3 models do NOT support the Responses API** through Copilot
247- Error: `{"error":{"message":"model gemini-3-pro-preview is not supported via Responses API.","code":"unsupported_api_for_model"}}`
248- Gemini 3 ONLY supports the Chat Completions API
249
250### Key Finding: `reasoning_opaque` Location in JSON
251
252Through detailed logging and JSON inspection, discovered Copilot sends thought signatures in Chat Completions API:
253- Field name: **`reasoning_opaque`** (not `thought_signature`)
254- Location: **At the `delta` level**, NOT at the `tool_calls` level!
255
256JSON structure from Copilot response:
257```json
258{
259  "choices": [{
260    "delta": {
261      "role": "assistant",
262      "tool_calls": [{
263        "function": {"arguments": "...", "name": "list_directory"},
264        "id": "call_...",
265        "index": 0,
266        "type": "function"
267      }],
268      "reasoning_opaque": "sPsUMpfe1YZXLkbc0TNW/mJLT..."  // <-- HERE!
269    }
270  }]
271}
272```
273
274### Implementation Status
275
276#### ✅ Completed Changes
277
2781. **Added `reasoning_opaque` field to `ResponseDelta`** (`crates/copilot/src/copilot_chat.rs`)
279   ```rust
280   pub struct ResponseDelta {
281       pub content: Option<String>,
282       pub role: Option<Role>,
283       pub tool_calls: Vec<ToolCallChunk>,
284       pub reasoning_opaque: Option<String>,  // Added this
285   }
286   ```
287
2882. **Added `thought_signature` fields to Chat Completions structures** (`crates/copilot/src/copilot_chat.rs`)
289   - `FunctionContent` now has `thought_signature: Option<String>`
290   - `FunctionChunk` now has `thought_signature: Option<String>`
291
2923. **Updated mapper to capture `reasoning_opaque` from delta** (`crates/language_models/src/provider/copilot_chat.rs`)
293   - Captures `reasoning_opaque` from `delta.reasoning_opaque`
294   - Applies it to all tool calls in that delta
295   - Stores in `thought_signature` field of accumulated tool call
296
2974. **Verified thought signature is being sent back**
298   - Logs show: `📤 Chat Completions: Sending tool call list_directory with thought_signature: Some("sPsUMpfe...")`
299   - Signature is being included in subsequent requests
300
301#### ❌ Current Issue: Still Getting 400 Error
302
303Despite successfully capturing and sending back the thought signature, Copilot still returns:
304```
305400 Bad Request {"error":{"message":"invalid request body","code":"invalid_request_body"}}
306```
307
308This happens on the SECOND request (after tool execution), when sending conversation history back.
309
310### Debug Logging Added
311
312Current logging shows the full flow:
313- `📥 Chat Completions: Received reasoning_opaque (length: XXX)` - Successfully captured
314- `🔍 Tool call chunk: index=..., id=..., has_function=...` - Delta processing
315- `📤 Chat Completions: Emitting ToolUse for ... with thought_signature: Some(...)` - Event emission
316- `📤 Chat Completions: Sending tool call ... with thought_signature: Some(...)` - Sending back
317- `📤 Chat Completions Request JSON: {...}` - Full request being sent
318- `📥 Chat Completions Response Event: {...}` - Full response received
319
320### Potential Issues to Investigate
321
3221. **Field name mismatch on send**: We're sending `thought_signature` but should we send `reasoning_opaque`?
323   - We added `thought_signature` to `FunctionContent` 
324   - But Copilot might expect `reasoning_opaque` in the request just like it sends it
325
3262. **Serialization issue**: Check if serde is properly serializing the field
327   - Added `#[serde(skip_serializing_if = "Option::is_none")]` - might be skipping it?
328   - Should verify field appears in actual JSON being sent
329
3303. **Location issue**: Even when sending back, should `reasoning_opaque` be at delta level?
331   - Currently putting it in `function.thought_signature`
332   - Might need to be at a different level in the request structure
333
3344. **Format validation**: The signature is a base64-encoded string ~1464 characters
335   - Copilot might be validating the signature format/content
336   - Could be rejecting it if it's malformed or in wrong structure
337
338### Next Steps to Debug
339
3401. **Check actual JSON being sent**: Look at the `📤 Chat Completions Request JSON` logs
341   - Search for `thought_signature` in the JSON
342   - Verify it's actually in the serialized output (not skipped)
343   - Check its exact location in the JSON structure
344
3452. **Try renaming field**: Change `thought_signature` to `reasoning_opaque` in request structures
346   - In `FunctionContent` struct
347   - In `FunctionChunk` struct
348   - See if Copilot expects same field name in both directions
349
3503. **Compare request format to response format**: 
351   - Response has `reasoning_opaque` at delta level
352   - Request might need it at function level OR delta level
353   - May need to restructure where we put it
354
3554. **Test with tool choice parameter**: Some APIs are sensitive to request structure
356   - Try with/without `tool_choice` parameter
357   - Try with minimal conversation history
358
3595. **Check Copilot API documentation**: 
360   - Search for official docs on `reasoning_opaque` handling
361   - Look for examples of tool calls with reasoning/thinking in Copilot API
362
363### Files Modified
364
365- ✅ `crates/copilot/src/copilot_chat.rs` - Added `reasoning_opaque` to `ResponseDelta`, `thought_signature` to function structs
366- ✅ `crates/language_models/src/provider/copilot_chat.rs` - Capture and send logic with debug logging
367- ⏳ Still need to verify serialization and field naming
368
369### References
370
371- [OpenRouter Reasoning Tokens Documentation](https://openrouter.ai/docs/use-cases/reasoning-tokens)
372- [Google Thought Signatures Documentation](https://ai.google.dev/gemini-api/docs/thinking#signatures)
373- [Original Issue #43024](https://github.com/zed-industries/zed/issues/43024)
374## ✅ FINAL FIX (2025-01-21)
375
376### The Critical Issues Found
377
378After testing, we discovered TWO problems:
379
3801. **Wrong Location**: We were sending `thought_signature` inside the `function` object, but Copilot expects `reasoning_opaque` at the **message level**
3812. **Wrong Content Format**: We were sending `"content": []` (empty array), but Copilot expects `"content": null` when there are tool calls
382
383### The Solution
384
385#### Issue 1: Message-Level Field
386- **Added** `reasoning_opaque: Option<String>` to `ChatMessage::Assistant`
387- **Removed** `thought_signature` from `FunctionContent` (it doesn't belong there)
388- **Updated** request builder to collect signature from first tool use and pass at message level
389
390#### Issue 2: Null vs Empty Array
391- **Changed** `content` field type from `ChatMessageContent` to `Option<ChatMessageContent>`
392- **Set** `content: None` when we have tool calls and no text (serializes to `null`)
393- **Set** `content: Some(text)` when we have text content
394
395### Correct Request Format
396
397```json
398{
399  "role": "assistant",
400  "content": null,  // ✅ Explicit null, not []
401  "tool_calls": [{
402    "id": "call_...",
403    "type": "function",
404    "function": {
405      "name": "list_directory",
406      "arguments": "{\"path\":\"deleteme\"}"
407      // NO thought_signature here!
408    }
409  }],
410  "reasoning_opaque": "XLn4be0..."  // ✅ At message level!
411}
412```
413
414### Files Modified in Final Fix
415
416- `zed/crates/copilot/src/copilot_chat.rs`:
417  - Added `reasoning_opaque` to `ChatMessage::Assistant`
418  - Changed `content` to `Option<ChatMessageContent>`
419  - Fixed vision detection pattern match
420- `zed/crates/language_models/src/provider/copilot_chat.rs`:
421  - Collect `reasoning_opaque` from first tool use
422  - Pass to Assistant message, not function
423  - Set `content: None` for tool-only messages
424  - Removed function-level thought_signature handling
425
426### Compilation Status
427
428✅ All packages compile successfully
429
430Ready for testing!