1# Configuring the Assistant
2
3## Providers {#providers}
4
5The following providers are supported:
6
7- [Zed AI (Configured by default when signed in)](#zed-ai)
8- [Anthropic](#anthropic)
9- [GitHub Copilot Chat](#github-copilot-chat) [^1]
10- [Google AI](#google-ai) [^1]
11- [Ollama](#ollama)
12- [OpenAI](#openai)
13
14To configure different providers, run `assistant: show configuration` in the command palette, or click on the hamburger menu at the top-right of the assistant panel and select "Configure".
15
16[^1]: This provider does not support the [`/workflow`](./commands#workflow-not-generally-available) command.
17
18To further customize providers, you can use `settings.json` to do that as follows:
19
20- [Configuring endpoints](#custom-endpoint)
21- [Configuring timeouts](#provider-timeout)
22- [Configuring default model](#default-model)
23
24### Zed AI {#zed-ai}
25
26A hosted service providing convenient and performant support for AI-enabled coding in Zed, powered by Anthropic's Claude 3.5 Sonnet and accessible just by signing in.
27
28### Anthropic {#anthropic}
29
30You can use Claude 3.5 Sonnet via [Zed AI](#zed-ai) for free. To use other Anthropic models you will need to configure it by providing your own API key.
31
321. Sign up for Anthropic and [create an API key](https://console.anthropic.com/settings/keys)
332. Make sure that your Anthropic account has credits
343. Open the configuration view (`assistant: show configuration`) and navigate to the Anthropic section
354. Enter your Anthropic API key
36
37Even if you pay for Claude Pro, you will still have to [pay for additional credits](https://console.anthropic.com/settings/plans) to use it via the API.
38
39Zed will also use the `ANTHROPIC_API_KEY` environment variable if it's defined.
40
41#### Anthropic Custom Models {#anthropic-custom-models}
42
43You can add custom models to the Anthropic provider by adding the following to your Zed `settings.json`:
44
45```json
46{
47 "language_models": {
48 "anthropic": {
49 "available_models": [
50 {
51 "name": "some-model",
52 "display_name": "some-model",
53 "max_tokens": 128000,
54 "max_output_tokens": 2560,
55 "cache_configuration": {
56 "max_cache_anchors": 10,
57 "min_total_token": 10000,
58 "should_speculate": false
59 },
60 "tool_override": "some-model-that-supports-toolcalling"
61 }
62 ]
63 }
64 }
65}
66```
67
68Custom models will be listed in the model dropdown in the assistant panel.
69
70### GitHub Copilot Chat {#github-copilot-chat}
71
72You can use GitHub Copilot chat with the Zed assistant by choosing it via the model dropdown in the assistant panel.
73
74### Google AI {#google-ai}
75
76You can use Gemini 1.5 Pro/Flash with the Zed assistant by choosing it via the model dropdown in the assistant panel.
77
781. Go the Google AI Studio site and [create an API key](https://aistudio.google.com/app/apikey).
792. Open the configuration view (`assistant: show configuration`) and navigate to the Google AI section
803. Enter your Google AI API key and press enter.
81
82The Google AI API key will be saved in your keychain.
83
84Zed will also use the `GOOGLE_AI_API_KEY` environment variable if it's defined.
85
86#### Google AI custom models {#google-ai-custom-models}
87
88By default Zed will use `stable` versions of models, but you can use specific versions of models, including [experimental models](https://ai.google.dev/gemini-api/docs/models/experimental-models) with the Google AI provider by adding the following to your Zed `settings.json`:
89
90```json
91{
92 "language_models": {
93 "google": {
94 "available_models": [
95 {
96 "name": "gemini-1.5-flash-latest",
97 "display_name": "Gemini 1.5 Flash (Latest)",
98 "max_tokens": 1000000
99 }
100 ]
101 }
102 }
103}
104```
105
106Custom models will be listed in the model dropdown in the assistant panel.
107
108### Ollama {#ollama}
109
110Download and install Ollama from [ollama.com/download](https://ollama.com/download) (Linux or macOS) and ensure it's running with `ollama --version`.
111
1121. Download one of the [available models](https://ollama.com/models), for example, for `mistral`:
113
114 ```sh
115 ollama pull mistral
116 ```
117
1182. Make sure that the Ollama server is running. You can start it either via running Ollama.app (MacOS) or launching:
119
120 ```sh
121 ollama serve
122 ```
123
1243. In the assistant panel, select one of the Ollama models using the model dropdown.
125
1264. (Optional) Specify a [custom api_url](#custom-endpoint) or [custom `low_speed_timeout_in_seconds`](#provider-timeout) if required.
127
128#### Ollama Context Length {#ollama-context}
129
130Zed has pre-configured maximum context lengths (`max_tokens`) to match the capabilities of common models. Zed API requests to Ollama include this as `num_ctx` parameter, but the default values do not exceed `16384` so users with ~16GB of ram are able to use most models out of the box. See [get_max_tokens in ollama.rs](https://github.com/zed-industries/zed/blob/main/crates/ollama/src/ollama.rs) for a complete set of defaults.
131
132**Note**: Tokens counts displayed in the assistant panel are only estimates and will differ from the models native tokenizer.
133
134Depending on your hardware or use-case you may wish to limit or increase the context length for a specific model via settings.json:
135
136```json
137{
138 "language_models": {
139 "ollama": {
140 "low_speed_timeout_in_seconds": 120,
141 "available_models": [
142 {
143 "provider": "ollama",
144 "name": "mistral:latest",
145 "max_tokens": 32768
146 }
147 ]
148 }
149 }
150}
151```
152
153If you specify a context length that is too large for your hardware, Ollama will log an error. You can watch these logs by running: `tail -f ~/.ollama/logs/ollama.log` (MacOS) or `journalctl -u ollama -f` (Linux). Depending on the memory available on your machine, you may need to adjust the context length to a smaller value.
154
155### OpenAI {#openai}
156
1571. Visit the OpenAI platform and [create an API key](https://platform.openai.com/account/api-keys)
1582. Make sure that your OpenAI account has credits
1593. Open the configuration view (`assistant: show configuration`) and navigate to the OpenAI section
1604. Enter your OpenAI API key
161
162The OpenAI API key will be saved in your keychain.
163
164Zed will also use the `OPENAI_API_KEY` environment variable if it's defined.
165
166#### OpenAI Custom Models {#openai-custom-models}
167
168The Zed Assistant comes pre-configured to use the latest version for common models (GPT-3.5 Turbo, GPT-4, GPT-4 Turbo, GPT-4o, GPT-4o mini). If you wish to use alternate models, perhaps a preview release or a dated model release, you can do so by adding the following to your Zed `settings.json`:
169
170```json
171{
172 "language_models": {
173 "openai": {
174 "available_models": [
175 {
176 "provider": "openai",
177 "name": "gpt-4o-2024-08-06",
178 "max_tokens": 128000
179 }
180 ]
181 }
182 }
183}
184```
185
186You must provide the model's Context Window in the `max_tokens` parameter, this can be found [OpenAI Model Docs](https://platform.openai.com/docs/models). Custom models will be listed in the model dropdown in the assistant panel.
187
188### Advanced configuration {#advanced-configuration}
189
190#### Example Configuration
191
192```json
193{
194 "assistant": {
195 "enabled": true,
196 "default_model": {
197 "provider": "zed.dev",
198 "model": "claude-3-5-sonnet"
199 },
200 "version": "2",
201 "button": true,
202 "default_width": 480,
203 "dock": "right"
204 }
205}
206```
207
208#### Custom endpoints {#custom-endpoint}
209
210You can use a custom API endpoint for different providers, as long as it's compatible with the providers API structure.
211
212To do so, add the following to your Zed `settings.json`:
213
214```json
215{
216 "language_models": {
217 "some-provider": {
218 "api_url": "http://localhost:11434"
219 }
220 }
221}
222```
223
224Where `some-provider` can be any of the following values: `anthropic`, `google`, `ollama`, `openai`.
225
226#### Custom timeout {#provider-timeout}
227
228You can customize the timeout that's used for LLM requests, by adding the following to your Zed `settings.json`:
229
230```json
231{
232 "language_models": {
233 "some-provider": {
234 "low_speed_timeout_in_seconds": 10
235 }
236 }
237}
238```
239
240Where `some-provider` can be any of the following values: `anthropic`, `copilot_chat`, `google`, `ollama`, `openai`.
241
242#### Configuring the default model {#default-model}
243
244The default model can be set via the model dropdown in the assistant panel's top-right corner. Selecting a model saves it as the default.
245You can also manually edit the `default_model` object in your settings:
246
247```json
248{
249 "assistant": {
250 "version": "2",
251 "default_model": {
252 "provider": "zed.dev",
253 "model": "claude-3-5-sonnet"
254 }
255 }
256}
257```
258
259#### Common Panel Settings
260
261| key | type | default | description |
262| -------------- | ------- | ------- | ------------------------------------------------------------------------------------- |
263| enabled | boolean | true | Setting this to `false` will completely disable the assistant |
264| button | boolean | true | Show the assistant icon in the status bar |
265| dock | string | "right" | The default dock position for the assistant panel. Can be ["left", "right", "bottom"] |
266| default_height | string | null | The pixel height of the assistant panel when docked to the bottom |
267| default_width | string | null | The pixel width of the assistant panel when docked to the left or right |