PLAN.md

  1# Staged Docs Releases Plan
  2
  3## Background & Current State
  4
  5| Component | Status | Location |
  6|---|---|---|
  7| Docs build (CI check) | **xtask-generated** (in `run_tests.rs`) | `check_docs()` at L514–560 |
  8| Docs deploy workflow | **Hand-written YAML** | `.github/workflows/deploy_cloudflare.yml` |
  9| Composite build action | **Hand-written YAML** | `.github/actions/build_docs/action.yml` |
 10| Docs-proxy worker | **Runtime JS** | `.cloudflare/docs-proxy/src/worker.js` |
 11| book.toml | **Config** | `docs/book.toml` (site-url = `/docs/`) |
 12
 13There's already a TODO in the codebase acknowledging the duplication:
 14
 15```
 16// todo(ci): un-inline build_docs/action.yml here
 17```
 18(`tooling/xtask/src/tasks/workflows/run_tests.rs` L549)
 19
 20---
 21
 22## Step 1: Move Docs Build & Deploy into xtask Workflow Generation
 23
 24### 1.1 Create `deploy_docs.rs` with shared build helpers and deploy workflow
 25
 26Create `tooling/xtask/src/tasks/workflows/deploy_docs.rs`. This module owns all docs build and deploy logic and exports shared helpers used by both the new deploy workflow and `check_docs()` in `run_tests.rs`.
 27
 28**Shared helpers** — export as `pub(crate)`:
 29
 30```rust
 31pub(crate) fn lychee_link_check(dir: &str) -> Step<Use> { ... }
 32pub(crate) fn install_mdbook() -> Step<Use> { ... }
 33pub(crate) fn build_docs_book() -> Step<Run> { ... }
 34```
 35
 36Copy each implementation verbatim from the inline private functions currently inside `check_docs()` in `run_tests.rs`.
 37
 38**Build job** — mirror `.github/actions/build_docs/action.yml` step-for-step, using the patterns from `check_docs()` in `run_tests.rs` as the reference for how shared steps are expressed in xtask. Steps in order:
 39
 401. `steps::checkout_repo()`
 412. A step that runs `cp ./.cargo/collab-config.toml ./.cargo/config.toml` — the deploy workflow uses `collab-config.toml` (not `ci-config.toml`), so this is distinct from `steps::setup_cargo_config`
 423. `steps::cache_rust_dependencies_namespace()`
 434. `steps::install_linux_dependencies` (covers `./script/linux`, `./script/install-mold`, `./script/download-wasi-sdk`)
 445. `steps::script("./script/generate-action-metadata")`
 456. `lychee_link_check("./docs/src/**/*")` — check Markdown links
 467. `install_mdbook()`
 478. `build_docs_book()`
 489. `lychee_link_check("target/deploy/docs")` — check links in generated HTML
 49
 50Set `DOCS_AMPLITUDE_API_KEY` on the build job (currently passed via `env:` to the composite action in `deploy_cloudflare.yml`).
 51
 52### 1.1.1
 53**Deploy job** — copy each step verbatim from the `deploy-docs` job in `deploy_cloudflare.yml`. Steps in order:
 54
 551. Wrangler: `pages deploy target/deploy --project-name=docs`
 562. Wrangler: `r2 object put -f script/install.sh zed-open-source-website-assets/install.sh`
 573. Wrangler: `deploy .cloudflare/docs-proxy/src/worker.js`
 584. Upload Wrangler logs artifact (`always()`, path `~/.config/.wrangler/logs/`, name `wrangler_logs`)
 59
 60Note: steps "Deploy Docs Workers" and "Deploy Install Workers" in `deploy_cloudflare.yml` run the identical `wrangler deploy .cloudflare/docs-proxy/src/worker.js` command — this is a copy-paste bug. Include it only once (step 3 above).
 61
 62Each Wrangler step uses `cloudflare/wrangler-action@da0e0dfe58b7a431659754fdf3f186c529afbe65` with `apiToken` and `accountId` from `vars::CLOUDFLARE_API_TOKEN` and `vars::CLOUDFLARE_ACCOUNT_ID`.
 63
 64**Update `check_docs()`** — once the module exists, replace the inline private `lychee_link_check`, `install_mdbook`, and `build_docs` functions in `run_tests.rs` with calls to `deploy_docs::lychee_link_check`, `deploy_docs::install_mdbook`, and `deploy_docs::build_docs_book`. This resolves the TODO at L549.
 65
 66### 1.2 Register the new workflow
 67
 68In `tooling/xtask/src/tasks/workflows.rs`:
 69- Add `mod deploy_docs;`
 70- Add `WorkflowFile::zed(deploy_docs::deploy_docs)` to the workflows array
 71
 72### 1.3 Add Cloudflare secrets to `vars.rs`
 73
 74Add typed secret references:
 75
 76```rust
 77secret!(CLOUDFLARE_API_TOKEN);
 78secret!(CLOUDFLARE_ACCOUNT_ID);
 79secret!(DOCS_AMPLITUDE_API_KEY);
 80```
 81
 82### 1.4 Verify generated output matches the hand-written workflow
 83
 84- Run `cargo xtask workflows` to generate `.github/workflows/deploy_docs.yml`
 85- Diff the generated file against `.github/workflows/deploy_cloudflare.yml` and `.github/actions/build_docs/action.yml` to confirm all steps, env vars, secrets, triggers, and job dependencies are equivalent
 86- Pay attention to: step ordering, `if:` conditions, artifact names, the Wrangler action hash, and that the duplicate worker deploy step is now deduplicated
 87
 88### 1.5 Ask the user to approve the generated files
 89
 90**Stop and show the generated `.github/workflows/deploy_docs.yml` to the user.** Ask them to confirm the output looks correct before proceeding to delete the hand-written files. Do not continue to step 1.6 without explicit approval.
 91
 92### 1.6 Delete hand-written files
 93
 94- Delete `.github/workflows/deploy_cloudflare.yml` (replaced by the generated workflow)
 95- Delete `.github/actions/build_docs/action.yml` (steps are now inlined in xtask)
 96
 97### 1.7 Final check
 98
 99The CI self-check (`check_xtask_workflows`) will enforce that the generated file stays in sync going forward.
100
101---
102
103## Step 2: Split Deployments into Nightly, Preview, and Stable
104
105### 2.1 Introduce `DocsChannel` and parameterize the build
106
107Add a `DocsChannel` enum to `deploy_docs.rs`:
108
109```rust
110pub(crate) enum DocsChannel {
111    Nightly,  // site-url = "/docs/nightly/", project = "docs-nightly"
112    Preview,  // site-url = "/docs/preview/", project = "docs-preview"
113    Stable,   // site-url = "/docs/",          project = "docs"
114}
115```
116
117mdBook supports overriding `book.toml` values at build time via `MDBOOK_`-prefixed environment variables, using `__` for TOML key nesting. The `site-url` field lives under `[book]`, so setting `MDBOOK_BOOK__SITE_URL=/docs/nightly/` before `mdbook build` overrides it without touching the file. The `book.toml` in the repository remains unchanged at `site-url = "/docs/"`.
118
119The `build_docs_book()` step stays as-is; the channel env var is applied to the build job:
120
121```rust
122fn build_job(channel: DocsChannel, deps: &[&NamedJob]) -> NamedJob {
123    // ...
124    .add_env(("MDBOOK_BOOK__SITE_URL", channel.site_url()))
125}
126```
127
128The deploy step uses `--project-name` matching the channel:
129
130```rust
131fn pages_deploy_step(channel: &DocsChannel) -> Step<Use> {
132    // wrangler: pages deploy target/deploy --project-name=<channel.project_name()>
133}
134```
135
136Export `pub(crate) fn deploy_docs_job(channel: DocsChannel, deps: &[&NamedJob]) -> NamedJob` for use in `release.rs`.
137
138### 2.2 Create `deploy_docs_nightly.rs`
139
140Create `tooling/xtask/src/tasks/workflows/deploy_docs_nightly.rs`. This is the standalone workflow for nightly docs, triggered on push to `main`. It calls `deploy_docs::deploy_docs_job(DocsChannel::Nightly, &[])`. The deploy job includes the `install.sh` R2 upload and docs-proxy worker deploy (these are `main`-push operations, currently bundled in `deploy_cloudflare.yml`).
141
142Register it in `workflows.rs`: `WorkflowFile::zed(deploy_docs_nightly::deploy_docs_nightly)`.
143
144### 2.3 Add preview and stable deploy jobs to `release.rs`
145
146In `release.rs`, call `deploy_docs::deploy_docs_job` twice after `validate_release_assets` completes:
147
148```rust
149let deploy_docs_preview = deploy_docs::deploy_docs_job(
150    DocsChannel::Preview,
151    &[&validate_release_assets],
152);
153let deploy_docs_stable = deploy_docs::deploy_docs_job(
154    DocsChannel::Stable,
155    &[&validate_release_assets],
156);
157```
158
159Apply an `if:` condition to each job:
160- Preview: `startsWith(github.ref, 'refs/tags/v') && contains(github.ref, '-pre')`
161- Stable: `startsWith(github.ref, 'refs/tags/v') && !contains(github.ref, '-pre')`
162
163Add both jobs to the release workflow with `.add_job(...)`.
164
165### 2.4 Update the docs-proxy worker
166
167Modify `.cloudflare/docs-proxy/src/worker.js` to route all three channels:
168
169```javascript
170export default {
171  async fetch(request, _env, _ctx) {
172    const url = new URL(request.url);
173
174    let hostname;
175    if (url.pathname.startsWith("/docs/nightly")) {
176      hostname = "docs-nightly.pages.dev";
177    } else if (url.pathname.startsWith("/docs/preview")) {
178      hostname = "docs-preview.pages.dev";
179    } else {
180      hostname = "docs-anw.pages.dev";
181    }
182
183    url.hostname = hostname;
184    let res = await fetch(url, request);
185
186    if (res.status === 404) {
187      res = await fetch("https://zed.dev/404");
188    }
189
190    return res;
191  },
192};
193```
194
195The `docs-nightly` and `docs-preview` Pages project hostnames will be auto-assigned by Cloudflare on first deploy — verify the actual `*.pages.dev` hostnames and update accordingly.
196
197**Important:** Confirm no existing stable docs pages have a path starting with `nightly` or `preview` — grep `docs/src/SUMMARY.md` to verify.
198
199### 2.5 Wire install.sh and worker deploys
200
201- **install.sh R2 upload** → stays in the nightly workflow (runs on push to `main`, matching current behavior)
202- **docs-proxy worker deploy** → stays in the nightly workflow (worker routes all three channels, deploying once on `main` push is sufficient)
203
204---
205
206### 2.6 Add `noindex` meta tag for nightly and preview
207
208Also set a `DOCS_CHANNEL` env var (`nightly`, `preview`, or `stable`) on the build job alongside `MDBOOK_BOOK__SITE_URL` in step 2.1. Add a `channel_name() -> &'static str` method to `DocsChannel` returning the appropriate string.
209
210In `docs/theme/index.hbs`, add a `#noindex#` placeholder in `<head>` directly after the existing `{{#if is_print}}` noindex block:
211
212```html
213#noindex#
214```
215
216In `crates/docs_preprocessor/src/main.rs`'s `handle_postprocessing()`, read `DOCS_CHANNEL` and replace the placeholder — following the same pattern as the existing `#amplitude_key#` and `#description#` replacements:
217
218- `nightly` or `preview`: replace `#noindex#` with `<meta name="robots" content="noindex, nofollow">`
219- anything else: replace `#noindex#` with an empty string