1<!--
  2SPDX-FileCopyrightText: Amolith <amolith@secluded.site>
  3
  4SPDX-License-Identifier: CC0-1.0
  5-->
  6
  7# BadgerDB Schema Design for nasin pali
  8
  9## TL;DR
 10
 11- Use a single BadgerDB with a small, hierarchical keyspace: dir/…, idx/…, and s/{sid}/… (per-session).
 12- Store session documents (goal, tasks, meta) under s/{sid}/…, append-only events under s/{sid}/evt/{seq}, and small secondary indexes for fast lookups (active session by dir, tasks by status, active-session set).
 13- Rely on prefix scans and Badger subscriptions over these prefixes to drive the TUI in real time. Keep values JSON for readability; keep index values empty when possible; small payloads OK for convenience.
 14
 15## Recommended approach (simple path)
 16
 17### Keyspace layout (prefixes and values)
 18
 19#### Schema/version
 20- meta/schema_version -> "1"
 21
 22#### Working directory → active session lookup (with parent-walk)
 23- dir/{dir_hash}/active -> {sid}
 24  - Path canonicalization:
 25    - Start with current working directory
 26    - Convert to absolute path
 27    - Resolve all symlinks (EvalSymlinks)
 28    - Normalize path separators to forward slashes
 29    - On Windows, always case-fold to lowercase; on other OSes, do not case-fold
 30    - Result: canonical_path
 31  - {dir_hash} = full blake3-256(canonical_path), hex-encoded lowercase (64 hex chars)
 32  - This prevents issues with symlinks, bind mounts, and cross-platform path differences
 33  - Value is the current active {sid} (string)
 34  - Parent-walk resolution: from canonical_path, compute {dir_hash} and check dir/{dir_hash}/active; if absent, ascend to the parent directory and repeat until root; first match wins
 35  - Symlinks are resolved during canonicalization via EvalSymlinks; cycles are prevented by EvalSymlinks itself and an additional visited set for safety during parent-walk
 36
 37#### Active sessions set (for fast listing + subscribe)
 38- idx/active/{sid} -> {dir_hash} (or tiny JSON summary later if desired)
 39
 40#### Archived sessions (queryable by time and by dir)
 41- idx/archived/{ts_be}/{sid} -> {dir_hash}          // for global archive lists in chronological order
 42- dir/{dir_hash}/archived/{ts_be} -> {sid}          // for "archive history" per working directory
 43  - {ts_be} = 8-byte big-endian Unix nanos, hex-encoded lowercase, zero-padded to 16 hex chars; sorts chronologically
 44
 45#### Per-session namespace
 46- s/{sid}/meta -> JSON
 47  - { sid, dir_path, dir_hash, state: "active"|"archived", created_at, archived_at: null|ts, last_updated_at }
 48- s/{sid}/goal -> JSON
 49  - { title, description, updated_at }
 50- s/{sid}/task/{task_id} -> JSON
 51  - { id, title, description, status: "pending"|"in_progress"|"completed"|"failed"|"cancelled", created_at, updated_at, created_seq }
 52  - {task_id} = first 6 hex chars of blake3(normalize(title)+'|'+normalize(description)+'|'+sid); scoped per session, so cross-session collisions are irrelevant
 53- s/{sid}/idx/status/{status}/{task_id} -> "" (empty value; presence = membership)
 54  - Maintained atomically on task create/status update/delete
 55- s/{sid}/meta/evt_seq -> 8-byte big-endian counter stored as raw bytes (monotonic per session)
 56- s/{sid}/evt/{seq_be} -> JSON event record
 57  - {seq_be} = 8-byte big-endian u64 counter, hex-encoded lowercase, zero-padded to 16 hex chars; assures correct chronological iteration
 58  - { seq, at, type, reason: null|string, cmd: "np ...", payload: {...} }
 59  - Types you'll likely use: session_started, goal_set, goal_updated, task_added, task_updated, task_status_changed, session_archived, note
 60
 61### Core operations (transactional)
 62
 63#### Start session (np s)
 641) Canonicalize working directory using the process described in the "Working directory → active session lookup" section.
 652) Compute dir_hash from canonical_path.
 663) Check if dir/{dir_hash}/active exists:
 67   - If it exists, read the active {sid} and print: "Session {sid} is already active for this directory. There's already an active session for this directory; ask your operator whether they want to resume or archive it."
 68   - Return 0 (idempotent operation).
 694) If no active session exists, begin txn:
 70   - Generate new sid = ULID (time-ordered) or blake3(rand) hex; keep as short as practical (e.g., 26 char ULID).
 71   - Put s/{sid}/meta (state=active), s/{sid}/meta/evt_seq=0 (do NOT create goal yet; goal is created when first set).
 72   - Put dir/{dir_hash}/active -> {sid}.
 73   - Put idx/active/{sid} -> {dir_hash}.
 74   - Append event s/{sid}/evt/{0000000000000001} type=session_started.
 755) Commit.
 76
 77#### Set goal (np g s …)
 781) Lookup sid via dir/{dir_hash}/active.
 792) Check if s/{sid}/goal exists:
 80   - If it exists: error with message "Goal already set. Use 'np g u' to update it (requires -r/--reason flag)."
 81   - If it does not exist: proceed to step 3.
 823) txn:
 83   - Create s/{sid}/goal JSON.
 84   - Update s/{sid}/meta.last_updated_at.
 85   - Increment s/{sid}/meta/evt_seq and write s/{sid}/evt/{seq} with type=goal_set (no reason required).
 86
 87#### Update goal (np g u …)
 881) Lookup sid via dir/{dir_hash}/active.
 892) Check if s/{sid}/goal exists:
 90   - If it does not exist: error with message "No goal set yet. Use 'np g s' to set it first."
 91   - If it exists: proceed to step 3.
 923) Require reason flag (-r or --reason) with brief explanation for the update.
 934) txn:
 94   - Update s/{sid}/goal JSON.
 95   - Update s/{sid}/meta.last_updated_at.
 96   - Increment s/{sid}/meta/evt_seq and write s/{sid}/evt/{seq} with type=goal_updated, reason in payload.
 97
 98#### Add tasks (np t a …)
 991) For each task:
100   - Compute deterministic id = first 6 hex chars of blake3(normalize(title)+"|"+normalize(description)+"|"+sid)
101     - normalize(x) = strings.TrimSpace(x), then case-fold to lowercase, then apply Unicode NFC normalization
102     - Deterministic across runs and processes; stable within session; won't change on edits because id derives from initial content + sid.
103     - Treat adds as idempotent: if the same task (same title+description) is re-added, it will resolve to the same id and be a no-op.
104     - If a user wants to retry a cancelled task with the exact same title and description, they should update the existing task's status rather than adding a new one, or modify the title/description slightly to differentiate it.
105   - txn:
106     - If s/{sid}/task/{id} absent, increment evt_seq and create task with status=pending, created_at=now, created_seq=evt_seq.
107     - Put s/{sid}/idx/status/pending/{id}.
108     - Update meta.last_updated_at.
109     - Append event task_added with task payload.
110   - Note: When adding multiple tasks in a single np t a invocation, use a single transaction and increment evt_seq for each task to preserve the order they were provided on the command line. Tasks should be displayed sorted by created_seq (not created_at) to maintain stable, predictable ordering.
111
112#### Update task status/title/description (np t u …)
1131) Determine if reason is required:
114   - Required (-r or --reason flag) if updating title or description
115   - Required (-r or --reason flag) if changing status to "cancelled" or "failed"
116   - Not required for status changes to "pending", "in_progress", or "completed"
1172) txn:
118   - Read s/{sid}/task/{id}.
119   - If status changes: delete old s/{sid}/idx/status/{old}/{id}, put new s/{sid}/idx/status/{new}/{id}.
120   - Update task JSON.
121   - Update meta.last_updated_at.
122   - Increment evt_seq, append event task_updated or task_status_changed with reason (if provided).
123
124#### Archive session (np a)
1251) Read s/{sid}/meta:
126   - If already archived (state="archived"), return 0 (idempotent operation).
1272) If active, txn:
128   - Delete dir/{dir_hash}/active.
129   - Delete idx/active/{sid}.
130   - Update s/{sid}/meta.state="archived", archived_at=now, last_updated_at.
131   - Put idx/archived/{ts_be}/{sid} -> {dir_hash}.
132   - Put dir/{dir_hash}/archived/{ts_be} -> {sid}.
133   - Increment evt_seq, append session_archived event.
134
135### Query patterns
136
137- **Active session for current dir:**
138  - Get dir/{dir_hash}/active -> sid; if absent, parent-walk by recomputing dir_hash for parents until found or root
139- **List sessions (for TUI picker, not real-time):**
140  - One-time fetch: iterate idx/active/ for active sessions and idx/archived/ (descending) for archived sessions, limited to {terminal_height} total entries
141  - For each sid, read s/{sid}/meta for dir_path/timestamps and s/{sid}/goal for title
142  - Support paging through additional results if needed
143- **List tasks by status quickly:**
144  - Iterate s/{sid}/idx/status/{status}/, collect ids
145  - For each id, Get s/{sid}/task/{id}
146- **List all tasks:**
147  - Iterate s/{sid}/task/ prefix; sort by created_seq ascending to preserve insertion/provided order
148- **Events chronologically for a session:**
149  - Iterate s/{sid}/evt/ prefix ascending; {seq_be} sorts by event time order
150- **Resume after interruption (np r):**
151  - Use dir/{dir_hash}/active -> sid
152  - Read s/{sid}/goal and s/{sid}/task/* to render plan
153  - Optionally stream recent events from s/{sid}/evt/
154
155### Real-time monitoring via Badger subscribe
156
157#### Session view TUI (viewing a single session) subscribes to these prefixes:
158- s/{sid}/goal
159- s/{sid}/task/
160- s/{sid}/evt/
161- s/{sid}/meta (for last_updated_at/state)
162
163On receiving a change event, re-fetch the affected document(s) and redraw. Keep all write ops in a single txn so subscribers see atomic changes.
164
165#### Session list (NOT real-time)
166- Listing sessions is a one-time fetch operation, not real-time.
167- Fetch the most recent {terminal_height} sessions (active and archived combined) and allow paging.
168- Query pattern: iterate idx/active/ for active sessions, iterate idx/archived/ (descending by timestamp) for archived sessions.
169- For each session, read s/{sid}/meta and s/{sid}/goal to display directory, goal title, and timestamps.
170- No subscriptions needed for the list view—refresh only when user explicitly opens/refreshes the picker.
171
172### Value shapes (JSON)
173
174#### s/{sid}/meta:
175```json
176{
177  "sid": "01JP…",
178  "dir_path": "/abs/path",
179  "dir_hash": "…",
180  "state": "active",
181  "created_at": "RFC3339",
182  "archived_at": null,
183  "last_updated_at": "RFC3339"
184}
185```
186
187#### s/{sid}/goal:
188```json
189{
190  "title": "…",
191  "description": "…",
192  "updated_at": "RFC3339"
193}
194```
195
196#### s/{sid}/task/{id}:
197```json
198{
199  "id": "a1b2c3",
200  "title": "…",
201  "description": "…",
202  "status": "pending",
203  "created_at": "RFC3339",
204  "updated_at": "RFC3339",
205  "created_seq": 1
206}
207```
208
209#### s/{sid}/evt/{seq_be}:
210```json
211{
212  "seq": 1,
213  "at": "RFC3339Nano",
214  "type": "task_added",
215  "reason": null,
216  "cmd": "np t a …",
217  "payload": {}
218}
219```
220
221Note on `reason` field:
222- The `reason` field is optional (null or string) in all events
223- Reason is REQUIRED (via -r/--reason flag) only when:
224  - Updating existing goal or task content (title/description) - explain the change (e.g., "clarified scope", "fixed typo")
225  - Changing task status to "cancelled" or "failed" - explain why (e.g., "no longer needed", "blocked by missing dependency")
226- Reason is NOT required (and should be omitted/null) for expected operations:
227  - Setting goal initially
228  - Adding tasks
229  - Changing task status to "pending", "in_progress", or "completed"
230
231### Concurrency and safety
232
233- Always perform multi-key updates in a single Update txn:
234  - Example: task status change must atomically update task doc, move index keys, and append an event.
235  - When adding multiple tasks in a single CLI invocation, use one transaction that appends all events in order by incrementing evt_seq within the transaction.
236- Event sequencing:
237  - Keep s/{sid}/meta/evt_seq as a u64 counter (stored as 8-byte big-endian bytes, not decimal string, to avoid parse overhead). In txn:
238    - Read current value, increment for each event in the transaction, write back the final value, and write each event at s/{sid}/evt/{seq_be}.
239    - If txn conflict occurs (e.g., concurrent processes updating the same session), retry with short exponential backoff.
240    - This guarantees no gaps or overwrites in the event sequence; sequence numbers strictly increase.
241- Concurrent writers across multiple processes:
242  - Use Badger's optimistic concurrency control with retry logic on transaction conflicts.
243  - Keep transactions small and focused to minimize conflict probability.
244  - For batch operations (multiple tasks added by one command), use a single transaction to ensure atomicity and preserve ordering.
245- Enforce one active session per dir:
246  - np s checks if dir/{dir_hash}/active exists before creating a new session (idempotent behavior documented above).
247- Deterministic task IDs:
248  - Use blake3(normalized(title)+"|"+normalized(description)+"|"+sid), take first 8 hex.
249  - Never recompute on updates; the id is "creation id," not a content hash of current state.
250  - Re-adding the same task (same title+description) resolves to the same id and is treated as a no-op.
251
252### Denormalization (minimal, targeted)
253
254- Status index per session (s/{sid}/idx/status/{status}/{task_id}) to filter efficiently by status.
255- Active sessions index (idx/active/{sid}) for quick listing and subscribe.
256- Archives indices for chronological browsing and per-dir history.
257- Everything else is normalized to keep writes simple and atomic.
258- Task ordering: tasks are displayed sorted by created_seq (not created_at timestamp) to preserve the exact order they were provided via command-line flags, avoiding issues with clock precision and race conditions.
259
260### Key naming conventions
261
262- Lowercase ASCII, "/" as namespace delimiter, fixed segment order.
263- Use big-endian (zero-padded hex) counters/timestamps in keys to preserve sort order.
264- Keep values small; JSON for human-inspectable records; empty values for set-like indexes.
265
266### How subscriptions map to UI updates
267
268- **Session pane (single session view):** Subscribe to s/{sid}/task/ and s/{sid}/goal; when a mutation arrives, read the document(s) and refresh.
269- **Event log pane (single session view):** Subscribe to s/{sid}/evt/; stream latest events as they're appended by iterating from the last seen {seq_be}.
270- **Session list picker:** No subscriptions. One-time fetch on open/refresh: scan idx/active/ and idx/archived/ (limited by terminal height), read s/{sid}/meta and s/{sid}/goal for each session to display directory, title, and timestamps. Support paging through results.
271
272### Effort/scope
273
274- Schema + helpers (hashing, key builders, JSON structs): S (≤1–3h).
275- Implement start, goal set/update, task add/update, archive with atomic txns + events: M (1–2d to get right and tested).
276- TUI subscriptions wired to these prefixes: M (1–2d) once data model is in place.
277
278## Rationale and trade-offs
279
280- Prefix-first design matches Badger's strengths: fast iterators, prefix scans, and subscriptions.
281- Using per-session monotonic seq keys for events is simpler and more reliable than time-sorted keys, and you can still store timestamps in the value.
282- Minimal denormalization: we only add what's necessary for speed (status index, active session index). Everything else is derivable.
283- Hashing dir paths keeps keys safe/short and avoids platform-specific path issues while keeping full path in values for display.
284- JSON values keep early development simple and debuggable; you can binary-encode later if needed, without changing keys.
285- GOOS/GOARCH were removed from dir_hash computation to support cross-platform shared home directories (e.g., NFS, Dropbox, synced directories). The same working directory should map to the same session regardless of which OS accesses it.
286
287## Risks and guardrails
288
289- **Concurrent writers:** Use retries on txn conflicts (especially when incrementing evt_seq). Keep transactions small and focused.
290- **Clock weirdness:** Event order is based on seq, not time, so your chronology is stable even if system clock changes; human-readable timestamps live in the value.
291- **Task ID determinism vs edits:** Because id is based on initial content + sid, it won't change during edits. If you attempt to re-add the same task, you'll get the same id; treat it as idempotent add.
292- **Orphaned indexes:** Always mutate doc + index keys in the same txn; on startup or rarely, you can offer a "repair" command to reconcile status indexes, but careful txns should make this unnecessary.
293- **Path hashing collisions:** With blake3 and 8+ bytes hex, collision risk is negligible; still store full dir_path in meta for verification.
294- **Subscription storms:** Subscribe narrowly (per-session) and debounce UI redraws.
295
296## When to consider the advanced path
297
298- **Thousands of tasks/events per session and UI becomes slow scanning JSON:**
299  - Switch task/event values to a compact binary encoding (e.g., flatbuffers/msgpack) to reduce I/O.
300- **Cross-session global event feed (optional):**
301  - Maintain a global index per event: idx/events/{ts_be}/{sid}/{seq_be} -> "" pointing to s/{sid}/evt/{seq_be}.
302  - Pros: simple "recent activity" TUI across sessions, single subscription to idx/events/, efficient time-range scans.
303  - Cons: extra write per event (write amplification), hot global prefix under high throughput, time-ordering relies on event timestamps (minor clock skew acceptable), large histories require pagination/retention.
304- **Need ordering of tasks:**
305  - Add s/{sid}/idx/task_order/{seq_be} -> {task_id} and maintain it on inserts/moves.
306- **Multi-tenant or remote DB:**
307  - Add tenant prefixes or shard DBs; preserve the same key layout.
308
309## Optional advanced path (brief)
310
311- Use ULID for sids to naturally order sessions by creation time (helps chronological archive listings without extra metadata).
312- Maintain s/{sid}/stats -> {pending, in_progress, …, updated_at} and update counts transactionally on task status changes; lets the session list show counts without reading all tasks.
313- Add CAS-like guards using a version field inside task JSON (optimistic concurrency within your app domain), rejecting stale updates gracefully.
314
315## Summary
316
317This schema gives you:
318- O(1) active-session lookup by working directory.
319- O(#tasks with status) retrieval for status-filtered views via a single prefix iteration.
320- Stable, strictly-ordered per-session event logs.
321- Simple, efficient prefixes to subscribe for real-time TUI updates.
322- Clean archival with preserved queryability and chronological browsing.