@@ -0,0 +1,434 @@
+# Soft Serve Development Documentation
+
+## Important Instructions
+
+- Use the gopls MCP tooling as much as appropriate. It provides dense, highly-relevant symbolic information. Before making any edit, use the workspace tool, check the relevant APIs and references if necessary, and _then_ proceed with edits.
+
+## Code Style Guidelines
+
+### Formatting
+
+- **Go files**: Use tabs for indentation
+- **Other files**: Use 2 spaces for indentation
+- Use LF line endings
+- Insert final newline
+- Trim trailing whitespace
+- UTF-8 character encoding
+
+### Go Code Style
+
+- Format with `gofumpt` and `goimports` (configured in `.golangci.yml`)
+- Use standard Go naming conventions (PascalCase for exported, camelCase for unexported)
+- Wrap exported types with helper methods rather than exposing underlying library types directly
+- Implement standard interfaces where appropriate (e.g., `sort.Interface`)
+
+### Comments
+
+- Exported functions, types, and methods must have godoc comments
+- Comments should end with proper punctuation (enforced by `godot` linter)
+- Start comments with the name of the item being documented
+
+### Error Handling
+
+- Wrap errors with context using `wrapcheck` patterns
+- Check all error returns (enforced by various linters)
+- Close resources properly (enforced by `bodyclose`, `sqlclosecheck`, `rowserrcheck`)
+
+## How to Write Unit Tests
+
+### Tooling
+
+- Unit tests: Go's built-in `testing` package with `matryer/is` for assertions
+- Integration tests: `testscript` framework (see `testscript/` directory)
+
+### Running Tests
+
+```bash
+# Run all tests
+go test ./...
+
+# Run tests for specific package
+go test ./pkg/backend
+
+# Run integration tests
+cd testscript && go test -v
+
+# Run with PostgreSQL
+SOFT_SERVE_DB_DRIVER=postgres \
+SOFT_SERVE_DB_DATA_SOURCE="postgres://postgres:postgres@localhost/postgres?sslmode=disable" \
+go test ./...
+
+# Disable race detection (for slower systems)
+SOFT_SERVE_DISABLE_RACE_CHECKS=1 go test ./...
+```
+
+### Writing Unit Tests
+
+- Place test files alongside the code being tested (e.g., `tree_test.go` next to `tree.go`)
+- Use `_test.go` suffix for test files
+- Use `matryer/is` for simple, readable assertions
+- Mock databases using transactions that rollback in deferred functions
+- Test both SQLite and PostgreSQL code paths for database-related changes
+- Use `pkg/test/test.go` helpers for fixtures (e.g., `test.RandomPort()`)
+
+### Writing Integration Tests
+
+- Add `.txtar` files to `testscript/testdata/`
+- Each test spins up a complete Soft Serve instance with randomized ports
+- Available commands: `soft`, `usoft`, `git`, `ugit`, `ui`, `uui`, `curl`, `mkfile`, `readfile`, `envfile`
+- Use `ensureserverrunning`/`ensureservernotrunning` to wait for server state
+
+## Development Commands and Environment
+
+### Build and Development Commands
+
+#### Building
+
+```bash
+go build -o soft ./cmd/soft
+```
+
+### Environment Variables
+
+All configuration can be set via environment variables with the `SOFT_SERVE_` prefix:
+- `SOFT_SERVE_DATA_PATH`: Data directory location (default: `data`)
+- `SOFT_SERVE_CONFIG_LOCATION`: Custom config file path
+- `SOFT_SERVE_INITIAL_ADMIN_KEYS`: SSH keys for initial admin (setup only, newline-separated)
+- `SOFT_SERVE_SSH_LISTEN_ADDR`: SSH server address (default: `:23231`)
+- `SOFT_SERVE_HTTP_LISTEN_ADDR`: HTTP server address (default: `:23232`)
+- `SOFT_SERVE_GIT_LISTEN_ADDR`: Git daemon address (default: `:9418`)
+- `SOFT_SERVE_DB_DRIVER`: Database driver (`sqlite` or `postgres`)
+- `SOFT_SERVE_DB_DATA_SOURCE`: Database connection string
+- `SOFT_SERVE_NO_COLOR`: Disable color output (set to any truthy value)
+- `SOFT_SERVE_DEBUG`: Enable debug logging (enables auth callback logging)
+- `SOFT_SERVE_VERBOSE`: Enable verbose logging (logs DB queries, SSH envs/keys; requires `SOFT_SERVE_DEBUG` to also be set)
+
+Environment variables always override config file settings via `cfg.ParseEnv()`.
+
+## Advanced Features and Storage
+
+### TUI Architecture
+
+The TUI (`pkg/ui/`) is built with Bubble Tea v2 and organized as:
+- `pages/`: Main views (repo browser, file viewer, log viewer, refs viewer)
+- `components/`: Reusable UI elements (code viewer, tabs, header, footer, viewport)
+- `common/`: Shared state and utilities including `common.Common` struct passed to all components
+- `styles/`: Lipgloss style definitions
+- `keymap/`: Keyboard shortcuts
+
+The TUI runs in PTY sessions and uses the BubbleTea middleware (`bm.MiddlewareWithProgramHandler`).
+
+### Git Protocol Version
+
+The git daemon and SSH/HTTP handlers support git protocol v2. The protocol version is negotiated by the client:
+- HTTP: Client sends `Git-Protocol` header (e.g., `Git-Protocol: version=2`)
+- SSH: Client sets `GIT_PROTOCOL` in session environment
+- Git daemon: Client specifies protocol in pktline extra parameters
+
+The server passes the `GIT_PROTOCOL` environment variable through to git processes.
+
+## Architecture and Core Components
+
+### Project Overview
+
+Soft Serve is a self-hostable Git server with SSH/HTTP/Git protocol support, featuring a terminal UI (TUI), Git LFS support, webhooks, and access control. It's built with Go and uses Bubble Tea for the TUI.
+
+### Core Components
+
+1. **Backend** (`pkg/backend/`): Central orchestrator that manages all business logic. The Backend struct ties together the database, store, config, cache, and task manager. It's the primary interface for repository, user, and settings operations. Key responsibilities:
+ - Repository CRUD operations with cache management
+ - User authentication and authorization
+ - Hook execution (implements `hooks.Hooks` interface)
+ - LFS object management
+ - Webhook event creation
+
+2. **Store** (`pkg/store/`): Database abstraction layer implementing repository, user, collaborator, LFS, access token, webhook, and settings persistence. The actual implementation lives in `pkg/store/database/`. This is a pure data layer with no business logic.
+
+3. **Database** (`pkg/db/`): Low-level database wrapper around `sqlx` supporting both SQLite (default) and PostgreSQL. Provides transaction management and migrations (in `pkg/db/migrate/`). Uses context-aware logging when verbose mode is enabled.
+
+4. **Config** (`pkg/config/`): Configuration management supporting both YAML files and environment variables. All `SOFT_SERVE_*` env vars override config file settings. Config is immutable once loaded and passed through context.
+
+5. **Git Operations** (`git/` package): Git command execution and repository manipulation using `aymanbagabas/git-module`. Handles commits, tags, references, patches, trees, and server-side hooks. This is a separate top-level package, not under `pkg/`.
+
+### Server Protocols
+
+1. **SSH Server** (`pkg/ssh/`): Provides both interactive TUI access and git protocol operations. Uses a middleware chain pattern:
+ - `AuthenticationMiddleware`: Validates SSH public key fingerprint matches approved key
+ - `ContextMiddleware`: Injects config, backend, store, DB, logger into session context
+ - `CommandMiddleware`: Routes non-PTY sessions to CLI commands using Cobra
+ - `LoggingMiddleware`: Logs connection, commands, and duration
+
+ CLI commands live in `pkg/ssh/cmd/` as Cobra commands.
+
+2. **HTTP Server** (`pkg/web/`): Serves git HTTP protocol, Git LFS endpoints, and provides basic auth using access tokens. Uses Gorilla Mux for routing with these controllers:
+ - `GitController`: Git smart HTTP protocol
+ - `HealthController`: Health check endpoint
+ - Also uses middleware chain for compression, recovery, logging, and context
+
+3. **Git Daemon** (`pkg/daemon/`): Native git:// protocol support for anonymous read access. Implements connection pooling with configurable max connections and timeouts.
+
+4. **Stats Server**: Prometheus metrics endpoint for monitoring.
+
+### Access Control
+
+The `pkg/access/` package implements a four-level access system defined as iota constants:
+- `NoAccess` (0): Denies all access
+- `ReadOnlyAccess` (1): Can read public repos
+- `ReadWriteAccess` (2): Full control of repos
+- `AdminAccess` (3): Full server control
+
+Anonymous access is controlled by two settings:
+- `allow-keyless`: Whether to allow connections without SSH keys (affects HTTP/git:// protocols)
+- `anon-access`: Default access level for anonymous users (defaults to read-only)
+
+Access evaluation order:
+1. Admin users (via `initial_admin_keys` or user.is_admin flag) → AdminAccess
+2. Repository owner (user_id matches repo.user_id) → ReadWriteAccess
+3. Explicit collaborators → their assigned level
+4. Authenticated users → read-only for public repos
+5. Anonymous users → `anon-access` setting (if `allow-keyless` is true)
+
+## Data Flow and Git Hooks
+
+### Key Data Flow
+
+1. **Context Propagation**: Config, Backend, Store, and DB are passed through `context.Context` using type-safe context helpers (e.g., `config.FromContext(ctx)`). Each package defines its own context key as an unexported type to prevent collisions.
+
+2. **Initialization Sequence** (see `cmd/cmd.go:InitBackendContext`):
+ ```
+ Config → DB → Store → Backend → Context
+ ```
+
+ This happens in `InitBackendContext` which is used as a PersistentPreRunE hook in commands that need backend access.
+
+3. **Request Flow**:
+ ```
+ SSH/HTTP/Git Protocol → Middleware → Backend → Store → Database
+ ↓
+ Cache (LRU, 1000 items)
+ ```
+
+4. **SSH Command Flow**:
+ ```
+ SSH Session → CommandMiddleware → Cobra Command → Backend Method → Store → DB
+ ```
+
+ Non-PTY sessions are routed to CLI commands. PTY sessions go to the BubbleTea TUI.
+
+### Git Hooks System
+
+Git hooks use a clever two-tier system:
+
+1. **Hook Dispatcher** (generated in each repo's `hooks/` directory): Bash script that reads stdin, then calls all executables in `hooks/<hookname>.d/` directory, passing both stdin and arguments. Exits on first non-zero exit code.
+
+2. **Soft Serve Hook** (in `hooks/<hookname>.d/soft-serve`): Calls back into the `soft` binary via `soft hook <hookname>` command, which invokes Backend methods that implement the `hooks.Hooks` interface.
+
+Hook generation (`pkg/hooks/gen.go`):
+- Creates hook dispatcher at `hooks/<hookname>`
+- Creates `hooks/<hookname>.d/` directory
+- Generates `hooks/<hookname>.d/soft-serve` that calls `${SOFT_SERVE_BIN_PATH} hook <hookname>`
+- Supports: pre-receive, update, post-receive, post-update
+
+This design allows:
+- Multiple hooks per event (run in alphabetical order from `.d/` directory)
+- User-defined hooks alongside Soft Serve's built-in hooks in each repo's `.d/` directory
+
+### Hook Environment Variables
+
+Git hooks receive environment variables set in the git daemon, SSH middleware, or HTTP handler:
+- `SOFT_SERVE_REPO_NAME`: Repository name (without .git)
+- `SOFT_SERVE_REPO_PATH`: Full path to repository
+- `SOFT_SERVE_PUBLIC_KEY`: The authorized SSH key (in authorized_keys format)
+- `SOFT_SERVE_USERNAME`: Username for authenticated users
+- `SOFT_SERVE_BIN_PATH`: Path to soft binary (defaults to "soft" if not set)
+- `SOFT_SERVE_HOST`: Hostname from git protocol handshake
+- `SOFT_SERVE_LOG_PATH`: Path to hooks log file
+- `GIT_DIR`: Standard git variable
+- `GIT_PROTOCOL`: Git protocol version and capabilities
+
+The backend reads `SOFT_SERVE_PUBLIC_KEY` and `SOFT_SERVE_USERNAME` to identify users in hooks.
+
+## Database Schema and Migrations
+
+### Database Schema
+
+Key tables (see `pkg/db/models/`):
+- `users`: User accounts with admin flags, created_at, updated_at
+- `public_keys`: SSH public keys linked to users (one-to-many)
+- `repos`: Repository metadata with user_id owner (required)
+- `collabs`: Repository collaborators with access levels (many-to-many users↔repos)
+- `settings`: Server-wide settings (key-value store)
+- `lfs_objects`: Git LFS object metadata linked to repos
+- `lfs_locks`: Git LFS file locks with path and user
+- `access_tokens`: User access tokens for HTTP auth with optional expiry
+- `webhooks`: Repository webhook configurations with secret and events
+- `webhook_events`: Webhook event type subscriptions (many-to-many webhooks↔events)
+- `webhook_deliveries`: Webhook delivery attempts with request/response logs
+
+Important schema notes:
+- `repos.user_id` is `NOT NULL` (legacy repos are assigned to first admin during migration)
+- `public_keys` has a unique constraint on the key content (prevents duplicate keys)
+- `collabs` has a composite unique key on (user_id, repo_id)
+- `lfs_objects` has a composite unique key on (oid, repo_id)
+- `lfs_locks` has a composite unique key on (repo_id, path)
+- `webhook_events` has a composite unique key on (webhook_id, event)
+- Foreign key constraints are enabled and critical for data integrity
+
+### Database Migrations
+
+Migrations live in `pkg/db/migrate/` with separate SQL files for SQLite and PostgreSQL:
+- `NNNN_migration_name_sqlite.up.sql` / `.down.sql`
+- `NNNN_migration_name_postgres.up.sql` / `.down.sql`
+- Migration number prefixes must be unique and sequential
+- Go files (`NNNN_migration_name.go`) can handle data migrations
+
+The migration system in `migrate.go` automatically runs pending migrations on startup.
+
+### Transaction Patterns
+
+Always use `db.TransactionContext` for database operations:
+
+```go
+err := d.db.TransactionContext(ctx, func(tx *db.Tx) error {
+ // All operations use tx, not db
+ return d.store.CreateRepo(ctx, tx, name, ...)
+})
+return db.WrapError(err) // Wraps db errors as proto errors
+```
+
+The transaction automatically:
+- Commits on `nil` return
+- Rolls back on error return
+- Handles `sql.ErrTxDone` gracefully
+
+### Common Gotchas
+
+#### SQLite Foreign Keys
+
+SQLite requires `_pragma=foreign_keys(1)` in the DSN. The default config includes this, but custom configs must include it:
+```
+soft-serve.db?_pragma=busy_timeout(5000)&_pragma=foreign_keys(1)
+```
+
+Without this, cascade deletes won't work and referential integrity is not enforced.
+
+#### Transaction Context
+
+When creating database transactions, use `TransactionContext(ctx, fn)` to properly propagate context with logger and other values. The `Transaction(fn)` method uses `context.Background()` which loses context values.
+
+## Development Workflow Guides
+
+### Adding a New SSH Command
+
+1. Create command file in `pkg/ssh/cmd/` (e.g., `mycommand.go`)
+2. Define Cobra command with `RunE` function
+3. Use `checkIfReadable`, `checkIfCollab`, or `checkIfAdmin` for authorization
+4. Add command to `CommandMiddleware` in `pkg/ssh/middleware.go`
+5. Test with testscript in `testscript/testdata/`
+
+### Adding a New Database Table
+
+1. Create model in `pkg/db/models/`
+2. Create migration files in `pkg/db/migrate/` (both SQLite and Postgres)
+3. Add store interface methods to `pkg/store/` and implement in `pkg/store/database/`
+4. Update `pkg/backend/` to expose operations
+5. Test migrations with both databases
+
+### Adding a New Webhook Event
+
+1. Define event type in `pkg/webhook/event.go`
+2. Create event constructor in `pkg/webhook/` (e.g., `NewMyEvent`)
+3. Call `webhook.SendEvent(ctx, event)` where event should fire
+4. Add test in `testscript/testdata/`
+
+### Working with Git Operations
+
+All git operations should:
+- Use the `git` package, not exec.Command directly
+- Set appropriate timeouts (`CommandOptions.Timeout`)
+- Pass context for cancellation
+- Handle `git.ErrInvalidRepo` for non-existent repos
+- Use `git.EnsureWithin()` to prevent directory traversal attacks
+
+## Patterns and Conventions
+
+### Context Usage
+
+Always retrieve dependencies from context rather than passing them explicitly:
+```go
+cfg := config.FromContext(ctx)
+be := backend.FromContext(ctx)
+dbx := db.FromContext(ctx)
+store := store.FromContext(ctx)
+logger := log.FromContext(ctx)
+user := proto.UserFromContext(ctx)
+```
+
+### Repository Naming
+
+- User-facing repo names omit the `.git` extension (e.g., `dotfiles`)
+- On-disk repos are always `<name>.git` (e.g., `dotfiles.git`)
+- Repos are stored in `<data_path>/repos/<name>.git`
+- Nested repos are supported (e.g., `user/project.git` → `repos/user/project.git`)
+- Always use `utils.SanitizeRepo()` before using user input as repo name
+- Always validate with `utils.ValidateRepo()` before creating/renaming repos
+
+### Cache Invalidation
+
+The backend maintains an LRU cache (1000 items) of repository objects. **Critical**: Always invalidate cache after mutations:
+
+```go
+// Delete cache before Db transaction
+d.cache.Delete(name)
+
+return d.db.TransactionContext(ctx, func(tx *db.Tx) error {
+ // ... database operations
+})
+```
+
+Cache invalidation happens in:
+- `DeleteRepository`: After deletion
+- `RenameRepository`: For old name
+- `SetDescription`, `SetPrivate`, `SetHidden`, `SetProjectName`: Before update
+
+Failure to invalidate cache will cause stale data to be served until cache eviction.
+
+### SSH Authentication Flow
+
+1. Client connects with public key
+2. `PublicKeyHandler` checks if key belongs to a user, stores fingerprint in `Permissions.Extensions["pubkey-fp"]`
+3. If all public keys fail, `KeyboardInteractiveHandler` checks `allow-keyless` setting
+4. `AuthenticationMiddleware` validates that the public key fingerprint in context matches the actual auth key (gossh doesn't guarantee this)
+
+This double-check prevents a security issue where gossh's PublicKeyCallback doesn't guarantee the approved key is the one actually used.
+
+### Task Manager for Long-Running Operations
+
+Repository imports run asynchronously using `pkg/task`:
+
+```go
+tid := "import:" + name
+d.manager.Add(tid, func(ctx context.Context) error {
+ // Long-running operation
+ return nil
+})
+
+done := make(chan error, 1)
+go d.manager.Run(tid, done)
+err := <-done
+```
+
+Tasks are identified by string ID and automatically cleaned up on completion. The manager respects context cancellation.
+
+### Repository Path Construction
+
+Always use `backend.repoPath()` or construct paths consistently:
+```go
+// Correct
+rp := filepath.Join(cfg.DataPath, "repos", repoName+".git")
+
+// Wrong - missing .git suffix
+rp := filepath.Join(cfg.DataPath, "repos", repoName)
+```
+
+The `.git` suffix is mandatory for git to recognize bare repositories.