Soft Serve Development Documentation

Important Instructions

Use the gopls MCP tooling as much as appropriate. It provides dense, highly-relevant symbolic information. Before making any edit, use the workspace tool, check the relevant APIs and references if necessary, and then proceed with edits.

Code Style Guidelines

Formatting

Go files: Use tabs for indentation
Other files: Use 2 spaces for indentation
Use LF line endings
Insert final newline
Trim trailing whitespace
UTF-8 character encoding

Go Code Style

Format with gofumpt and goimports (configured in .golangci.yml)
Use standard Go naming conventions (PascalCase for exported, camelCase for unexported)
Wrap exported types with helper methods rather than exposing underlying library types directly
Implement standard interfaces where appropriate (e.g., sort.Interface)

Comments

Exported functions, types, and methods must have godoc comments
Comments should end with proper punctuation (enforced by godot linter)
Start comments with the name of the item being documented

Error Handling

Wrap errors with context using wrapcheck patterns
Check all error returns (enforced by various linters)
Close resources properly (enforced by bodyclose, sqlclosecheck, rowserrcheck)

How to Write Unit Tests

Tooling

Unit tests: Go's built-in testing package with matryer/is for assertions
Integration tests: testscript framework (see testscript/ directory)

Running Tests

# Run all tests
go test ./...

# Run tests for specific package
go test ./pkg/backend

# Run integration tests
cd testscript && go test -v

# Run with PostgreSQL
SOFT_SERVE_DB_DRIVER=postgres \
SOFT_SERVE_DB_DATA_SOURCE="postgres://postgres:postgres@localhost/postgres?sslmode=disable" \
go test ./...

# Disable race detection (for slower systems)
SOFT_SERVE_DISABLE_RACE_CHECKS=1 go test ./...

Writing Unit Tests

Place test files alongside the code being tested (e.g., tree_test.go next to tree.go)
Use _test.go suffix for test files
Use matryer/is for simple, readable assertions
Mock databases using transactions that rollback in deferred functions
Test both SQLite and PostgreSQL code paths for database-related changes
Use pkg/test/test.go helpers for fixtures (e.g., test.RandomPort())

Writing Integration Tests

Add .txtar files to testscript/testdata/
Each test spins up a complete Soft Serve instance with randomized ports
Available commands: soft, usoft, git, ugit, ui, uui, curl, mkfile, readfile, envfile
Use ensureserverrunning/ensureservernotrunning to wait for server state

Development Commands and Environment

Build and Development Commands

Building

go build -o soft ./cmd/soft

CSS Cache-Busting

When editing CSS files in pkg/web/static/, increment the version query parameter in pkg/web/templates/base.html:

overrides.css?v=1 → overrides.css?v=2
syntax.css?v=1 → syntax.css?v=2

Environment Variables

All configuration can be set via environment variables with the SOFT_SERVE_ prefix:

SOFT_SERVE_DATA_PATH: Data directory location (default: data)
SOFT_SERVE_CONFIG_LOCATION: Custom config file path
SOFT_SERVE_INITIAL_ADMIN_KEYS: SSH keys for initial admin (setup only, newline-separated)
SOFT_SERVE_SSH_LISTEN_ADDR: SSH server address (default: :23231)
SOFT_SERVE_HTTP_LISTEN_ADDR: HTTP server address (default: :23232)
SOFT_SERVE_GIT_LISTEN_ADDR: Git daemon address (default: :9418)
SOFT_SERVE_DB_DRIVER: Database driver (sqlite or postgres)
SOFT_SERVE_DB_DATA_SOURCE: Database connection string
SOFT_SERVE_NO_COLOR: Disable color output (set to any truthy value)
SOFT_SERVE_DEBUG: Enable debug logging (enables auth callback logging)
SOFT_SERVE_VERBOSE: Enable verbose logging (logs DB queries, SSH envs/keys; requires SOFT_SERVE_DEBUG to also be set)

Environment variables always override config file settings via cfg.ParseEnv().

Advanced Features and Storage

TUI Architecture

The TUI (pkg/ui/) is built with Bubble Tea v2 and organized as:

pages/: Main views (repo browser, file viewer, log viewer, refs viewer)
components/: Reusable UI elements (code viewer, tabs, header, footer, viewport)
common/: Shared state and utilities including common.Common struct passed to all components
styles/: Lipgloss style definitions
keymap/: Keyboard shortcuts

The TUI runs in PTY sessions and uses the BubbleTea middleware (bm.MiddlewareWithProgramHandler).

Git Protocol Version

The git daemon and SSH/HTTP handlers support git protocol v2. The protocol version is negotiated by the client:

HTTP: Client sends Git-Protocol header (e.g., Git-Protocol: version=2)
SSH: Client sets GIT_PROTOCOL in session environment
Git daemon: Client specifies protocol in pktline extra parameters

The server passes the GIT_PROTOCOL environment variable through to git processes.

Architecture and Core Components

Project Overview

Soft Serve is a self-hostable Git server with SSH/HTTP/Git protocol support, featuring a terminal UI (TUI), Git LFS support, webhooks, and access control. It's built with Go and uses Bubble Tea for the TUI.

Core Components

Backend (pkg/backend/): Central orchestrator that manages all business logic. The Backend struct ties together the database, store, config, cache, and task manager. It's the primary interface for repository, user, and settings operations. Key responsibilities:
- Repository CRUD operations with cache management
- User authentication and authorization
- Hook execution (implements hooks.Hooks interface)
- LFS object management
- Webhook event creation
Store (pkg/store/): Database abstraction layer implementing repository, user, collaborator, LFS, access token, webhook, and settings persistence. The actual implementation lives in pkg/store/database/. This is a pure data layer with no business logic.
Database (pkg/db/): Low-level database wrapper around sqlx supporting both SQLite (default) and PostgreSQL. Provides transaction management and migrations (in pkg/db/migrate/). Uses context-aware logging when verbose mode is enabled.
Config (pkg/config/): Configuration management supporting both YAML files and environment variables. All SOFT_SERVE_* env vars override config file settings. Config is immutable once loaded and passed through context.
Git Operations (git/ package): Git command execution and repository manipulation using aymanbagabas/git-module. Handles commits, tags, references, patches, trees, and server-side hooks. This is a separate top-level package, not under pkg/.

Server Protocols

SSH Server (pkg/ssh/): Provides both interactive TUI access and git protocol operations. Uses a middleware chain pattern:
- AuthenticationMiddleware: Validates SSH public key fingerprint matches approved key
- ContextMiddleware: Injects config, backend, store, DB, logger into session context
- CommandMiddleware: Routes non-PTY sessions to CLI commands using Cobra
- LoggingMiddleware: Logs connection, commands, and duration
CLI commands live in pkg/ssh/cmd/ as Cobra commands.
HTTP Server (pkg/web/): Serves git HTTP protocol, Git LFS endpoints, and provides basic auth using access tokens. Uses Gorilla Mux for routing with these controllers:
- GitController: Git smart HTTP protocol
- HealthController: Health check endpoint
- Also uses middleware chain for compression, recovery, logging, and context
Git Daemon (pkg/daemon/): Native git:// protocol support for anonymous read access. Implements connection pooling with configurable max connections and timeouts.
Stats Server: Prometheus metrics endpoint for monitoring.

Access Control

The pkg/access/ package implements a four-level access system defined as iota constants:

NoAccess (0): Denies all access
ReadOnlyAccess (1): Can read public repos
ReadWriteAccess (2): Full control of repos
AdminAccess (3): Full server control

Anonymous access is controlled by two settings:

allow-keyless: Whether to allow connections without SSH keys (affects HTTP/git:// protocols)
anon-access: Default access level for anonymous users (defaults to read-only)

Access evaluation order:

Admin users (via initial_admin_keys or user.is_admin flag) → AdminAccess
Repository owner (user_id matches repo.user_id) → ReadWriteAccess
Explicit collaborators → their assigned level
Authenticated users → read-only for public repos
Anonymous users → anon-access setting (if allow-keyless is true)

Data Flow and Git Hooks

Key Data Flow

Context Propagation: Config, Backend, Store, and DB are passed through context.Context using type-safe context helpers (e.g., config.FromContext(ctx)). Each package defines its own context key as an unexported type to prevent collisions.
Initialization Sequence (see cmd/cmd.go:InitBackendContext):
```
Config → DB → Store → Backend → Context
```
This happens in InitBackendContext which is used as a PersistentPreRunE hook in commands that need backend access.

Request Flow:

SSH/HTTP/Git Protocol → Middleware → Backend → Store → Database
                                ↓
                             Cache (LRU, 1000 items)

SSH Command Flow:
```
SSH Session → CommandMiddleware → Cobra Command → Backend Method → Store → DB
```
Non-PTY sessions are routed to CLI commands. PTY sessions go to the BubbleTea TUI.

Git Hooks System

Git hooks use a clever two-tier system:

Hook Dispatcher (generated in each repo's hooks/ directory): Bash script that reads stdin, then calls all executables in hooks/<hookname>.d/ directory, passing both stdin and arguments. Exits on first non-zero exit code.
Soft Serve Hook (in hooks/<hookname>.d/soft-serve): Calls back into the soft binary via soft hook <hookname> command, which invokes Backend methods that implement the hooks.Hooks interface.

Hook generation (pkg/hooks/gen.go):

Creates hook dispatcher at hooks/<hookname>
Creates hooks/<hookname>.d/ directory
Generates hooks/<hookname>.d/soft-serve that calls ${SOFT_SERVE_BIN_PATH} hook <hookname>
Supports: pre-receive, update, post-receive, post-update

This design allows:

Multiple hooks per event (run in alphabetical order from .d/ directory)
User-defined hooks alongside Soft Serve's built-in hooks in each repo's .d/ directory

Hook Environment Variables

Git hooks receive environment variables set in the git daemon, SSH middleware, or HTTP handler:

SOFT_SERVE_REPO_NAME: Repository name (without .git)
SOFT_SERVE_REPO_PATH: Full path to repository
SOFT_SERVE_PUBLIC_KEY: The authorized SSH key (in authorized_keys format)
SOFT_SERVE_USERNAME: Username for authenticated users
SOFT_SERVE_BIN_PATH: Path to soft binary (defaults to "soft" if not set)
SOFT_SERVE_HOST: Hostname from git protocol handshake
SOFT_SERVE_LOG_PATH: Path to hooks log file
GIT_DIR: Standard git variable
GIT_PROTOCOL: Git protocol version and capabilities

The backend reads SOFT_SERVE_PUBLIC_KEY and SOFT_SERVE_USERNAME to identify users in hooks.

Database Schema and Migrations

Database Schema

Key tables (see pkg/db/models/):

users: User accounts with admin flags, created_at, updated_at
public_keys: SSH public keys linked to users (one-to-many)
repos: Repository metadata with user_id owner (required)
collabs: Repository collaborators with access levels (many-to-many users↔repos)
settings: Server-wide settings (key-value store)
lfs_objects: Git LFS object metadata linked to repos
lfs_locks: Git LFS file locks with path and user
access_tokens: User access tokens for HTTP auth with optional expiry
webhooks: Repository webhook configurations with secret and events
webhook_events: Webhook event type subscriptions (many-to-many webhooks↔events)
webhook_deliveries: Webhook delivery attempts with request/response logs

Important schema notes:

repos.user_id is NOT NULL (legacy repos are assigned to first admin during migration)
public_keys has a unique constraint on the key content (prevents duplicate keys)
collabs has a composite unique key on (user_id, repo_id)
lfs_objects has a composite unique key on (oid, repo_id)
lfs_locks has a composite unique key on (repo_id, path)
webhook_events has a composite unique key on (webhook_id, event)
Foreign key constraints are enabled and critical for data integrity

Database Migrations

Migrations live in pkg/db/migrate/ with separate SQL files for SQLite and PostgreSQL:

NNNN_migration_name_sqlite.up.sql / .down.sql
NNNN_migration_name_postgres.up.sql / .down.sql
Migration number prefixes must be unique and sequential
Go files (NNNN_migration_name.go) can handle data migrations

The migration system in migrate.go automatically runs pending migrations on startup.

Transaction Patterns

Always use db.TransactionContext for database operations:

err := d.db.TransactionContext(ctx, func(tx *db.Tx) error {
    // All operations use tx, not db
    return d.store.CreateRepo(ctx, tx, name, ...)
})
return db.WrapError(err) // Wraps db errors as proto errors

The transaction automatically:

Commits on nil return
Rolls back on error return
Handles sql.ErrTxDone gracefully

Common Gotchas

SQLite Foreign Keys

SQLite requires _pragma=foreign_keys(1) in the DSN. The default config includes this, but custom configs must include it:

soft-serve.db?_pragma=busy_timeout(5000)&_pragma=foreign_keys(1)

Without this, cascade deletes won't work and referential integrity is not enforced.

Transaction Context

When creating database transactions, use TransactionContext(ctx, fn) to properly propagate context with logger and other values. The Transaction(fn) method uses context.Background() which loses context values.

Development Workflow Guides

Adding a New SSH Command

Create command file in pkg/ssh/cmd/ (e.g., mycommand.go)
Define Cobra command with RunE function
Use checkIfReadable, checkIfCollab, or checkIfAdmin for authorization
Add command to CommandMiddleware in pkg/ssh/middleware.go
Test with testscript in testscript/testdata/

Adding a New Database Table

Create model in pkg/db/models/
Create migration files in pkg/db/migrate/ (both SQLite and Postgres)
Add store interface methods to pkg/store/ and implement in pkg/store/database/
Update pkg/backend/ to expose operations
Test migrations with both databases

Adding a New Webhook Event

Define event type in pkg/webhook/event.go
Create event constructor in pkg/webhook/ (e.g., NewMyEvent)
Call webhook.SendEvent(ctx, event) where event should fire
Add test in testscript/testdata/

Working with Git Operations

All git operations should:

Use the git package, not exec.Command directly
Set appropriate timeouts (CommandOptions.Timeout)
Pass context for cancellation
Handle git.ErrInvalidRepo for non-existent repos
Use git.EnsureWithin() to prevent directory traversal attacks

Patterns and Conventions

Context Usage

Always retrieve dependencies from context rather than passing them explicitly:

cfg := config.FromContext(ctx)
be := backend.FromContext(ctx)
dbx := db.FromContext(ctx)
store := store.FromContext(ctx)
logger := log.FromContext(ctx)
user := proto.UserFromContext(ctx)

Repository Naming

User-facing repo names omit the .git extension (e.g., dotfiles)
On-disk repos are always <name>.git (e.g., dotfiles.git)
Repos are stored in <data_path>/repos/<name>.git
Nested repos are supported (e.g., user/project.git → repos/user/project.git)
Always use utils.SanitizeRepo() before using user input as repo name
Always validate with utils.ValidateRepo() before creating/renaming repos

Cache Invalidation

The backend maintains an LRU cache (1000 items) of repository objects. Critical: Always invalidate cache after mutations:

// Delete cache before Db transaction
d.cache.Delete(name)

return d.db.TransactionContext(ctx, func(tx *db.Tx) error {
    // ... database operations
})

Cache invalidation happens in:

DeleteRepository: After deletion
RenameRepository: For old name
SetDescription, SetPrivate, SetHidden, SetProjectName: Before update

Failure to invalidate cache will cause stale data to be served until cache eviction.

SSH Authentication Flow

Client connects with public key
PublicKeyHandler checks if key belongs to a user, stores fingerprint in Permissions.Extensions["pubkey-fp"]
If all public keys fail, KeyboardInteractiveHandler checks allow-keyless setting
AuthenticationMiddleware validates that the public key fingerprint in context matches the actual auth key (gossh doesn't guarantee this)

This double-check prevents a security issue where gossh's PublicKeyCallback doesn't guarantee the approved key is the one actually used.

Task Manager for Long-Running Operations

Repository imports run asynchronously using pkg/task:

tid := "import:" + name
d.manager.Add(tid, func(ctx context.Context) error {
    // Long-running operation
    return nil
})

done := make(chan error, 1)
go d.manager.Run(tid, done)
err := <-done

Tasks are identified by string ID and automatically cleaned up on completion. The manager respects context cancellation.

Repository Path Construction

Always use backend.repoPath() or construct paths consistently:

// Correct
rp := filepath.Join(cfg.DataPath, "repos", repoName+".git")

// Wrong - missing .git suffix
rp := filepath.Join(cfg.DataPath, "repos", repoName)

The .git suffix is mandatory for git to recognize bare repositories.

AGENTS.md