Implement lazy-load git-bug cache with idle timeout

Amolith opened

Currently, every web request to view bugs opens a new cache, rebuilds identities and bugs, then closes it (pkg/web/webui_bugs.go:327-333, 415-421). As the number of bugs and identities grows, each request becomes more expensive.

Implement a lazy-load cache manager with idle timeout pattern:

  1. Lazy open: Open bug cache only when first requested for a specific repo
  2. Keep in memory: Hold the cache in memory while actively used
  3. Idle timeout: Close cache after 10 minutes of inactivity
  4. Hook integration: On push to refs/bugs/* or refs/identities/*:
    • Close and delete the in-memory cache if it exists
    • Cancel any pending idle timeout
    • Next request will rebuild from scratch

New file: pkg/backend/gitbug_cache.go

Create GitBugCacheManager struct:

  • sync.Mutex for concurrent access
  • map[string]*gitBugCacheEntry keyed by repo name
  • Each entry contains:
    • *cache.RepoCache
    • *time.Timer for idle timeout (10 minutes)
    • time.Time for last accessed tracking

Methods:

  • GetOrOpen(ctx context.Context, repo proto.Repository) (*cache.RepoCache, error)
    • Returns cached instance if exists and resets idle timer
    • Opens new cache if not exists, starts idle timer
    • Timer callback should close cache and remove from map
  • InvalidateCache(repoName string) error
    • Closes the cache if it exists (rc.Close())
    • Deletes cache entry from map
    • Cancels the idle timer
  • Close() error
    • Cleanup all caches on shutdown
    • Call from server shutdown sequence

Modify pkg/backend/backend.go

  • Add gitBugCache *GitBugCacheManager field to Backend struct (around line 21)
  • Initialize in New() function (around line 26-42)
  • Export accessor method: GitBugCache() *GitBugCacheManager

Modify pkg/backend/hooks.go

  • Add to PostReceive() or Update() (lines 18-87):
    • Parse hook args to detect refs/bugs/* or refs/identities/* updates
    • Use strings.HasPrefix(arg.RefName, "refs/bugs/") or strings.HasPrefix(arg.RefName, "refs/identities/")
    • Call d.gitBugCache.InvalidateCache(repo) when detected
    • Handle errors gracefully (log but don't block push)

Modify pkg/web/webui_bugs.go

  • In repoBugs() handler (line 316):
    • Replace openBugCache() call (line 327) with be.GitBugCache().GetOrOpen(ctx, repo)
    • Remove defer rc.Close() (line 333)
  • In repoBug() handler (line 402):
    • Replace openBugCache() call (line 415) with be.GitBugCache().GetOrOpen(ctx, repo)
    • Remove defer rc.Close() (line 421)
  • Keep openBugCache() as private helper for the cache manager to use

Server shutdown

  • Add be.GitBugCache().Close() to shutdown sequence (likely in cmd/)

  • Memory efficient: Only repos with active bug viewing are cached

  • Performance: Subsequent requests are fast (no rebuild required)

  • Automatic cleanup: Idle repos are removed from memory after 10min

  • Real-time updates: Hook integration keeps cache fresh on push

  • Scalable: Works efficiently with 1 repo or 1000 repos

  • Idle timeout pattern: pkg/daemon/conn.go:54-106 (serverConn with updateDeadline)

  • Ref prefix matching: pkg/web/webui_git.go uses strings.HasPrefix for refs/

  • Task manager pattern: pkg/task/manager.go for goroutine management

  • Existing cache: pkg/backend/cache.go for LRU cache example

References: bug-3823b04 Co-authored-by: Crush crush@charm.land

Amolith closed the bug