Detailed changes
@@ -0,0 +1,39 @@
+---
+name: builtin-skills
+description: Guide for adding and modifying builtin skills embedded in the Crush binary. Use when creating a new builtin skill, editing an existing one, or understanding how the embedded skill system works (internal/skills/builtin/).
+---
+
+# Builtin Skills
+
+Crush embeds skills directly into the binary via `internal/skills/builtin/`.
+These are always available without user configuration.
+
+## How It Works
+
+- Each skill lives in `internal/skills/builtin/<skill-name>/SKILL.md`.
+- The tree is embedded at compile time via `//go:embed builtin/*` in
+ `internal/skills/embed.go`.
+- `DiscoverBuiltin()` walks the embedded FS, parses each `SKILL.md`, and
+ sets paths with the `crush://skills/` prefix (e.g.,
+ `crush://skills/jq/SKILL.md`).
+- The View tool resolves `crush://` paths from the embedded FS, not disk.
+- User skills with the same name override builtins (last occurrence wins
+ in `Deduplicate()`).
+
+## Adding a New Builtin Skill
+
+1. Create `internal/skills/builtin/<skill-name>/SKILL.md` with YAML
+ frontmatter (`name`, `description`) and markdown instructions. The
+ directory name must match the `name` field.
+2. No extra wiring needed — `//go:embed builtin/*` picks up new
+ directories automatically.
+3. Add a test assertion in `TestDiscoverBuiltin` in
+ `internal/skills/skills_test.go` to verify discovery.
+4. Build and test: `go build . && go test ./internal/skills/...`
+
+## Existing Builtin Skills
+
+| Skill | Directory | Description |
+|-------|-----------|-------------|
+| `crush-config` | `builtin/crush-config/` | Crush configuration help |
+| `jq` | `builtin/jq/` | jq JSON processor usage guide |
@@ -0,0 +1,51 @@
+---
+name: shell-builtins
+description: Guide for adding and modifying shell builtin commands in Crush's embedded shell (internal/shell/). Use when creating a new builtin command, editing an existing one, or understanding how commands are intercepted in the mvdan/sh interpreter.
+---
+
+# Shell Builtins
+
+Crush's shell (`internal/shell/`) uses `mvdan.cc/sh/v3` for POSIX shell
+emulation. Commands can be intercepted before they reach the OS by adding
+**builtins** — functions handled in-process.
+
+## How Builtins Work
+
+Builtins live in `Shell.builtinHandler()` in `internal/shell/shell.go`.
+This is an `interp.ExecHandlerFunc` middleware registered in
+`execHandlers()` **before** the block handler, so builtins run even for
+commands that would otherwise be blocked.
+
+The handler is a switch on `args[0]`. Each case either handles the command
+inline or delegates to a helper function.
+
+## Adding a New Builtin
+
+1. **Add the case** to the switch in `builtinHandler()` in `shell.go`.
+2. **Get I/O from the handler context**, not from `os.Stdin`/`os.Stdout`.
+ This ensures the builtin works with pipes and redirections:
+ ```go
+ case "mycommand":
+ hc := interp.HandlerCtx(ctx)
+ return handleMyCommand(args, hc.Stdin, hc.Stdout, hc.Stderr)
+ ```
+3. **Implement the handler** in its own file (e.g.,
+ `internal/shell/mycommand.go`). The function signature should accept
+ args, stdin, stdout, and stderr:
+ ```go
+ func handleMyCommand(args []string, stdin io.Reader, stdout, stderr io.Writer) error {
+ // args[0] is the command name ("mycommand"), args[1:] are arguments.
+ // Write output to stdout, errors to stderr.
+ // Return nil on success, or interp.ExitStatus(n) for non-zero exit codes.
+ }
+ ```
+4. **Return values**: return `nil` for success, `interp.ExitStatus(n)` for
+ non-zero exit codes. Write error messages to `stderr` before returning.
+5. **No extra wiring needed** — `builtinHandler()` is already registered
+ in `execHandlers()`.
+
+## Existing Builtins
+
+| Command | File | Description |
+|---------|------|-------------|
+| `jq` | `jq.go` | JSON processor using `github.com/itchyny/gojq` |
@@ -44,6 +44,7 @@ require (
github.com/go-git/go-git/v5 v5.17.2
github.com/google/uuid v1.6.0
github.com/invopop/jsonschema v0.13.0
+ github.com/itchyny/gojq v0.12.19
github.com/joho/godotenv v1.5.1
github.com/jordanella/go-ansi-paintbrush v0.0.0-20240728195301-b7ad996ecf3d
github.com/lucasb-eyer/go-colorful v1.4.0
@@ -142,6 +143,7 @@ require (
github.com/gorilla/websocket v1.5.3 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
+ github.com/itchyny/timefmt-go v0.1.8 // indirect
github.com/jackmordaunt/icns/v3 v3.0.1 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/josharian/intern v1.0.0 // indirect
@@ -250,6 +250,10 @@ github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/invopop/jsonschema v0.13.0 h1:KvpoAJWEjR3uD9Kbm2HWJmqsEaHt8lBUpd0qHcIi21E=
github.com/invopop/jsonschema v0.13.0/go.mod h1:ffZ5Km5SWWRAIN6wbDXItl95euhFz2uON45H2qjYt+0=
+github.com/itchyny/gojq v0.12.19 h1:ttXA0XCLEMoaLOz5lSeFOZ6u6Q3QxmG46vfgI4O0DEs=
+github.com/itchyny/gojq v0.12.19/go.mod h1:5galtVPDywX8SPSOrqjGxkBeDhSxEW1gSxoy7tn1iZY=
+github.com/itchyny/timefmt-go v0.1.8 h1:1YEo1JvfXeAHKdjelbYr/uCuhkybaHCeTkH8Bo791OI=
+github.com/itchyny/timefmt-go v0.1.8/go.mod h1:5E46Q+zj7vbTgWY8o5YkMeYb4I6GeWLFnetPy5oBrAI=
github.com/jackmordaunt/icns/v3 v3.0.1 h1:xxot6aNuGrU+lNgxz5I5H0qSeCjNKp8uTXB1j8D4S3o=
github.com/jackmordaunt/icns/v3 v3.0.1/go.mod h1:5sHL59nqTd2ynTnowxB/MDQFhKNqkK8X687uKNygaSQ=
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 h1:BQSFePA1RWJOlocH6Fxy8MmwDt+yVQYULKfN0RoTN8A=
@@ -0,0 +1,277 @@
+package shell
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "os"
+ "strings"
+
+ "github.com/itchyny/gojq"
+ "mvdan.cc/sh/v3/interp"
+)
+
+const jqUsage = `jq - Go implementation of jq (gojq 0.12.19 builtin)
+
+Synopsis:
+ %% echo '{"foo": 128}' | jq '.foo'
+
+Usage:
+ jq [OPTIONS] [FILTER] [FILE...]
+
+Options:
+ -r, --raw-output output raw strings
+ -j, --join-output implies -r with no newline delimiter
+ -c, --compact-output output without pretty-printing
+ -s, --slurp read all inputs into an array
+ -n, --null-input use null as input value
+ -e, --exit-status exit 1 when the last value is false or null
+ -R, --raw-input read input as raw strings
+ --arg name value set a string value to a variable
+ --argjson name value set a JSON value to a variable
+ -h, --help display this help
+`
+
+// handleJQ implements the jq builtin using gojq. It supports a subset of jq
+// flags: -r (raw output), -c (compact output), -s (slurp), -n (null input),
+// -e (exit status), -R (raw input), and --arg name value.
+//
+// Note that this is somewhat of a reimplmentation of the CLI of the glorious
+// github.com/itchyny/gojq, and we'd ideally get the CLI exposed upstream to
+// avoid this falling out of sync.
+func handleJQ(args []string, stdin io.Reader, stdout, stderr io.Writer) error {
+ var (
+ rawOutput bool
+ compact bool
+ slurp bool
+ nullInput bool
+ exitStatus bool
+ rawInput bool
+ joinOutput bool
+ argNames []string
+ argValues []any
+ )
+
+ // Parse flags and extract the query.
+ var queryStr string
+ var fileArgs []string
+ i := 1 // skip "jq"
+ for i < len(args) {
+ arg := args[i]
+ switch {
+ case arg == "-h" || arg == "--help":
+ fmt.Fprint(stdout, jqUsage)
+ return nil
+ case arg == "-r" || arg == "--raw-output":
+ rawOutput = true
+ case arg == "-j" || arg == "--join-output":
+ joinOutput = true
+ rawOutput = true
+ case arg == "-c" || arg == "--compact-output":
+ compact = true
+ case arg == "-s" || arg == "--slurp":
+ slurp = true
+ case arg == "-n" || arg == "--null-input":
+ nullInput = true
+ case arg == "-e" || arg == "--exit-status":
+ exitStatus = true
+ case arg == "-R" || arg == "--raw-input":
+ rawInput = true
+ case arg == "--arg":
+ if i+2 >= len(args) {
+ fmt.Fprintf(stderr, "jq: --arg requires name and value\n")
+ return interp.ExitStatus(2)
+ }
+ argNames = append(argNames, "$"+args[i+1])
+ argValues = append(argValues, args[i+2])
+ i += 2
+ case arg == "--argjson":
+ if i+2 >= len(args) {
+ fmt.Fprintf(stderr, "jq: --argjson requires name and value\n")
+ return interp.ExitStatus(2)
+ }
+ var val any
+ if err := json.Unmarshal([]byte(args[i+2]), &val); err != nil {
+ fmt.Fprintf(stderr, "jq: invalid JSON for --argjson %s: %s\n", args[i+1], err)
+ return interp.ExitStatus(2)
+ }
+ argNames = append(argNames, "$"+args[i+1])
+ argValues = append(argValues, val)
+ i += 2
+ case arg == "--":
+ i++
+ // Remaining args are file arguments.
+ for i < len(args) {
+ fileArgs = append(fileArgs, args[i])
+ i++
+ }
+ continue
+ case strings.HasPrefix(arg, "-") && queryStr != "":
+ fmt.Fprintf(stderr, "jq: unknown option: %s\n", arg)
+ return interp.ExitStatus(2)
+ default:
+ if queryStr == "" {
+ queryStr = arg
+ } else {
+ fileArgs = append(fileArgs, arg)
+ }
+ }
+ i++
+ }
+
+ if queryStr == "" {
+ queryStr = "."
+ }
+
+ query, err := gojq.Parse(queryStr)
+ if err != nil {
+ fmt.Fprintf(stderr, "jq: %s\n", err)
+ return interp.ExitStatus(3)
+ }
+
+ opts := []gojq.CompilerOption{
+ gojq.WithEnvironLoader(os.Environ),
+ }
+ if len(argNames) > 0 {
+ opts = append(opts, gojq.WithVariables(argNames))
+ }
+
+ code, err := gojq.Compile(query, opts...)
+ if err != nil {
+ fmt.Fprintf(stderr, "jq: %s\n", err)
+ return interp.ExitStatus(3)
+ }
+
+ // Build input values.
+ inputs, err := readInputs(stdin, fileArgs, nullInput, rawInput, slurp)
+ if err != nil {
+ fmt.Fprintf(stderr, "jq: %s\n", err)
+ return interp.ExitStatus(2)
+ }
+
+ var lastFalsy bool
+ for _, input := range inputs {
+ iter := code.Run(input, argValues...)
+ for {
+ v, ok := iter.Next()
+ if !ok {
+ break
+ }
+ if err, ok := v.(error); ok {
+ fmt.Fprintf(stderr, "jq: %s\n", err)
+ return interp.ExitStatus(5)
+ }
+ if exitStatus {
+ lastFalsy = v == nil || v == false
+ }
+ if err := writeValue(stdout, v, rawOutput, compact, joinOutput); err != nil {
+ return err
+ }
+ }
+ }
+
+ if exitStatus && lastFalsy {
+ return interp.ExitStatus(1)
+ }
+ return nil
+}
+
+// readInputs reads JSON (or raw) input values from stdin or files.
+func readInputs(stdin io.Reader, files []string, nullInput, rawInput, slurp bool) ([]any, error) {
+ if nullInput {
+ return []any{nil}, nil
+ }
+
+ var readers []io.Reader
+ if len(files) > 0 {
+ for _, f := range files {
+ file, err := os.Open(f)
+ if err != nil {
+ return nil, err
+ }
+ defer file.Close()
+ readers = append(readers, file)
+ }
+ } else {
+ readers = []io.Reader{stdin}
+ }
+
+ var vals []any
+ for _, r := range readers {
+ data, err := io.ReadAll(r)
+ if err != nil {
+ return nil, err
+ }
+
+ if rawInput {
+ lines := strings.Split(string(data), "\n")
+ if slurp {
+ vals = append(vals, strings.Join(lines, "\n"))
+ } else {
+ for _, line := range lines {
+ if line != "" || !slurp {
+ vals = append(vals, line)
+ }
+ }
+ }
+ continue
+ }
+
+ // Decode potentially multiple JSON values from the stream.
+ dec := json.NewDecoder(strings.NewReader(string(data)))
+ var streamVals []any
+ for {
+ var v any
+ if err := dec.Decode(&v); err != nil {
+ if err == io.EOF {
+ break
+ }
+ return nil, fmt.Errorf("parse error: %w", err)
+ }
+ streamVals = append(streamVals, v)
+ }
+
+ if slurp {
+ vals = append(vals, streamVals)
+ } else {
+ vals = append(vals, streamVals...)
+ }
+ }
+
+ if len(vals) == 0 {
+ return []any{nil}, nil
+ }
+ return vals, nil
+}
+
+// writeValue writes a single jq output value.
+func writeValue(w io.Writer, v any, raw, compact, join bool) error {
+ if raw {
+ if s, ok := v.(string); ok {
+ if _, err := fmt.Fprint(w, s); err != nil {
+ return err
+ }
+ if !join {
+ _, err := fmt.Fprint(w, "\n")
+ return err
+ }
+ return nil
+ }
+ }
+
+ var bs []byte
+ var err error
+ if compact {
+ bs, err = gojq.Marshal(v)
+ } else {
+ bs, err = json.MarshalIndent(v, "", " ")
+ }
+ if err != nil {
+ return err
+ }
+ if _, writeErr := w.Write(bs); writeErr != nil {
+ return writeErr
+ }
+ _, err = fmt.Fprint(w, "\n")
+ return err
+}
@@ -244,6 +244,25 @@ func (s *Shell) blockHandler() func(next interp.ExecHandlerFunc) interp.ExecHand
}
}
+func (s *Shell) builtinHandler() func(next interp.ExecHandlerFunc) interp.ExecHandlerFunc {
+ return func(next interp.ExecHandlerFunc) interp.ExecHandlerFunc {
+ return func(ctx context.Context, args []string) error {
+ if len(args) == 0 {
+ return next(ctx, args)
+ }
+
+ // Builtins.
+ switch args[0] {
+ case "jq":
+ hc := interp.HandlerCtx(ctx)
+ return handleJQ(args, hc.Stdin, hc.Stdout, hc.Stderr)
+ default:
+ return next(ctx, args)
+ }
+ }
+ }
+}
+
// newInterp creates a new interpreter with the current shell state
func (s *Shell) newInterp(stdout, stderr io.Writer) (*interp.Runner, error) {
return interp.New(
@@ -307,6 +326,7 @@ func (s *Shell) execStream(ctx context.Context, command string, stdout, stderr i
func (s *Shell) execHandlers() []func(next interp.ExecHandlerFunc) interp.ExecHandlerFunc {
handlers := []func(next interp.ExecHandlerFunc) interp.ExecHandlerFunc{
+ s.builtinHandler(),
s.blockHandler(),
}
if useGoCoreUtils {
@@ -0,0 +1,97 @@
+---
+name: jq
+description: Process, filter, transform, generate, and construct JSON data using the jq command-line processor. Crush includes a built-in jq implementation (powered by gojq) available in the bash tool. Use when the user needs to query, filter, reshape, extract, create, or construct JSON — including API responses, config files, log output, or any structured data. Also use when generating JSON from scratch with jq -n, or when helping users write or debug jq expressions.
+---
+
+# jq — Built-in JSON Processor
+
+Crush ships a built-in `jq` command (via `github.com/itchyny/gojq`) available
+in the bash tool. No external binary is required.
+
+## Supported Flags
+
+| Flag | Description |
+|------|-------------|
+| `-r`, `--raw-output` | Output strings without quotes |
+| `-j`, `--join-output` | Like `-r` but no trailing newline |
+| `-c`, `--compact-output` | One-line JSON output |
+| `-s`, `--slurp` | Read all inputs into an array |
+| `-n`, `--null-input` | Use `null` as input (ignore stdin) |
+| `-e`, `--exit-status` | Exit 1 if last output is `false` or `null` |
+| `-R`, `--raw-input` | Read each line as a string, not JSON |
+| `--arg name value` | Bind `$name` to a string value |
+| `--argjson name value` | Bind `$name` to a parsed JSON value |
+
+File arguments after the filter are also supported: `jq '.foo' file.json`.
+
+## Differences from Standard jq
+
+The built-in uses gojq, which is a pure-Go jq implementation. Key
+differences:
+
+- **No object key ordering** — keys are sorted by default; `keys_unsorted`
+ and `-S` are unavailable.
+- **Arbitrary-precision integers** — large integers keep full precision
+ (addition, subtraction, multiplication, modulo, division when divisible).
+- **String indexing** — `"abcde"[2]` returns `"c"`.
+- **Not supported** — `--ascii-output`, `--seq`, `--stream`,
+ `--stream-errors`, `-f`/`--from-file`, `--slurpfile`, `--rawfile`,
+ `--args`, `--jsonargs`, `input_line_number`, `$__loc__`, some regex
+ features (backreferences, look-around).
+- **YAML** — gojq supports `--yaml-input`/`--yaml-output` but the
+ built-in does not currently expose these flags.
+
+## Common Patterns
+
+Extract a field:
+```sh
+echo '{"name":"crush"}' | jq '.name'
+```
+
+Filter an array:
+```sh
+echo '[1,2,3,4,5]' | jq '[.[] | select(. > 3)]'
+```
+
+Reshape objects:
+```sh
+echo '{"first":"Ada","last":"Lovelace"}' | jq '{full: (.first + " " + .last)}'
+```
+
+Use variables:
+```sh
+echo '{}' | jq --arg host localhost --argjson port 8080 '{host: $host, port: $port}'
+```
+
+Slurp multiple JSON values:
+```sh
+echo '{"a":1}{"b":2}' | jq -s '.'
+```
+
+Compact output for piping:
+```sh
+echo '{"a":1}' | jq -c '.a += 1'
+```
+
+Raw string output:
+```sh
+echo '["one","two","three"]' | jq -r '.[]'
+```
+
+Process a file:
+```sh
+jq '.dependencies | keys' package.json
+```
+
+Null input for constructing JSON:
+```sh
+jq -n --arg msg hello '{"message": $msg}'
+```
+
+## Tips
+
+- Pipe jq output to other commands: `jq -r '.url' data.json | xargs curl`
+- Chain filters with `|` inside the expression, not shell pipes.
+- Use `try` to suppress errors on missing keys: `jq 'try .foo.bar'`
+- Use `// "default"` for fallback values: `jq '.name // "unknown"'`
+- Use `@csv`, `@tsv`, `@base64`, `@html`, `@uri` for format strings.
@@ -424,6 +424,19 @@ func TestDiscoverBuiltin(t *testing.T) {
}
}
require.True(t, found, "crush-config builtin skill not found")
+
+ var foundJQ bool
+ for _, s := range discovered {
+ if s.Name == "jq" {
+ foundJQ = true
+ require.Equal(t, "crush://skills/jq/SKILL.md", s.SkillFilePath)
+ require.Equal(t, "crush://skills/jq", s.Path)
+ require.NotEmpty(t, s.Description)
+ require.NotEmpty(t, s.Instructions)
+ require.True(t, s.Builtin)
+ }
+ }
+ require.True(t, foundJQ, "jq builtin skill not found")
}
func TestDeduplicate(t *testing.T) {