nucleic/nucleic
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
0c81858bcaa663ff230028d32872a6d506bc2b30
nucleic/docs/NASH.md
T

48 KiB
Raw Blame History

nash — the Nucleic Agent Shell

Status: M0–M2 complete; M3's image + host work is implemented via narOS N2 (NAROS.md) — its gates are pending. M3 state: the divert is baked (naros-base, since narOS N1), naros-agent is built as the N2 image layer, and the §7.2 Swift swap is in — Project.swift pin → naros-agent:26.07, exec argv/passwd prefer nash, the §7.6 probe gained the ID=naros fast path, and the legacyShell rollback lever is live (ContainerServiceSettings.legacyShell → execs use the preserved bash.real and every hook env carries NUCLEIC_NASH_DISABLE=1). Still open before the forcing is shipped: the M3 gate set — ≥99% corpus parity replayed in-image, <3% overhead, one clean dogfood week, zero unresolved fallback regressions. Runner surface (§7.3) implemented via narOS N3: nucleic-runner rebases FROM naros-agent (nash forced by inheritance), and — because runner sessions are host runs (RunSpec.container = nil, no shim seeding) — ClaudeCodeBackend now detects a narOS host (CommandInterceptor.hostIsNarOS, /etc/os-release ID=naros), registers the git/gh/command/shell report routes for those runs, and merges hookEnv (tracer off, legacyShell lever honored) into the host spawn env pointed at the loopback approval server — so nash events flow to the feed, and git/gh observation on the runner arrives via observeShellEvent's argv convergence rather than shims. Codex/Grok host runs still start no approval server; wiring them is a follow-up. M4's Linux-VM spool-drain (§6.2, §7.4) is implemented; N4-baked nash in the guest is the remaining dependency. A VM-resident nash can't reach the host control socket, so it spools batches to /var/spool/nucleic-nash/*.jsonl; nucleic-linux-agent gained a non-streaming shell-events op (agent.c) that reads those files, returns them base64 as NDJSON, and truncates what it delivered — at-least-once (files are unlinked only after a successful send). The agent also setenvs NUCLEIC_SHELL_SPOOL + NUCLEIC_SHELL_CAPTURE=tap in its own environment so every exec'd shell (nash under the N4 /bin/sh→nash divert) spools by inheritance — inert on the current Ubuntu guest where the shell isn't nash. Host side: MacVMAgentWire.ShellEvents + MacVMEngine.drainShellEvents decode the reply (NDJSON split skips any mid-write partial line), and ClaudeCodeBackend.handleLinuxVMExecCall polls the drain after every manager.run, feeding each batch through the existing parseShellBatch → observeShellEvent seam — the same pipeline as the container surface, with the backend's session id authoritative (as in handleShellReport). No agent.c exec swap and no provisioner changes (nash forcing is narOS N4's baked divert). Verified: guest loopback tests (ping/exec unchanged, plus drain of two spool files, truncate-on-drain, non-.jsonl skip, and missing-dir → empty), and Swift ShellReportParsingTests 7/7 incl. base64-NDJSON split with a partial-trailing-line skip. Live linux_vm_exec feed events await narOS N4 baking nash into the guest rootfs. M5's macOS-VM path (§7.5) is implemented behind the nash-binary gate; the universal nash build (§8) is the remaining dependency. The macOS guest agent (NucleicVMAgent) gained the same shell-events drain op — VMAgentCore.ShellSpool reads ~/Library/Caches/nucleic-nash/*.jsonl (per-user, since the agent runs as agent, not root), the Ops+ShellDrain handler returns them base64/NDJSON and truncates after a successful send (at-least-once), and AgentListener routes it alongside exec. Ops+Exec now runs commands under /usr/local/bin/nash -lc when nash is installed — falling back to /bin/zsh -c when it's absent, when NUCLEIC_NASH_DISABLE=1, or when the host asks shell:"zsh" (the §7.5 escape hatch) — and points nash at the spool via env. Host side: the M4 drain helper generalized to drainVMShellEvents and is now polled after mac_vm_exec too (the drain wire is guest-OS-agnostic, so it needed no new host protocol code). Provisioning (provision-macos-guest.sh) installs a staged nash, mirrors the toolchain PATH into /etc/profile.d for nash -l, and sets the agent login shell — all guarded to a clean skip when no nash binary is staged. Verified: the NucleicVMAgent package builds and VMAgentCoreTests 11/11 incl. the drain (concatenate + non-.jsonl skip, missing-dir → empty, oversize-file-left-for-next-poll). Because the swap is gated on the binary's presence and no nash.yml darwin universal artifact exists yet, the macOS VM stays on zsh until that binary ships and provisioning stages it — matching M5's "gated, last" posture (activation waits on the dogfood zsh-ism metric). M2: the Gate gained redirect/cmdsub/pipe hooks. nash reads back the affected byte range of every regular-file redirection (> head, >> delta, </2>/&> head), captures heredoc/here-string inline, captures $(…) output, and tees every a | b pipe link — all bounded (64 KiB), redacted in-guest, hashed, previewed base64. Transparency proven, not asserted: regular files keep the real fd (a dd seek= write is byte-identical to bash); a tapped pipe passes binary byte-for-byte (md5 matches bash); yes | head gets SIGPIPE and terminates (no hang); a 10 MB stream reports the full total with bounded memory; special files (/dev/null, ttys, fifos) are metadata-only. The pipe tee is only installed when observation is active — no extra pipe/thread on the unobserved fast path. Verified: 16/16 Rust integration tests, corpus 100% with all taps active (redirect + cmdsub + 32 pipe events captured, output byte-identical to bash), brush compat suite unchanged (1684/6). Swift: redirect/cmdsub/pipe decode into ShellReportCall (48/48 tests), logged via CMDTRACE pending the §9.5 feed rows.

Prior milestone detail: M0: vendored brush (brush-shell-v0.4.0) at third_party/brush/; replay harness at shell/corpus/ reports 94/94 = 100% parity vs bash after fixing fork divergence D1 (shell/corpus/DIVERGENCES.md). M1 (Rust half): the Gate trait is live in brush-core (allow-all, verdict-shaped, §4.2 as locked); nash-observe emits exec/cd/export-names/fallback/dropped events with near-live batching over unix-socket HTTP → TCP → JSONL spool; the -c parse-failure fallback re-execs preserved real bash after posting a fallback event; NUCLEIC_NASH_DISABLE=1 kill switch works. Verified: 8/8 end-to-end observation tests (shell/nash/tests/observe.rs), corpus 100% with capture off AND on, brush compat suite unchanged vs baseline (1684/6/377/38 — environmental failures only). M1 (Swift half) also complete: /shell-event route + ShellReportCall decoding (parseShellBatch, 3 tests) → registerShellReport in all four backends → ConflictCoordinator → AppStore.observeShellEvent (exec events converge onto the command/git/gh feeds tagged source: .nash; cd/export/fallback/dropped CMDTRACE-logged until M2's feed rows); hookEnv now wires NUCLEIC_SHELL_HOOK_URL + NUCLEIC_SHELL_CAPTURE always (tracer demoted to opt-in legacy); linux_container + monitor execs use CommandInterceptor.agentShellArgv — a probe that prefers /usr/local/bin/nash and degrades to /bin/sh (§7.6), so the switch activates the moment nash lands in an image or seed. Wire contract cross-checked against a real nash spool batch. Full package builds; MCPApprovalServer suites 33/33. Remaining for M3: nash in the sandbox image (divert) and boot-time seeding (§7.1, §8) — image-side delivery now rides narOS (NAROS.md): the planned v8 sandbox image is superseded by naros-agent, whose first stable release (narOS milestone N2) is the M3 vehicle.

nash ("Nucleic agent shell") is a Bourne/bash-compatible shell, forked in Rust, whose job is to make every shell action an agent takes observable by construction. Where the current shim system watches two binaries (git, gh) and an opt-in bash DEBUG-trap tracer watches command metadata, nash sits at the choke point itself: it is the shell, so it sees every simple command, every pipeline, and — the new capability — the data flowing through redirection operators and pipes, which it taps and passes through unmodified. It is the logical end-state of the command-shim system: instead of shimming individual tools ahead of PATH, Nucleic controls the interpreter that launches all of them.

This doc locks down: the fork base, the crate/repo layout, the interception design (exec events, redirection taps, pipe tees), the event schema and transport, host-side integration, how nash becomes the forced default shell on every surface (sandbox containers, the control/runner container, MCP linux_container, Linux VM, macOS VM), build/distribution, testing, rollout phases, and risks.

Related docs: NAROS.md (the distro that bakes and forces nash on every Linux surface — narOS v1 is the M3 vehicle; §7.1/§7.4/§7.6/§8 below now deliver through it), VSOCK_CONTROL_PLANE.md (transport this rides on), CONTAINER_ISOLATION.md, LINUX_VM.md, MACOS_VM.md, OBSERVABILITY_AND_TESTING.md (redaction posture), LOCKING.md (consumer of git observations).

Locked decisions

Decision Choice
Fork base brush (brush_core::Shell embedded; vendored subtree fork)
Role Hooks now, enforce later — the fork hook is verdict-shaped (Gate), but v1 always allows; enforcement is a later per-surface activation (§4.2)
Data-flow retention Persist to transcript — redirect/pipe events (incl. redacted previews) join the session transcript/GRDB as durable audit history, with retention caps (§9.4)
Compat posture Auto-fallback + telemetry — on parse failure, re-exec preserved real bash and emit a fallback event; agents never break (§4.1)
Event delivery Near-live batching — ~500 ms / 200-event / on-exit flushes over the existing POST pipeline; no persistent streaming channel in v1 (§6.1)
Forced surfaces Sandbox, runner, linux_container, Linux VM; macOS VM last and gated (§7.5). User's real Mac excluded
Fork home In-repo subtree — shell/ workspace + third_party/brush/ subtree in this repo; Rust CI isolated from Swift CI (§3, §8)
Update channel Hybrid — image-baked baseline + host boot-time seed override of a newer binary via the rootfs file-copy path (§8)
Capture depth Redirections, pipes, and command-substitution results; plain un-redirected stdout/stderr stays untapped (transcript already has it) (§5.3)
Env mutations Names only — export/declare events carry variable names, never values (§5.4)
Remote sync Full event sync — the complete shell-event stream (incl. redacted previews) syncs to the iPhone over the E2EE channel, with phone-side retention caps (§9.7)
Shim sunset Coexist + dedupe through M5; locks/autoship flip to nash-driven and shims deleted at M6 after a full clean dogfood cycle (§11)
M3 forcing gates ≥99% corpus parity vs bash, <3% wall-clock overhead, one clean dogfood week (§11)
UI surface Feed rows in the existing Control Panel + click-through detail popover; no dedicated panel in this plan (§9.5)

1. Goals and non-goals

Goals

  1. Total command visibility. Every simple command executed by an agent shell — argv, resolved path, cwd, exit code, duration, pipeline position, nesting (subshell / function / command substitution) — reported to the host, always on, with no reliance on bash's BASH_ENV/DEBUG-trap fragility.
  2. Data-flow visibility. Bounded, redaction-aware capture of bytes crossing redirection operators (>, >>, <, 2>, &>, here-docs/strings, >|, fd dups) and pipes (a | b), passed through byte-for-byte to preserve semantics.
  3. Transparency. Agents and their tools must not behave differently under nash. Existing scripts, .bashrc-style init, and the Claude Code / Codex Bash tool must run unchanged. Observation is best-effort and may never block, slow measurably, or alter a command's outcome. On any internal nash failure, the command still runs.
  4. Forced default everywhere Nucleic controls the OS image: sandbox containers, the cloud runner container, agent-created linux_containers, the Linux VM, the macOS VM.
  5. Subsume the tracer, converge the shims. The BASH_ENV command tracer (CommandInterceptor.swift) is retired once nash lands; the git/gh shims remain initially (they feed locks/autoship) and are retired once nash-native classification is proven.

Non-goals

  • The user's real Mac (host zsh surfaces: TerminalPanel.swift, BuildRunPanel.swift, host_exec via /bin/zsh -lc, LoginShellPATH.swift). We do not change the operator's machine shell. host_exec stays on the user's shell; its guardrail remains HostExecPolicy.swift.
  • An interactive daily-driver shell. nash runs non-interactive agent commands; interactive features (highlighting, completion) are stripped from the build.
  • A security boundary in v1. nash ships as an observability layer; enforcement stays where it is today (approval server, HostExecPolicy, PreToolUse hooks). A determined command can still exec /usr/bin/dash.real; we record that it did. The hook API is deliberately verdict-shaped so enforcement can be activated later without re-patching the fork (§4.2), but that activation is out of scope for this plan's milestones.

2. Fork base: brush

We do not port the C Almquist/dash lineage to Rust; we fork [brush] (https://github.com/reubeno/brush), a bash/POSIX-compatible shell in Rust:

  • MIT-licensed, actively developed (v0.4.0, May 2026), validated against bash with ~1,700 compatibility tests; bash major features (set -e/-u, pipefail, traps incl. ERR, coprocesses) implemented.
  • Designed for embedding: brush_core::Shell is an embeddable engine; the workspace splits brush-parser (syntax), brush-core (interpreter: interp.rs, commands.rs, processes.rs, openfiles.rs, sys/), brush-builtins, brush-shell (CLI binary).
  • Builds static for x86_64/aarch64 Linux (musl) and native macOS — exactly our matrix.

Alternatives considered and rejected:

Option Why not
Fork dash/busybox-ash (C) Not Rust; instrumenting C fd plumbing is the bug-prone path this project exists to avoid
Shell from scratch in Rust Years of compat work brush has already done (and tested)
nsh / rusty_bash / others Much lower bash compatibility and maturity than brush
Pure wrapper around real bash (pty + strace/eBPF) No structural knowledge of redirections; eBPF unavailable in our unprivileged container guests; fragile

Fork mechanics. Vendor the fork as a git subtree at third_party/brush/ (matching the existing third_party/containerization/ convention), tracking a fork repo (github.com/abkslm/brush) so upstream merges stay routine. Patches to brush-core are kept minimal and hook-shaped (see §4.2) to keep the merge surface small; everything else lives in our own crates.

Naming. nash stands for Nucleic agent shell. Beyond naming the thing for what it is, the acronym has a useful side effect: it avoids the name of the historical Almquist shell — ash, still shipped by busybox, with Debian's dash as its descendant — which a bare "agent shell" contraction would have collided with. No binary named nash ships in our images or VM rootfs, so there is no on-disk conflict. The binary installs as /usr/local/bin/nash, nash --version reports nash (Nucleic agent shell, brush fork) x.y.z, and it sets NUCLEIC_NASH=1 in its own environment so scripts/tests can detect it. Nothing in the design depends on the name; renaming remains cheap if prior art ever bites.

3. Repo layout

third_party/brush/            # vendored fork (git subtree of abkslm/brush)
shell/                        # new Cargo workspace
  Cargo.toml
  nash/                       # bin crate: CLI entry, arg parsing (-c/-lc/-s/files),
                              #   config env, bash-fallback, embeds brush_core::Shell
  nash-observe/               # lib crate: event model, taps, batching, spool,
                              #   transport (HTTP over unix socket / TCP), redaction
os/                           # narOS build (NAROS.md) — bakes nash + the divert into
                              #   the naros-agent image (§7.1), superseding a
                              #   nucleic-sandbox Dockerfile change
.github/workflows/nash.yml    # build + publish (see §8)
docs/NASH.md                  # this doc

shell/ is intentionally outside Sources/ (it's not Swift) and outside guest/ (it ships to containers and VMs). The Cargo workspace pins the vendored brush via path = "../third_party/brush/brush-core" dependencies.

4. Shell architecture

4.1 The binary

nash is a thin binary embedding brush_core::Shell:

  • Accepts the bash-compatible invocation matrix we actually use: nash -c <cmd>, nash -lc <cmd>, nash <script> [args], stdin scripts, -e/-x/-o pipefail passthrough. Login (-l) sources /etc/profile + profile.d so the existing nucleic-path.sh PATH setup keeps working. -c follows bash's rule that short options keep parsing after it and the command is the first operand, so nash -c -l <cmd> / nash -c -e -u <cmd> run <cmd> (the fork normalizes the ordering before clap sees it; clap alone would reject them for a missing -c value).
  • Reads observation config from env at startup (all optional — absent config means "run silently as a plain shell"):
    • NUCLEIC_SHELL_HOOK_URL — event endpoint (http://…/shell-event)
    • NUCLEIC_SHELL_SOCKET — unix socket to speak HTTP over directly (default /run/nucleic/control.sock), preferred over the URL when present
    • NUCLEIC_HOOK_TOKEN, NUCLEIC_SESSION_ID — existing auth/session vars
    • NUCLEIC_SHELL_SPOOL — spool dir for offline batches (default /var/spool/nucleic-nash)
    • NUCLEIC_SHELL_CAPTURE — off | meta | tap (default tap; see §5)
    • NUCLEIC_NASH_DISABLE=1 — kill switch: immediately exec the real bash (NUCLEIC_REAL_BASH, default /usr/bin/bash.real, falling back to /bin/bash) with identical argv. This is the "get out of jail" lever for compat emergencies.
  • Parse-failure fallback: if brush-parser rejects input that looks like valid bash (parse error on -c input or a sourced file), nash emits a fallback event and re-execs the real bash with the same argv, so the agent's command still succeeds. This converts brush compat gaps from breakage into telemetry we can fix.

4.2 Patch surface inside brush-core (the fork proper)

Kept deliberately tiny — a Gate trait injected into the shell, with call sites at the four choke points. Everything else (transport, batching, redaction) is in nash-observe.

The trait is verdict-shaped by design (locked decision: hooks now, enforce later): every pre-execution hook returns a Verdict, so the same patch supports pure observation today and policy enforcement later. In v1 the nash-observe implementation returns Verdict::Allow unconditionally and synchronously — there is no host round-trip on the command path, and transport stays fire-and-forget. Activating enforcement later is a per-surface change in nash-observe (a Hold verdict awaiting a synchronous POST /shell-gate decision, with explicit fail-open/fail-closed timeout semantics), not a fork re-patch; the endpoint name is reserved but not built in this plan.

// brush-core addition (new file gate.rs, ~1 trait + 2 enums + allow-all default):
pub enum Verdict { Allow(ExecToken), Deny { status: u8, message: String } }
                                     // Hold is modeled as Allow/Deny resolved
                                     // inside the hook; v1 never blocks.
pub trait Gate: Send + Sync {
    fn on_exec(&self, ev: ExecStart) -> Verdict;            // before spawn
    fn on_exit(&self, tok: ExecToken, status: ExecEnd);     // after wait
    fn on_redirect(&self, ev: RedirectEvent) -> RedirTap;   // during redir setup
    fn on_pipe(&self, ev: PipeEvent) -> Option<TeeFds>;     // pipeline wiring
}

Call sites (file names from brush-core main):

Choke point brush-core location What the hook sees/does
Simple-command spawn/wait commands.rs / processes.rs (external cmds), builtin dispatch in interp.rs argv, resolved path, cwd, pipeline index, nesting depth; later pid, exit status, duration
Redirection setup interp.rs + openfiles.rs (where redirs become fd mappings) operator, target (path/fd/heredoc), open mode; may substitute a tapped fd (§5.2)
Pipeline wiring interp.rs pipe construction may interpose a tee pipe (§5.3)
Word-level facts interp.rs (cd, export, source, function def) state-change events for the feed

The Gate default impl is allow-all/no-op, so the vendored fork remains mergeable with upstream and even upstreamable (a generic hook/gate API is a plausible upstream contribution, which would shrink our fork to zero).

5. Observation semantics

5.1 Exec events (always, capture ≥ meta)

One event per simple command (external or builtin), including inside functions, subshells, command substitutions, and sourced files:

  • argv (post-expansion), resolvedPath, cwd, pid, exitCode, durationMs
  • pipeline: {id, index, len} when part of a | b | c
  • nesting: {depth, kind} (top, subshell, cmdsub, function:NAME, source:PATH)
  • This strictly supersedes the bash tracer's {line, cwd, exitCode, durationMs} — richer (structured argv vs. raw line) and reliable (no BASH_ENV inheritance games).

5.2 Redirection taps (capture = tap)

The user-visible requirement: intercept data flowing through redirection operators to observe it, then pass it through as expected. Two mechanisms, chosen per target to guarantee transparency:

(a) Regular-file redirections → open-real-fd + bounded read-back (default). For > file, >> file, < file, 2> file: nash opens the real file and hands the child the real fd — semantics (seekability, O_APPEND atomicity, fstat identity, tools that lseek their output) are untouched by construction. Observation happens around it:

  • record {operator, path, mode, sizeBefore} at setup;
  • after the command completes, record sizeAfter and read back the affected range (for >: 0..min(size, CAP); for >>: sizeBefore..sizeBefore+CAP; for <: head of the file), attaching bytes, truncated, sha256, and a preview (first CAP bytes, base64) to the event.
  • CAP default 64 KiB per redirection, configurable; binary data detected and previewed as hex head. Special files (/dev/null, /dev/tty, sockets, device nodes) are metadata-only, never read back.

This is chosen over inline tee-ing for files because a tee pipe would silently break any program that seeks on its redirected fd — an unacceptable transparency violation for a default shell.

(b) Stream redirections → inline tee. Where the data is already a stream — pipes between commands, here-docs/here-strings, process substitution <(…)/>(…), and fd dups onto pipes — nash interposes: child gets a pipe; an async task (brush-core is tokio-based) copies bytes through to the real destination while mirroring the first CAP bytes and a running count/hash into the event buffer. Backpressure is preserved (the tee task copies, never buffers unboundedly); if the tee task dies, the copy loop degrades to a raw splice-style passthrough and the event is flagged tapFailed.

5.3 Pipe tees and command substitution

a | b reports one pipe event per link: {pipelineId, fromIndex, toIndex, bytes, truncated, preview, sha256} via mechanism (b). This surfaces what the transcript never shows today — the intermediate data between stages (curl … | sh becomes visible).

Command substitution is captured; plain stdio is not (locked decision). $(…) and backtick results are invisible to the agent transcript yet routinely carry decisions — tokens, branch names, resolved paths — so nash records a cmdsub event with a bounded, redacted preview of the substituted text (the shell already holds the full result in memory; no extra tee needed). Un-redirected stdout/stderr of ordinary commands is not tapped: the agent CLI transcript already records it, and double-capturing would roughly double event volume for no new signal.

5.4 Caps, redaction, volume

  • Per-event preview cap (64 KiB), per-command total capture cap (256 KiB), per-batch cap (1 MiB); beyond caps → counts/hashes only, truncated: true.
  • Redaction runs inside nash before bytes leave the guest: the same posture as OBSERVABILITY_AND_TESTING.md — pattern-based masking of obvious credential shapes (bearer/PAT/AWS-style tokens, PRIVATE KEY blocks) in previews; env vars are never captured wholesale (exec events carry argv, not environment).
  • Environment mutations are names only (locked decision): export/declare/unset produce events carrying the variable name (export AWS_SECRET_ACCESS_KEY) and never the value — the name alone is the signal that a credential-shaped variable was set, with zero secret-value risk. Host-side classification can flag known-sensitive names in the feed.
  • Everything is best-effort: full buffers → drop-oldest with a dropped counter event.

5.5 Failure doctrine

Observation must never break work. Concretely: no hook may propagate an error into command execution; transport is fire-and-forget with short timeouts (inherit the shims' 1.5–2 s budget) and disk spool fallback; panics in nash-observe are caught at the hook boundary; and the kill switch (NUCLEIC_NASH_DISABLE) plus the preserved real bash mean any regression has a one-env-var mitigation while we fix forward.

6. Event schema and transport

6.1 Batch shape

Extends the existing /command-event family with a new endpoint POST /shell-event (bearer-token gated, 202-always like /git-event — see MCPApprovalServer.swift:2868):

{
  "type": "shell-batch",
  "source": "nash",
  "sessionId": "…",            // NUCLEIC_SESSION_ID
  "shellId": "…",              // random per nash process; correlates events
  "parentShell": "…|null",     // shellId inherited via env → process tree of shells
  "events": [
    { "kind": "exec", "seq": 1, "ts": 1721…, "argv": ["git","push"],
      "resolvedPath": "/usr/local/bin/git", "cwd": "/workspace", "pid": 412,
      "exitCode": 0, "durationMs": 812,
      "pipeline": null, "nesting": {"depth": 0, "kind": "top"} },
    { "kind": "redirect", "seq": 2, "cmdSeq": 1, "op": ">>",
      "target": {"path": "/workspace/out.log"}, "mode": "append",
      "bytes": 1832, "truncated": false, "sha256": "…", "previewB64": "…" },
    { "kind": "pipe", "seq": 3, "pipelineId": "p1", "fromIndex": 0, "toIndex": 1,
      "bytes": 90210, "truncated": true, "previewB64": "…", "sha256": "…" },
    { "kind": "cd", "seq": 4, "from": "/workspace", "to": "/workspace/app" },
    { "kind": "cmdsub", "seq": 5, "cmdSeq": 6, "bytes": 41, "truncated": false,
      "previewB64": "…" },
    { "kind": "export", "seq": 7, "names": ["AWS_REGION"], "op": "export" },
    { "kind": "fallback", "seq": 8, "reason": "parse-error", "input": "…" }
  ]
}

(export events carry names only, never values — §5.4.)

Batching (locked decision: near-live batching): flush every 200 events / 500 ms / at shell exit — matching the tracer's tuning (CommandInterceptor.swift:122-127) so host-side pacing assumptions hold. This makes the feed feel live for typical commands without a persistent streaming channel; long-running pipes update once per flush. seq is strictly ordered per shellId, so a true streaming transport can be swapped in later without changing the event model.

6.2 Transport per surface

Surface Path to host
Sandbox / runner / linux_container Directly HTTP over the vsock-relayed unix socket /run/nucleic/control.sock (NUCLEIC_SHELL_SOCKET). nash is a native binary, so unlike the Node shims it needs no loopback bridge; this removes a Node cold-start per command and works even if control-bridge.js is down. Fallback: NUCLEIC_SHELL_HOOK_URL via the existing 127.0.0.1:9099 bridge.
Linux VM Phase 1: spool to NUCLEIC_SHELL_SPOOL (/var/spool/nucleic-nash/*.jsonl); the existing nucleic-linux-agent gains a tiny shell-events drain op the host polls after each exec. Phase 2 (optional): agent exposes a local socket forwarding to its vsock channel.
macOS VM Same spool-and-drain via NucleicVMAgent (new wire op alongside exec in AgentWire.swift).

Auth is the existing per-session bearer token (NUCLEIC_HOOK_TOKEN); unknown tokens get 401 and nash stops posting (spools) for that process.

7. Forcing nash as the default shell — per surface

The provisioning inventory below is exhaustive (from a full-repo sweep). "Force" means: (1) nash is on PATH and is $SHELL; (2) Nucleic's own exec argv uses nash; (3) /bin/sh and /bin/bash are diverted to nash so even shebangs and tools that hardcode /bin/sh -c (including the agent CLI's internal Bash tool spawn) land in nash. Real shells are preserved at .real paths — nash's fallback depends on them.

7.1 Sandbox container image (base for runner and linux_container)

Delivery update (NAROS.md): these mechanics now ship as narOS build steps rather than sandbox-Dockerfile layers — the binary installs from the nash .deb and the divert block below becomes the nash-default-shell package's maintainer scripts, baked into the naros-agent image (NAROS §3.1, §4). The fragment is kept as the semantic reference for what the image must contain:

# nash: Nucleic agent shell (static musl, per-arch)
COPY --from=nash-dist /nash-${TARGETARCH} /usr/local/bin/nash
RUN chmod 0755 /usr/local/bin/nash \
 && dpkg-divert --divert /usr/bin/bash.real --rename /bin/bash \
 && dpkg-divert --divert /usr/bin/dash.real --rename /bin/dash \
 && ln -sf /usr/local/bin/nash /bin/bash \
 && ln -sf /usr/local/bin/nash /bin/sh \
 && ln -sf /usr/local/bin/nash /bin/dash
ENV SHELL=/usr/local/bin/nash NUCLEIC_REAL_BASH=/usr/bin/bash.real

(Debian merged-usr: /bin/sh symlink handling verified in M0; dpkg-divert keeps apt upgrades from clobbering the links.)

Version lockstep: the four-place v7 → v8 bump originally planned here (Dockerfile content, sandbox-image.yml:36 IMAGE_TAG, Project.swift:119 defaultImage, nucleic-runner/Dockerfile:38 base pin) is superseded by narOS — it collapses to the single os/VERSION constant plus the Project.swift pin flip to ghcr.io/abkslm/naros-agent:<stable>, with a CI assertion that the pin matches a published stable tag (NAROS §8). containers/nucleic-sandbox/ retires into os/images/agent/; the runner rebases FROM naros-agent (NAROS §7.2).

7.2 Swift-side exec paths (sandbox + linux_container)

Site Today Change
ClaudeCodeBackend.swift:2213 (linux_container exec op) ["/bin/sh","-lc",command] ["/usr/local/bin/nash","-lc",command] (explicit path; custom images without nash get the §7.6 probe fallback)
ContainerEngine+Rootfs.swift:184 (seeded /etc/passwd) …:/tmp:/bin/bash …:/tmp:/usr/local/bin/nash
CommandInterceptor.hookEnv (CommandInterceptor.swift:61-77) git/gh/tracer env add NUCLEIC_SHELL_SOCKET, NUCLEIC_SHELL_HOOK_URL, reuse token/session; drop BASH_ENV tracer wiring once nash ships
ContainerEngine.swift:547 (PID-1 keepalive), ContainerEngine+Rootfs.swift:192,202, ContainerManager.swift:598 (seed/probe sh -c) sh -c unchanged argv — they now resolve to nash via the divert; keep them observation-silent by not passing hook env (plumbing noise, not agent activity)

The agent CLI itself (Claude Code Bash tool) needs no change: it spawns bash/sh by path/$SHELL, all of which now resolve to nash. Its BASH_ENV dependence disappears with the tracer.

7.3 Control / runner container

containers/nucleic-runner/Dockerfile inherits everything from §7.1 via the base-image bump; nucleicd stays PID 1 (no ENTRYPOINT change). Runner-side execs that shell out pick up nash through the divert. Cloudflare registry push pipeline unchanged.

Implemented with narOS N3 (NAROS.md §7.2). The runner's one wrinkle vs the sandbox: its sessions are host runs (RunSpec.container = nil — the container is the sandbox), so no ContainerSpec seeding and no shims, and the containerized hookEnv merge never fires. Delivered instead: CommandInterceptor.hostIsNarOS (/etc/os-release ID=naros) keys a host-run variant in ClaudeCodeBackend — the report routes register and hookEnv (tracer off; legacyShell → NUCLEIC_NASH_DISABLE=1 still honored) merges into the host spawn env, pointed at the run's loopback approval server. nash exec/data-flow events reach the feed as on any surface, and the §9.4 argv convergence (observeShellEvent → git/gh ops) substitutes for the absent shims. Codex/Grok host runs start no approval server today, so their runner sessions stay observation-silent — a follow-up, not a regression (they were equally silent pre-narOS).

7.4 Linux VM

Site Today Change
provision-linux-guest.sh (profile.d PATH) targets /bin/sh execs unchanged — nash -l/profile sourcing honors it
Agent publish linux-vm-agents.yml gains the drain op (§6.2) and republishes

With narOS N4 (NAROS.md §7.4) the guest rootfs is narOS, so nash forcing here is entirely a rootfs property: nash + the /bin/sh→nash divert arrive baked via the naros-tier-vm meta-package, exactly as on the sandbox/runner tiers. Because the divert makes /bin/sh resolve to nash, the agent's existing execl("/bin/sh"…) (agent.c:440) already lands in nash — no agent.c exec swap is needed, and the old hand-installed provisioner steps (useradd -s nash, fetch-and-install nash, apply the divert) collapse into the tier. The only VM-specific M4 work left is the spool-drain op (§6.2). The GNOME 50 desktop flavor rides narOS N5.

Bootstrap-phase scripts (guest/linux-base/bootstrap/init, busybox sh at scripts/build-linux-bootstrap.sh:46) stay on busybox — they run before provisioning, never run agent commands, and must stay minimal.

7.5 macOS VM

The one semantically delicate surface: today mac_vm_exec runs /bin/zsh -c (guest/NucleicVMAgent/…/Ops+Exec.swift:26-27) specifically so /etc/zshenv PATH is sourced. nash is bash-compatible, not zsh-compatible; user commands occasionally use zsh-isms, and macOS system shell cannot be diverted (SIP). Plan:

  • Build nash as a universal macOS binary; install to /usr/local/bin/nash in scripts/provision-macos-guest.sh; mirror /etc/zshenv's PATH into /etc/profile.d-equivalent sourcing that nash -l reads.
  • Switch Ops+Exec.swift to /usr/local/bin/nash -lc, keeping a zsh escape hatch in the wire op (shell: "zsh") for compat, and set the agent account's login shell via sysadminctl/dscl during provisioning.
  • This is the last phase (M5) and gated on M1–M4 compat data; if zsh-isms show up meaningfully in VM transcripts, the fallback posture is nash-by-default + auto-fallback-to-zsh on parse failure (same doctrine as §4.1).

7.6 Custom images and degradation

linux_container allows custom images (guarded today by a node+bridge probe, ContainerEngine+Rootfs.swift:200-207). Extend the probe to check for /usr/local/bin/nash; when absent, seed the static binary the same way shims are seeded (base64 install won't work for a ~5 MB binary — instead relay it via the existing rootfs clone/seed file-copy path), or degrade: exec falls back to /bin/sh -lc and the session records shellObservability: degraded. Never refuse to run.

narOS adds a fast path and an upgrade path (NAROS §7.3): the probe first checks /etc/os-release for ID=naros — nash, the bridge, and node are then known-present and the probe is done — and any Debian-family custom image can be converted in place with apt install naros-keyring naros-tier-agent. The seed/degrade path above remains for everything else.

8. Build and distribution

  • Toolchain: Rust stable, cargo workspace at shell/. Targets: aarch64-unknown-linux-musl, x86_64-unknown-linux-musl (static, no libc drift across Debian/VM rootfs), aarch64-apple-darwin + x86_64-apple-darwin lipo'd universal.
  • CI: new .github/workflows/nash.yml — build matrix, run brush's bash-compat suite against the fork, run nash-observe tests, then publish:
    • Linux binaries → GHCR OCI artifact ghcr.io/abkslm/nucleic-nash:<ver> (same single-blob pattern as nucleic-linux-agent, docs/LINUX_VM.md:98-104), consumed by the sandbox Dockerfile (build stage nash-dist) and the Linux VM provisioner.
    • macOS universal binary → artifact consumed by provision-macos-guest.sh.
  • Debian packaging (narOS): the same CI also emits nash + nash-default-shell .debs into the signed narOS apt repository (NAROS §3) — the deterministic baseline the naros images install at build, and a third in-place update channel (apt upgrade nash) alongside image releases and boot-time seeding.
  • Update channel (locked decision: hybrid): the image bakes a known-good nash as the deterministic baseline; at container/VM boot, the host compares versions and, when it holds a newer binary (bundled with the app or fetched from the GHCR artifact), seeds it over the baked one via the existing rootfs file-copy path — the same lifecycle as the git/gh shim seeding, gated by the seedRecipe version. Shell fixes thus reach dogfood in an app update without an image bump; image tags still move at milestone boundaries so the baseline never drifts far. The §7.6 custom-image seeding is this same mechanism.
  • Version pinning: nash version recorded in the image (label + nash --version) and re-read after any seed override; SessionController logs the effective version per session so feed events are attributable to a shell build.

9. Host-side integration (Swift)

Follows the existing report pipeline shape exactly (shim → route → struct → backend → coordinator → AppStore → feed):

  1. Route: shellEventPath = "/shell-event" in MCPApprovalServer.swift (beside gitEventPath:649); dispatch beside lines 1752-1760; handleShellEvent parses shell-batch into [ShellReportCall] (new struct beside CommandReportCall:570), one per event, carrying the typed payload (exec / redirect / pipe / cd / fallback / dropped).
  2. Registration: registerShellReport(token:handler:) beside registerCommandReport (1410-1425); wired in all four backends (ClaudeCodeBackend.swift:2721-2744 and Codex/Grok twins).
  3. Coordinator: ConflictCoordinator.observeShellEvent(sessionID:call:) forwarding to AppStore.
  4. AppStore: observeShellEvent —
    • exec events flow into the existing observeCommand path (AppStore.swift:4666) via CommandSummary.classify(argv:), tagged source: .nash, preserving the NVRSION residue-capture backstop (4679-4689) and CMDTRACE logging;
    • exec events whose argv[0] is git/gh also feed observeGitOp/observeGhOp — this is the convergence path that eventually retires the Node shims. While shims coexist, dedupe on (sessionId, argv, exitCode, ±2 s) preferring the shim event (it's the proven locks/autoship trigger); flip preference once M4 exits.
    • redirect/pipe events land in a new DataFlowEvent value type in ControlPanel.swift (beside CommandInterceptorEvent:63) and are persisted (locked decision): they join the session transcript store (GRDB) as durable, searchable audit history — redacted previews included — alongside an in-memory capped ring for the live feed. Retention: per-session row cap + a global size budget with oldest-first eviction; both configurable (§9.6).
  5. UI (locked decision: feed rows + detail popover): ControlPanelView.swift activity feed gains expandable rows for data-flow events — operator glyph, target path, byte count, and preview (monospace, redacted, truncation badge) — with a click-through detail popover showing the full bounded preview, hashes, pipeline structure, and nesting. fallback events render as a warning row (these are compat bugs to file upstream). No dedicated shell panel in this plan; the persisted history (§9.4) leaves that open as a future milestone.
  6. Settings: nucleic.container.commandTracing (Project.swift:572-576) is superseded by nucleic.container.shellCapture = off | meta | tap (default tap), mapped to NUCLEIC_SHELL_CAPTURE. nucleic.container.shellRetention bounds the persisted data-flow history (per-session rows + global size budget, §9.4). A separate rollback lever nucleic.container.legacyShell (default false) injects NUCLEIC_NASH_DISABLE=1 and restores the tracer wiring — one toggle to fully revert behavior without an image rollback.
  7. iPhone sync (locked decision: full event sync): shell events enter the normalized AgentEvent/HostMsg stream (SYNC_PROTOCOL.md) as their own event family with the canonical seq cursor, so catch-up after disconnect works unchanged. The complete stream syncs — exec, data-flow, cmdsub, export-name, and fallback events including their redacted, capped previews — over the existing E2EE SecureChannel; previews never travel un-redacted because redaction already happened in-guest (§5.4). Two accommodations for the thin client: (a) data-flow payload frames are marked low-priority so approval traffic always preempts them on LAN/relay; (b) the phone keeps a bounded local ring (per-session row + size caps mirroring §9.4) with oldest-first eviction — full stream, bounded retention. Relay-mode bandwidth is the main cost; see risk table.

10. Testing

  1. Fork conformance (CI-gating): brush's ~1,700-case bash compat suite must pass on the fork with observation on — proving hooks don't perturb semantics. Run twice (capture=off, capture=tap) and diff outcomes.
  2. Tap semantics (Rust integration tests, nash-observe): append preserves interleaving under concurrent writers; seek-after-redirect (dd of=…) byte-identical to bash; set -o pipefail exit codes through tees; SIGPIPE propagation through tees (yes | head); binary data through pipes uncorrupted (hash comparison); /dev/null, fifos, 2>&1 dup chains; heredoc capture; cmdsub preview matches $(…) result byte-for-byte under cap; export events never contain values; 100 MB stream under cap → correct byte count, bounded memory.
  3. Transcript replay corpus: extract the Bash-tool command corpus from existing session transcripts (fixtures pattern per OBSERVABILITY_AND_TESTING.md) and replay under nash vs bash in the sandbox image, diffing stdout/stderr/exit. This is the M0 gate and a permanent regression suite.
  4. Swift tests: ShellReportParsingTests (batch → calls) beside MCPApprovalServerTests; ContainerSandboxTests extension asserting the naros-agent image resolves /bin/sh → nash and that a linux_container exec produces exec + pipe events end-to-end; dedupe tests for the git-shim coexistence window.
  5. Fault injection: control socket absent (spool fills, commands unaffected); host 401 (posting stops, commands unaffected); nash-observe panic (hook boundary catches; command completes).

11. Rollout phases

Phase Deliverable Exit criteria
M0 — spike Fork subtree; nash -c runs via embedded brush; transcript-replay corpus harness ≥99% corpus parity vs bash; list of divergences filed
M1 — exec events Gate trait patch (allow-all); exec/cd/fallback events; unix-socket transport; /shell-event route → feed; SHELL+linux_container argv switched (no divert yet); tracer retired Feed shows nash events for a live session; tracer deleted; compat suite green
M2 — data-flow taps Redirect read-back + pipe/heredoc tees; caps + redaction; UI preview rows Tap test matrix green; overhead <3% on corpus wall-clock
M3 — force sandbox/runner narOS v1 with the divert baked (naros-agent, NAROS.md N2 — supersedes the v8 image); Project.swift pin flip; custom-image probe/degrade incl. the narOS fast path; legacyShell rollback lever Week of dogfood sessions, zero fallback-event regressions unresolved (these double as narOS v1 ship gates)
M4 — Linux VM spool drain op + host poll (nash forcing is handled by narOS N4's baked divert — nash via naros-tier-vm on a narOS guest rootfs — so no agent.c exec swap or hand-installed provisioner steps) linux_vm_exec events in feed; VM compat parity
M5 — macOS VM (gated) universal binary, Ops+Exec swap with zsh escape hatch zsh-ism fallback rate ≈0 in dogfood, else hold
M6 — converge git/gh shim retirement behind nash-native classification; upstream the Gate trait locks/autoship driven by nash events for a full dogfood cycle

Post-M6 (explicitly out of scope here, enabled by the locked hook design): activating enforcement — Hold/Deny verdicts backed by a synchronous /shell-gate policy round-trip, per surface, with its own design doc.

12. Risks and mitigations

Risk Mitigation
brush compat gaps break agent commands Parse-fallback to preserved real bash + fallback telemetry; kill switch env; legacyShell setting; M0 corpus gate before any forcing
Tee alters semantics for seek-dependent tools Design rule: regular files never tee'd (real fd + read-back); tees only where a pipe already exists
Performance (async tees, event posting) Static musl binary (no Node cold start — strictly faster than today's shims); fire-and-forget posts; <3% overhead gate in M2
Data volume / secrets in captured bytes Caps at three levels; in-guest redaction before transport; previews only, never full payloads; capture level setting incl. meta-only
Upstream drift of brush fork Subtree + minimal hook-shaped patch; aim to upstream the Gate trait (fork → zero)
Persisted previews put (redacted) command data at rest Redaction runs in-guest before transport (§5.4); previews only, hard caps; transcript retention caps + shellCapture level can drop payloads entirely; transcript store already holds session data under the same protections
Full iPhone sync strains LAN/relay bandwidth and phone storage Previews are capped/redacted before they ever reach the sync layer; data-flow frames are low-priority behind approvals; phone-side ring with retention caps; relay verbosity throttling remains the backstop if measured cost is too high
nash name prior art The acronym ("Nucleic agent shell") both describes the tool and avoids the Almquist ash collision; no nash binary ships in our images; version string + NUCLEIC_NASH=1 disambiguate; rename stays cheap
Image/tag lockstep mistakes Superseded by narOS: one os/VERSION constant + a CI assertion that the Project.swift pin matches a published stable naros-agent tag (NAROS §8)
Agent bypasses nash (exec /usr/bin/bash.real) Accepted (non-goal §1); the exec event recording the bypass is itself the observability win; .real paths can be policy-flagged in feed classification
macOS zsh-isms M5 gated; per-op shell: escape hatch; auto-fallback to zsh on parse failure
Reference in New Issue View Git Blame Copy Permalink