Merge nucleic/sleek-thistle-egret-fyej into dev

docs/ASH.md

@@ -0,0 +1,495 @@
+# ash — the Nucleic Agent Shell
+
+**Status: plan (not yet implemented).**
+
+`ash` ("agent shell") is a Bourne/bash-compatible shell, forked in Rust, whose job is to make
+every shell action an agent takes *observable by construction*. Where the current shim system
+watches two binaries (`git`, `gh`) and an opt-in bash `DEBUG`-trap tracer watches command
+metadata, ash sits at the choke point itself: it **is** the shell, so it sees every simple
+command, every pipeline, and — the new capability — **the data flowing through redirection
+operators and pipes**, which it taps and passes through unmodified. It is the logical
+end-state of the command-shim system: instead of shimming individual tools ahead of `PATH`,
+Nucleic controls the interpreter that launches all of them.
+
+This doc locks down: the fork base, the crate/repo layout, the interception design (exec
+events, redirection taps, pipe tees), the event schema and transport, host-side integration,
+how ash becomes the **forced default shell on every surface** (sandbox containers, the
+control/runner container, MCP `linux_container`, Linux VM, macOS VM), build/distribution,
+testing, rollout phases, and risks.
+
+Related docs: [VSOCK_CONTROL_PLANE.md](VSOCK_CONTROL_PLANE.md) (transport this rides on),
+[CONTAINER_ISOLATION.md](CONTAINER_ISOLATION.md), [LINUX_VM.md](LINUX_VM.md),
+[MACOS_VM.md](MACOS_VM.md), [OBSERVABILITY_AND_TESTING.md](OBSERVABILITY_AND_TESTING.md)
+(redaction posture), [LOCKING.md](LOCKING.md) (consumer of git observations).
+
+---
+
+## 1. Goals and non-goals
+
+**Goals**
+
+1. **Total command visibility.** Every simple command executed by an agent shell — argv,
+   resolved path, cwd, exit code, duration, pipeline position, nesting (subshell / function /
+   command substitution) — reported to the host, always on, with no reliance on bash's
+   `BASH_ENV`/`DEBUG`-trap fragility.
+2. **Data-flow visibility.** Bounded, redaction-aware capture of bytes crossing redirection
+   operators (`>`, `>>`, `<`, `2>`, `&>`, here-docs/strings, `>|`, fd dups) and pipes
+   (`a | b`), passed through byte-for-byte to preserve semantics.
+3. **Transparency.** Agents and their tools must not behave differently under ash. Existing
+   scripts, `.bashrc`-style init, and the Claude Code / Codex Bash tool must run unchanged.
+   Observation is best-effort and may *never* block, slow measurably, or alter a command's
+   outcome. On any internal ash failure, the command still runs.
+4. **Forced default everywhere Nucleic controls the OS image**: sandbox containers, the
+   cloud runner container, agent-created `linux_container`s, the Linux VM, the macOS VM.
+5. **Subsume the tracer, converge the shims.** The `BASH_ENV` command tracer
+   (`CommandInterceptor.swift`) is retired once ash lands; the git/gh shims remain initially
+   (they feed locks/autoship) and are retired once ash-native classification is proven.
+
+**Non-goals**
+
+- The **user's real Mac** (host `zsh` surfaces: `TerminalPanel.swift`, `BuildRunPanel.swift`,
+  `host_exec` via `/bin/zsh -lc`, `LoginShellPATH.swift`). We do not change the operator's
+  machine shell. `host_exec` stays on the user's shell; its guardrail remains
+  `HostExecPolicy.swift`.
+- An interactive daily-driver shell. ash runs non-interactive agent commands; interactive
+  features (highlighting, completion) are stripped from the build.
+- A security boundary. ash is an *observability* layer; enforcement stays where it is today
+  (approval server, `HostExecPolicy`, PreToolUse hooks). A determined command can still
+  `exec /usr/bin/dash.real`; we record that it did.
+
+---
+
+## 2. Fork base: brush
+
+We do not port the C Almquist/dash lineage to Rust; we fork **[brush]**
+(<https://github.com/reubeno/brush>), a bash/POSIX-compatible shell in Rust:
+
+- **MIT-licensed**, actively developed (v0.4.0, May 2026), validated against bash with
+  ~1,700 compatibility tests; bash major features (`set -e/-u`, `pipefail`, traps incl.
+  `ERR`, coprocesses) implemented.
+- **Designed for embedding**: `brush_core::Shell` is an embeddable engine; the workspace
+  splits `brush-parser` (syntax), `brush-core` (interpreter: `interp.rs`, `commands.rs`,
+  `processes.rs`, `openfiles.rs`, `sys/`), `brush-builtins`, `brush-shell` (CLI binary).
+- Builds static for `x86_64`/`aarch64` Linux (musl) and native macOS — exactly our matrix.
+
+Alternatives considered and rejected:
+
+| Option | Why not |
+| --- | --- |
+| Fork dash/busybox-ash (C) | Not Rust; instrumenting C fd plumbing is the bug-prone path this project exists to avoid |
+| Shell from scratch in Rust | Years of compat work brush has already done (and tested) |
+| `nsh` / `rusty_bash` / others | Much lower bash compatibility and maturity than brush |
+| Pure wrapper around real bash (pty + strace/eBPF) | No structural knowledge of redirections; eBPF unavailable in our unprivileged container guests; fragile |
+
+**Fork mechanics.** Vendor the fork as a git subtree at `third_party/brush/` (matching the
+existing `third_party/containerization/` convention), tracking a fork repo
+(`github.com/abkslm/brush`) so upstream merges stay routine. Patches to brush-core are kept
+minimal and hook-shaped (see §4.2) to keep the merge surface small; everything else lives in
+our own crates.
+
+**Naming.** `ash` collides with the historical Almquist shell (busybox `ash`; Debian's
+`dash` is its descendant). Our images (`node:22-bookworm-slim`, our VM rootfs) ship no
+busybox `ash` binary, so there is no on-disk conflict, and the name is intentional
+("agent shell"). To avoid ecosystem confusion: the binary installs as
+`/usr/local/bin/ash`, `ash --version` reports `ash (Nucleic agent shell, brush fork) x.y.z`,
+and it sets `NUCLEIC_ASH=1` in its own environment so scripts/tests can detect it. If the
+collision ever bites (e.g. a tool sniffing `ash` and assuming busybox semantics), the escape
+hatch is renaming the file; nothing in the design depends on the name.
+
+---
+
+## 3. Repo layout
+
+```
+third_party/brush/            # vendored fork (git subtree of abkslm/brush)
+shell/                        # new Cargo workspace
+  Cargo.toml
+  ash/                        # bin crate: CLI entry, arg parsing (-c/-lc/-s/files),
+                              #   config env, bash-fallback, embeds brush_core::Shell
+  ash-observe/                # lib crate: event model, taps, batching, spool,
+                              #   transport (HTTP over unix socket / TCP), redaction
+containers/nucleic-sandbox/   # Dockerfile gains ash install (see §7.1)
+.github/workflows/ash.yml     # build + publish (see §8)
+docs/ASH.md                   # this doc
+```
+
+`shell/` is intentionally *outside* `Sources/` (it's not Swift) and outside `guest/`
+(it ships to containers *and* VMs). The Cargo workspace pins the vendored brush via
+`path = "../third_party/brush/brush-core"` dependencies.
+
+---
+
+## 4. Shell architecture
+
+### 4.1 The binary
+
+`ash` is a thin binary embedding `brush_core::Shell`:
+
+- Accepts the bash-compatible invocation matrix we actually use: `ash -c <cmd>`,
+  `ash -lc <cmd>`, `ash <script> [args]`, stdin scripts, `-e`/`-x`/`-o pipefail`
+  passthrough. Login (`-l`) sources `/etc/profile` + profile.d so the existing
+  `nucleic-path.sh` PATH setup keeps working.
+- Reads observation config from env at startup (all optional — absent config means
+  "run silently as a plain shell"):
+  - `NUCLEIC_SHELL_HOOK_URL` — event endpoint (`http://…/shell-event`)
+  - `NUCLEIC_SHELL_SOCKET` — unix socket to speak HTTP over directly
+    (default `/run/nucleic/control.sock`), preferred over the URL when present
+  - `NUCLEIC_HOOK_TOKEN`, `NUCLEIC_SESSION_ID` — existing auth/session vars
+  - `NUCLEIC_SHELL_SPOOL` — spool dir for offline batches (default
+    `/var/spool/nucleic-ash`)
+  - `NUCLEIC_SHELL_CAPTURE` — `off | meta | tap` (default `tap`; see §5)
+  - `NUCLEIC_ASH_DISABLE=1` — kill switch: immediately `exec` the real bash
+    (`NUCLEIC_REAL_BASH`, default `/usr/bin/bash.real`, falling back to `/bin/bash`)
+    with identical argv. This is the "get out of jail" lever for compat emergencies.
+- **Parse-failure fallback**: if brush-parser rejects input that looks like valid bash
+  (parse error on `-c` input or a sourced file), ash emits a `fallback` event and re-execs
+  the real bash with the same argv, so the agent's command still succeeds. This converts
+  brush compat gaps from breakage into telemetry we can fix.
+
+### 4.2 Patch surface inside brush-core (the fork proper)
+
+Kept deliberately tiny — an `Observer` trait injected into the shell, with call sites at
+the four choke points. Everything else (transport, batching, redaction) is in `ash-observe`.
+
+```rust
+// brush-core addition (new file observer.rs, ~1 trait + no-op default):
+pub trait Observer: Send + Sync {
+    fn on_exec(&self, ev: ExecStart) -> ExecToken;          // before spawn
+    fn on_exit(&self, tok: ExecToken, status: ExecEnd);     // after wait
+    fn on_redirect(&self, ev: RedirectEvent) -> RedirTap;   // during redir setup
+    fn on_pipe(&self, ev: PipeEvent) -> Option<TeeFds>;     // pipeline wiring
+}
+```
+
+Call sites (file names from brush-core `main`):
+
+| Choke point | brush-core location | What the hook sees/does |
+| --- | --- | --- |
+| Simple-command spawn/wait | `commands.rs` / `processes.rs` (external cmds), builtin dispatch in `interp.rs` | argv, resolved path, cwd, pipeline index, nesting depth; later pid, exit status, duration |
+| Redirection setup | `interp.rs` + `openfiles.rs` (where redirs become fd mappings) | operator, target (path/fd/heredoc), open mode; may substitute a tapped fd (§5.2) |
+| Pipeline wiring | `interp.rs` pipe construction | may interpose a tee pipe (§5.3) |
+| Word-level facts | `interp.rs` (`cd`, `export`, `source`, function def) | state-change events for the feed |
+
+The `Observer` default impl is a no-op, so the vendored fork remains mergeable with
+upstream and even upstreamable (a generic observer/hook API is a plausible upstream
+contribution, which would shrink our fork to zero).
+
+---
+
+## 5. Observation semantics
+
+### 5.1 Exec events (always, capture ≥ `meta`)
+
+One event per simple command (external or builtin), including inside functions, subshells,
+command substitutions, and `source`d files:
+
+- `argv` (post-expansion), `resolvedPath`, `cwd`, `pid`, `exitCode`, `durationMs`
+- `pipeline: {id, index, len}` when part of `a | b | c`
+- `nesting: {depth, kind}` (`top`, `subshell`, `cmdsub`, `function:NAME`, `source:PATH`)
+- This strictly supersedes the bash tracer's `{line, cwd, exitCode, durationMs}` — richer
+  (structured argv vs. raw line) and reliable (no `BASH_ENV` inheritance games).
+
+### 5.2 Redirection taps (`capture = tap`)
+
+The user-visible requirement: *intercept data flowing through redirection operators to
+observe it, then pass it through as expected.* Two mechanisms, chosen per target to
+guarantee transparency:
+
+**(a) Regular-file redirections → open-real-fd + bounded read-back (default).**
+For `> file`, `>> file`, `< file`, `2> file`: ash opens the *real* file and hands the child
+the *real* fd — semantics (seekability, `O_APPEND` atomicity, `fstat` identity, tools that
+`lseek` their output) are untouched by construction. Observation happens around it:
+
+- record `{operator, path, mode, sizeBefore}` at setup;
+- after the command completes, record `sizeAfter` and read back the affected range
+  (for `>`: `0..min(size, CAP)`; for `>>`: `sizeBefore..sizeBefore+CAP`; for `<`: head of
+  the file), attaching `bytes`, `truncated`, `sha256`, and a `preview` (first `CAP` bytes,
+  base64) to the event.
+- `CAP` default 64 KiB per redirection, configurable; binary data detected and previewed
+  as hex head. Special files (`/dev/null`, `/dev/tty`, sockets, device nodes) are
+  metadata-only, never read back.
+
+This is chosen over inline tee-ing for files because a tee pipe would silently break any
+program that seeks on its redirected fd — an unacceptable transparency violation for a
+default shell.
+
+**(b) Stream redirections → inline tee.** Where the data is *already* a stream — pipes
+between commands, here-docs/here-strings, process substitution `<(…)`/`>(…)`, and fd dups
+onto pipes — ash interposes: child gets a pipe; an async task (brush-core is tokio-based)
+copies bytes through to the real destination while mirroring the first `CAP` bytes and a
+running count/hash into the event buffer. Backpressure is preserved (the tee task copies,
+never buffers unboundedly); if the tee task dies, the copy loop degrades to a raw
+`splice`-style passthrough and the event is flagged `tapFailed`.
+
+### 5.3 Pipe tees
+
+`a | b` reports one `pipe` event per link: `{pipelineId, fromIndex, toIndex, bytes,
+truncated, preview, sha256}` via mechanism (b). This surfaces what the transcript never
+shows today — the intermediate data between stages (`curl … | sh` becomes *visible*).
+
+### 5.4 Caps, redaction, volume
+
+- Per-event preview cap (64 KiB), per-command total capture cap (256 KiB), per-batch cap
+  (1 MiB); beyond caps → counts/hashes only, `truncated: true`.
+- Redaction runs *inside ash before bytes leave the guest*: the same posture as
+  [OBSERVABILITY_AND_TESTING.md](OBSERVABILITY_AND_TESTING.md) — pattern-based masking of
+  obvious credential shapes (bearer/PAT/AWS-style tokens, `PRIVATE KEY` blocks) in previews;
+  env vars are **never** captured wholesale (exec events carry argv, not environment).
+- Everything is best-effort: full buffers → drop-oldest with a `dropped` counter event.
+
+### 5.5 Failure doctrine
+
+Observation must never break work. Concretely: no hook may propagate an error into command
+execution; transport is fire-and-forget with short timeouts (inherit the shims' 1.5–2 s
+budget) and disk spool fallback; panics in `ash-observe` are caught at the hook boundary;
+and the kill switch (`NUCLEIC_ASH_DISABLE`) plus the preserved real bash mean any
+regression has a one-env-var mitigation while we fix forward.
+
+---
+
+## 6. Event schema and transport
+
+### 6.1 Batch shape
+
+Extends the existing `/command-event` family with a new endpoint `POST /shell-event`
+(bearer-token gated, 202-always like `/git-event` — see `MCPApprovalServer.swift:2868`):
+
+```jsonc
+{
+  "type": "shell-batch",
+  "source": "ash",
+  "sessionId": "…",            // NUCLEIC_SESSION_ID
+  "shellId": "…",              // random per ash process; correlates events
+  "parentShell": "…|null",     // shellId inherited via env → process tree of shells
+  "events": [
+    { "kind": "exec", "seq": 1, "ts": 1721…, "argv": ["git","push"],
+      "resolvedPath": "/usr/local/bin/git", "cwd": "/workspace", "pid": 412,
+      "exitCode": 0, "durationMs": 812,
+      "pipeline": null, "nesting": {"depth": 0, "kind": "top"} },
+    { "kind": "redirect", "seq": 2, "cmdSeq": 1, "op": ">>",
+      "target": {"path": "/workspace/out.log"}, "mode": "append",
+      "bytes": 1832, "truncated": false, "sha256": "…", "previewB64": "…" },
+    { "kind": "pipe", "seq": 3, "pipelineId": "p1", "fromIndex": 0, "toIndex": 1,
+      "bytes": 90210, "truncated": true, "previewB64": "…", "sha256": "…" },
+    { "kind": "cd", "seq": 4, "from": "/workspace", "to": "/workspace/app" },
+    { "kind": "fallback", "seq": 5, "reason": "parse-error", "input": "…" }
+  ]
+}
+```
+
+Batching: flush every 200 events / 500 ms / at shell exit — matching the tracer's tuning
+(`CommandInterceptor.swift:122-127`) so host-side pacing assumptions hold.
+
+### 6.2 Transport per surface
+
+| Surface | Path to host |
+| --- | --- |
+| Sandbox / runner / `linux_container` | **Directly** HTTP over the vsock-relayed unix socket `/run/nucleic/control.sock` (`NUCLEIC_SHELL_SOCKET`). ash is a native binary, so unlike the Node shims it needs no loopback bridge; this removes a Node cold-start per command and works even if `control-bridge.js` is down. Fallback: `NUCLEIC_SHELL_HOOK_URL` via the existing `127.0.0.1:9099` bridge. |
+| Linux VM | Phase 1: spool to `NUCLEIC_SHELL_SPOOL` (`/var/spool/nucleic-ash/*.jsonl`); the existing `nucleic-linux-agent` gains a tiny `shell-events` drain op the host polls after each exec. Phase 2 (optional): agent exposes a local socket forwarding to its vsock channel. |
+| macOS VM | Same spool-and-drain via `NucleicVMAgent` (new wire op alongside exec in `AgentWire.swift`). |
+
+Auth is the existing per-session bearer token (`NUCLEIC_HOOK_TOKEN`); unknown tokens get
+401 and ash stops posting (spools) for that process.
+
+---
+
+## 7. Forcing ash as the default shell — per surface
+
+The provisioning inventory below is exhaustive (from a full-repo sweep). "Force" means:
+(1) ash is on `PATH` and is `$SHELL`; (2) Nucleic's own exec argv uses ash; (3) `/bin/sh`
+and `/bin/bash` are **diverted** to ash so even shebangs and tools that hardcode
+`/bin/sh -c` (including the agent CLI's internal Bash tool spawn) land in ash. Real shells
+are preserved at `.real` paths — ash's fallback depends on them.
+
+### 7.1 Sandbox container image (base for runner and `linux_container`)
+
+`containers/nucleic-sandbox/Dockerfile` (base `node:22-bookworm-slim`, line 20):
+
+```dockerfile
+# ash: Nucleic agent shell (static musl, per-arch)
+COPY --from=ash-dist /ash-${TARGETARCH} /usr/local/bin/ash
+RUN chmod 0755 /usr/local/bin/ash \
+ && dpkg-divert --divert /usr/bin/bash.real --rename /bin/bash \
+ && dpkg-divert --divert /usr/bin/dash.real --rename /bin/dash \
+ && ln -sf /usr/local/bin/ash /bin/bash \
+ && ln -sf /usr/local/bin/ash /bin/sh \
+ && ln -sf /usr/local/bin/ash /bin/dash
+ENV SHELL=/usr/local/bin/ash NUCLEIC_REAL_BASH=/usr/bin/bash.real
+```
+
+(Debian merged-usr: `/bin/sh` symlink handling verified in M0; `dpkg-divert` keeps apt
+upgrades from clobbering the links.)
+
+Version lockstep (must move together, per existing coupling):
+1. `containers/nucleic-sandbox/Dockerfile` content (above)
+2. `.github/workflows/sandbox-image.yml:36` — `IMAGE_TAG: v7 → v8`
+3. `Sources/NucleicCore/Project.swift:119` — `defaultImage … :v7 → :v8`
+4. `containers/nucleic-runner/Dockerfile:38` — base pin `nucleic-sandbox:v7 → :v8`
+
+### 7.2 Swift-side exec paths (sandbox + `linux_container`)
+
+| Site | Today | Change |
+| --- | --- | --- |
+| `ClaudeCodeBackend.swift:2213` (`linux_container` exec op) | `["/bin/sh","-lc",command]` | `["/usr/local/bin/ash","-lc",command]` (explicit path; custom images without ash get the §7.6 probe fallback) |
+| `ContainerEngine+Rootfs.swift:184` (seeded `/etc/passwd`) | `…:/tmp:/bin/bash` | `…:/tmp:/usr/local/bin/ash` |
+| `CommandInterceptor.hookEnv` (`CommandInterceptor.swift:61-77`) | git/gh/tracer env | add `NUCLEIC_SHELL_SOCKET`, `NUCLEIC_SHELL_HOOK_URL`, reuse token/session; **drop** `BASH_ENV` tracer wiring once ash ships |
+| `ContainerEngine.swift:547` (PID-1 keepalive), `ContainerEngine+Rootfs.swift:192,202`, `ContainerManager.swift:598` (seed/probe `sh -c`) | `sh -c` | unchanged argv — they now resolve to ash via the divert; keep them observation-silent by not passing hook env (plumbing noise, not agent activity) |
+
+The agent CLI itself (Claude Code Bash tool) needs no change: it spawns bash/sh by
+path/`$SHELL`, all of which now resolve to ash. Its `BASH_ENV` dependence disappears with
+the tracer.
+
+### 7.3 Control / runner container
+
+`containers/nucleic-runner/Dockerfile` inherits everything from §7.1 via the base-image
+bump; `nucleicd` stays PID 1 (no ENTRYPOINT change). Runner-side execs that shell out pick
+up ash through the divert. Cloudflare registry push pipeline unchanged.
+
+### 7.4 Linux VM
+
+| Site | Today | Change |
+| --- | --- | --- |
+| `guest/nucleic-linux-agent/src/agent.c:440` | `execl("/bin/sh","sh","-c",command,…)` | `execl("/usr/local/bin/ash","ash","-c",command,…)` with fallback to `/bin/sh` if ash missing (old base images) |
+| `guest/linux-base/provision/provision-linux-guest.sh:147` | `useradd -m -s /bin/bash …` | `-s /usr/local/bin/ash`; provision step installs the ash binary (fetched like the agent, from the GHCR OCI artifact) and applies the same divert block as §7.1 |
+| `provision-linux-guest.sh:241-245` (profile.d PATH) | targets `/bin/sh` execs | unchanged — ash `-l`/profile sourcing honors it |
+| Agent publish | `linux-vm-agents.yml` | gains the drain op (§6.2) and republishes |
+
+Bootstrap-phase scripts (`guest/linux-base/bootstrap/init`, busybox `sh` at
+`scripts/build-linux-bootstrap.sh:46`) stay on busybox — they run before provisioning,
+never run agent commands, and must stay minimal.
+
+### 7.5 macOS VM
+
+The one semantically delicate surface: today `mac_vm_exec` runs `/bin/zsh -c`
+(`guest/NucleicVMAgent/…/Ops+Exec.swift:26-27`) specifically so `/etc/zshenv` PATH is
+sourced. ash is bash-compatible, not zsh-compatible; user commands occasionally use
+zsh-isms, and macOS system shell cannot be diverted (SIP). Plan:
+
+- Build ash as a universal macOS binary; install to `/usr/local/bin/ash` in
+  `scripts/provision-macos-guest.sh`; mirror `/etc/zshenv`'s PATH into
+  `/etc/profile.d`-equivalent sourcing that ash `-l` reads.
+- Switch `Ops+Exec.swift` to `/usr/local/bin/ash -lc`, keeping a zsh escape hatch in the
+  wire op (`shell: "zsh"`) for compat, and set the agent account's login shell via
+  `sysadminctl`/`dscl` during provisioning.
+- This is the **last** phase (M5) and gated on M1–M4 compat data; if zsh-isms show up
+  meaningfully in VM transcripts, the fallback posture is ash-by-default +
+  auto-fallback-to-zsh on parse failure (same doctrine as §4.1).
+
+### 7.6 Custom images and degradation
+
+`linux_container` allows custom images (guarded today by a node+bridge probe,
+`ContainerEngine+Rootfs.swift:200-207`). Extend the probe to check for
+`/usr/local/bin/ash`; when absent, seed the static binary the same way shims are seeded
+(base64 install won't work for a ~5 MB binary — instead relay it via the existing rootfs
+clone/seed file-copy path), or degrade: exec falls back to `/bin/sh -lc` and the session
+records `shellObservability: degraded`. Never refuse to run.
+
+---
+
+## 8. Build and distribution
+
+- **Toolchain**: Rust stable, `cargo` workspace at `shell/`. Targets:
+  `aarch64-unknown-linux-musl`, `x86_64-unknown-linux-musl` (static, no libc drift across
+  Debian/VM rootfs), `aarch64-apple-darwin` + `x86_64-apple-darwin` lipo'd universal.
+- **CI**: new `.github/workflows/ash.yml` — build matrix, run brush's bash-compat suite
+  against the fork, run ash-observe tests, then publish:
+  - Linux binaries → GHCR **OCI artifact** `ghcr.io/abkslm/nucleic-ash:<ver>` (same
+    single-blob pattern as `nucleic-linux-agent`, `docs/LINUX_VM.md:98-104`), consumed by
+    the sandbox Dockerfile (build stage `ash-dist`) and the Linux VM provisioner.
+  - macOS universal binary → artifact consumed by `provision-macos-guest.sh`.
+- **Version pinning**: ash version recorded in the image (label + `ash --version`);
+  `SessionController` logs it per session so feed events are attributable to a shell build.
+
+---
+
+## 9. Host-side integration (Swift)
+
+Follows the existing report pipeline shape exactly (shim → route → struct → backend →
+coordinator → AppStore → feed):
+
+1. **Route**: `shellEventPath = "/shell-event"` in `MCPApprovalServer.swift` (beside
+   `gitEventPath:649`); dispatch beside lines 1752-1760; `handleShellEvent` parses
+   `shell-batch` into `[ShellReportCall]` (new struct beside `CommandReportCall:570`),
+   one per event, carrying the typed payload (`exec` / `redirect` / `pipe` / `cd` /
+   `fallback` / `dropped`).
+2. **Registration**: `registerShellReport(token:handler:)` beside
+   `registerCommandReport` (1410-1425); wired in all four backends
+   (`ClaudeCodeBackend.swift:2721-2744` and Codex/Grok twins).
+3. **Coordinator**: `ConflictCoordinator.observeShellEvent(sessionID:call:)` forwarding to
+   `AppStore`.
+4. **AppStore**: `observeShellEvent` —
+   - `exec` events flow into the existing `observeCommand` path (`AppStore.swift:4666`)
+     via `CommandSummary.classify(argv:)`, tagged `source: .ash`, preserving the NVRSION
+     residue-capture backstop (4679-4689) and `CMDTRACE` logging;
+   - `exec` events whose argv[0] is git/gh **also** feed `observeGitOp`/`observeGhOp` —
+     this is the convergence path that eventually retires the Node shims. While shims
+     coexist, dedupe on `(sessionId, argv, exitCode, ±2 s)` preferring the shim event
+     (it's the proven locks/autoship trigger); flip preference once M4 exits.
+   - `redirect`/`pipe` events land in a new `DataFlowEvent` value type in
+     `ControlPanel.swift` (beside `CommandInterceptorEvent:63`), capped ring like
+     `commandInterceptorEvents`.
+5. **UI**: `ControlPanelView.swift` activity feed gains expandable rows for data-flow
+   events — operator glyph, target path, byte count, and preview (monospace, redacted,
+   truncation badge). `fallback` events render as a warning row (these are compat bugs to
+   file upstream).
+6. **Settings**: `nucleic.container.commandTracing` (`Project.swift:572-576`) is
+   superseded by `nucleic.container.shellCapture` = `off | meta | tap` (default `tap`),
+   mapped to `NUCLEIC_SHELL_CAPTURE`. A separate rollback lever
+   `nucleic.container.legacyShell` (default false) injects `NUCLEIC_ASH_DISABLE=1` and
+   restores the tracer wiring — one toggle to fully revert behavior without an image
+   rollback.
+
+---
+
+## 10. Testing
+
+1. **Fork conformance (CI-gating)**: brush's ~1,700-case bash compat suite must pass on
+   the fork with observation **on** — proving hooks don't perturb semantics. Run twice
+   (`capture=off`, `capture=tap`) and diff outcomes.
+2. **Tap semantics (Rust integration tests, `ash-observe`)**: append preserves
+   interleaving under concurrent writers; seek-after-redirect (`dd of=…`) byte-identical
+   to bash; `set -o pipefail` exit codes through tees; SIGPIPE propagation through tees
+   (`yes | head`); binary data through pipes uncorrupted (hash comparison); `/dev/null`,
+   fifos, `2>&1` dup chains; heredoc capture; 100 MB stream under cap → correct byte
+   count, bounded memory.
+3. **Transcript replay corpus**: extract the Bash-tool command corpus from existing
+   session transcripts (fixtures pattern per OBSERVABILITY_AND_TESTING.md) and replay
+   under ash vs bash in the sandbox image, diffing stdout/stderr/exit. This is the M0 gate
+   and a permanent regression suite.
+4. **Swift tests**: `ShellReportParsingTests` (batch → calls) beside
+   `MCPApprovalServerTests`; `ContainerSandboxTests` extension asserting the v8 image
+   resolves `/bin/sh → ash` and that a `linux_container` exec produces exec + pipe events
+   end-to-end; dedupe tests for the git-shim coexistence window.
+5. **Fault injection**: control socket absent (spool fills, commands unaffected); host 401
+   (posting stops, commands unaffected); `ash-observe` panic (hook boundary catches;
+   command completes).
+
+---
+
+## 11. Rollout phases
+
+| Phase | Deliverable | Exit criteria |
+| --- | --- | --- |
+| **M0 — spike** | Fork subtree; `ash -c` runs via embedded brush; transcript-replay corpus harness | ≥99% corpus parity vs bash; list of divergences filed |
+| **M1 — exec events** | Observer trait patch; exec/cd/fallback events; unix-socket transport; `/shell-event` route → feed; `SHELL`+`linux_container` argv switched (no divert yet); tracer retired | Feed shows ash events for a live session; tracer deleted; compat suite green |
+| **M2 — data-flow taps** | Redirect read-back + pipe/heredoc tees; caps + redaction; UI preview rows | Tap test matrix green; overhead <3% on corpus wall-clock |
+| **M3 — force sandbox/runner** | v8 image with divert; lockstep bumps; custom-image probe/degrade; `legacyShell` rollback lever | Week of dogfood sessions, zero fallback-event regressions unresolved |
+| **M4 — Linux VM** | agent.c exec swap + provisioner install + spool drain | `linux_vm_exec` events in feed; VM compat parity |
+| **M5 — macOS VM (gated)** | universal binary, `Ops+Exec` swap with zsh escape hatch | zsh-ism fallback rate ≈0 in dogfood, else hold |
+| **M6 — converge** | git/gh shim retirement behind ash-native classification; upstream the Observer trait | locks/autoship driven by ash events for a full dogfood cycle |
+
+---
+
+## 12. Risks and mitigations
+
+| Risk | Mitigation |
+| --- | --- |
+| brush compat gaps break agent commands | Parse-fallback to preserved real bash + `fallback` telemetry; kill switch env; `legacyShell` setting; M0 corpus gate before any forcing |
+| Tee alters semantics for seek-dependent tools | Design rule: regular files never tee'd (real fd + read-back); tees only where a pipe already exists |
+| Performance (async tees, event posting) | Static musl binary (no Node cold start — strictly faster than today's shims); fire-and-forget posts; <3% overhead gate in M2 |
+| Data volume / secrets in captured bytes | Caps at three levels; in-guest redaction before transport; previews only, never full payloads; capture level setting incl. `meta`-only |
+| Upstream drift of brush fork | Subtree + minimal hook-shaped patch; aim to upstream the Observer trait (fork → zero) |
+| `ash` name collision (Almquist) | No busybox in our images; version string + `NUCLEIC_ASH=1` disambiguate; rename is cheap if ever needed |
+| Image/tag lockstep mistakes | The four-place v8 bump listed in §7.1 is a single PR checklist; CI asserts Dockerfile tag == `Project.swift` tag |
+| Agent bypasses ash (`exec /usr/bin/bash.real`) | Accepted (non-goal §1); the exec event recording the bypass is itself the observability win; `.real` paths can be policy-flagged in feed classification |
+| macOS zsh-isms | M5 gated; per-op `shell:` escape hatch; auto-fallback to zsh on parse failure |