# Nash stream performance: findings & optimization plan

*Investigated 2026-07-29; P1–P4 and the regression guard implemented 2026-07-29. Companion to
[NASH.md](NASH.md) (§5 data-flow taps, §6 transport, §9 host ingest) and
[MAIN_THREAD_PERFORMANCE_PLAN.md](MAIN_THREAD_PERFORMANCE_PLAN.md).*

**Status: all four phases shipped.** What landed, and where it deviates from the plan below, is
recorded in [§4 Outcome](#4-outcome).

Nash's observation stream was suspected of significantly affecting performance. This doc
records what was measured, where the cost actually is, and a prioritized plan to reduce it.

**TL;DR: the dominant cost is not the shell-side tee — it is the host app's per-event,
main-thread ingest.** The shell side mostly costs *CPU* (not wall time) plus a small
per-pipeline setup tax; the host side turns every shell event from every concurrent agent
into an unstructured `Task`, a MainActor hop, a full copy-on-write ring copy, an
`@Observable` invalidation, and an os_log line — individually cheap, multiplied by
thousands of events per minute during builds.

---

## 1. How the stream works (as shipped)

- **Capture (guest, brush-core fork).** When a gate is installed, every pipeline link
  `a | b` gets *two* pipes instead of one plus a freshly spawned `nash-pipe-tee` thread
  that copies all bytes through userspace in 32 KiB chunks, mirroring a 64 KiB prefix
  (`third_party/brush/brush-core/src/interp.rs`, `nash_spawn_pipe_tee`). Redirects hand
  the child the real fd and read back ≤64 KiB post-exit; cmdsub output is hashed/previewed
  from memory.
- **Eventing (guest, `shell/nash-observe`).** Events flow through a bounded channel
  (drop-not-block) to a flusher thread batching 200 events / 500 ms; batches are one JSON
  body POSTed over a unix socket or TCP with `Connection: close`, falling back to a JSONL
  disk spool. Redaction + base64 of previews happens at event-construction time — i.e. on
  the interpreter thread (cmdsub), the tee thread (pipes), or the `on_exit` command path
  (redirect read-back).
- **Ingest (host).** `MCPApprovalServer.handleShellEvent` parses the batch and fans it out
  **one handler call per event** (`MCPApprovalServer.swift:3304`). Each registered handler
  wraps the call in an unstructured `Task` (`ClaudeCodeBackend.swift:936` and three sibling
  sites), which awaits `@MainActor AppStore.observeShellEvent` (`AppStore.swift:5660`) →
  `NashEventLog.record` (`@Observable`, 1000-row ring per session) → per-event `CMDTRACE`
  os_log, plus the `observeCommand` classify path for non-git execs.

Note for anyone re-measuring: **the agent sandbox's `/bin/bash` is nash** with
`require_observation=1` posting to the host. Benchmarks must use `/usr/bin/bash.real` as
the baseline and a root container (edit `/etc/nucleic/nash.conf`) to get a truly
unobserved nash. In-sandbox comparisons silently benchmark nash against itself.

## 2. Measurements

Best-of-3 wall / CPU (user+sys) ms, Linux (aarch64) container, release build at
`20c2c31c`. "off" = gate inactive (no tee, no events); "obs" = observation to a local
spool (deployed capture config minus network).

| Case | bash.real | nash off | nash obs | obs vs bash |
|---|---|---|---|---|
| 512 MiB through `a\|b` | 755 / 544 | 1129 / 744 | 1030 / 918 | +36% wall / **+69% CPU** |
| 512 MiB through `a\|b\|c` | 1036 / 912 | 943 / 880 | 564 / 1078 | −46% wall / +18% CPU |
| 2 GiB `/dev/zero → wc -c` | 1010 / 1665 | 909 / 1433 | 1226 / 1774 | +21% wall / +7% CPU |
| 300 tiny pipelines | 109 / 152 | 132 / 190 | 148 / 232 | +36% wall / **+53% CPU** |
| 1000 bare execs | 252 / 268 | 339 / 401 | 370 / 444 | +47% wall / +66% CPU |
| 300 `> file` redirects | 28 / 8 | 41 / 14 | 63 / 29 | +125% wall (0.12 ms/redirect) |
| 50 × `< 32MiB` reads | 191 / 190 | 212 / 214 | 200 / 209 | ~noise |

Readings:

- **The tee costs CPU, not necessarily wall.** On idle cores the extra copier thread
  actually adds pipelining (the 3-stage case got *faster*). But the deployed reality is
  many agents sharing one box: the +20–70% CPU and doubled context switches come out of
  everyone's wall clock under contention.
- **Per-unit costs:** ~0.13 ms per observed pipeline (2 pipes + thread spawn + events),
  ~+0.03 ms per observed command, ~0.12 ms per write-redirect read-back. Small numbers
  that build-system shell storms (configure scripts, npm/make recursion) multiply by
  thousands.
- **Brush baseline, not the stream:** nash *unobserved* is already ~+35% per bare exec vs
  bash. That is brush's spawn path, a separate workstream — worth tracking, but
  disabling observation would not recover it.
- **Host-side volume:** this modest benchmark alone produced ~9,000 events (2.8 MB spooled
  JSON). Each pipe/redirect event can carry ~85 KB of base64 preview; a 200-event batch
  can approach 17 MB.
- **Anomaly (coverage, not perf):** `x=$(cat 32MB-file)` emitted **no** cmdsub event —
  the `on_cmdsub` hook (`expansion.rs:1005`) appears to miss the assignment path. Also
  nash was ~25× faster than bash on the huge-cmdsub case (bash's own buffer growth is the
  bottleneck there). File separately.

## 3. Plan

### P1 — Batch-preserving host ingest (biggest win; no protocol change)

1. **Fan out batches, not events.** Change `ShellReportHandler` to take
   `[ShellReportCall]` and make `handleShellEvent` deliver the whole parsed batch: one
   `Task`, one MainActor hop per *batch* (≤200 events) instead of per event. This also
   fixes a latent correctness smell — per-event unstructured `Task`s do not preserve batch
   order, so `exec-start`/`exec`/`pipe` rows can land out of order today.
2. **Batch append in `NashEventLog`.** Add `record(batch:)`: one pass, one ring trim, one
   `@Observable` invalidation per batch. Independently, fix the per-event copy —
   `var bucket = eventsBySession[id] ?? []; bucket.append(…); eventsBySession[id] = bucket`
   forces a full CoW copy of the 1000-row ring (thousands of retain/releases) per event;
   `eventsBySession[id, default: []].append(…)` mutates in place via `_modify`.
3. **Tame CMDTRACE.** Demote per-event `.notice` lines to `.debug` (keep `.error` for
   policy events); os_log formatting with public interpolation is measurable at this
   volume.
4. **Coalesce UI updates** (optional, after 1–2): buffer feed-visible mutations and flush
   at ~10 Hz during bursts — the panel cannot meaningfully render 500 rows/sec anyway.

### P2 — Cheaper tee (guest, brush-core)

1. **`splice(2)` passthrough on Linux.** Capture the 64 KiB prefix with ordinary reads,
   then switch the loop to `splice` src→dst: zero userspace copies for the bulk of the
   stream, and `total_bytes` comes free from splice's return value. This removes
   essentially all tee CPU past the first 64 KiB. (NASH.md §5.2b already anticipates a
   "`splice`-style passthrough".)
2. **Portable fallback (macOS):** grow the copy buffer 32 KiB → 128–256 KiB, and on Linux
   also `F_SETPIPE_SZ` the pipes up — 4–8× fewer syscalls/context switches per MB even
   without splice.
3. **Cut per-pipeline setup:** reuse tee threads via a small pool (or at minimum spawn
   with a reduced stack) to shave the ~0.13 ms/pipeline that shell-heavy builds multiply.

### P3 — Move encoding off the hot paths (guest, nash-observe)

1. Carry **raw captured bytes** in events; run `redact` + base64 + FNV on the flusher
   thread just before serialization instead of on the interpreter/tee/exit paths.
2. Move **redirect read-back** (metadata/open/read ≤64 KiB) from `on_exit` — the shell's
   command path — to the flusher. It is post-hoc file I/O by design; nothing requires it
   inline. (Bounded staleness is acceptable and already inherent.)
3. Replace the `format!`-per-byte hex encoding of binary previews with a lookup table
   (micro, but free).

### P4 — Transport polish (guest ↔ host)

1. Keep the unix-socket/TCP connection alive across batches (drop `Connection: close`),
   or at least reuse one connection per flusher lifetime.
2. Consider a **per-batch preview-byte budget** so a pipeline-storm batch does not ship
   200 × 85 KB; drop previews (keep counts/hashes) beyond the budget and mark the events,
   mirroring the existing `dropped` doctrine.

### Regression guard

Extend `shell/corpus/overhead.py`:

- add a **CPU-time** column (rusage) alongside wall — the tee's cost is invisible to a
  wall-only gate on idle hardware;
- add a **stream-throughput case** (hundreds of MB through 1–2 links) so P2 is guarded;
- default `--bash` to `bash.real` and refuse to run if the baseline shell reports
  `NUCLEIC_NASH=1`, so the harness can never silently benchmark nash against itself.

### Out of scope, filed separately

- `on_cmdsub` coverage gap for substitutions in assignments (`x=$(…)`).
- Brush's ~+35% bare-exec baseline vs bash (spawn-path work, independent of observation).

---

## 4. Outcome

### What shipped

**P1 — batch-preserving host ingest.** `ShellReportHandler` takes `[ShellReportCall]`;
`handleShellEvent` delivers one parsed batch per POST, through one `Task` and one MainActor hop,
to `ConflictArbiter.observeShellEvents(sessionID:calls:)` →
`AppStore.observeShellEvents`. `NashEventLog.record(sessionID:calls:…)` lands the whole batch in
a single in-place mutation of the session ring (`withRing`, which threads `_modify` through the
`@Observable` property and the dictionary subscript), with one trim and one invalidation for the
batch instead of per event — and no copy-on-write of the 1000-row buffer, which the old
read-modify-write did on *every* event. `HostShellEventListener`'s app-scoped sink and the VM
spool drain are batch-shaped too. Per-event `CMDTRACE` lines dropped to `.debug`; `fallback`,
`dropped` and `policy` — the rare ones — kept their levels.

**P2 — cheaper tee.** After the 64 KiB prefix, a tapped link is moved with `splice(2)` on Linux
(pipe-to-pipe in the kernel, byte count from the same call), falling back to a 128 KiB
read/write loop on macOS or on `EINVAL`/`ENOSYS`/`EPERM`. Both pipes are grown with
`F_SETPIPE_SZ` where permitted, and copier threads are pooled per process (≤4 idle, 512 KiB
stacks) instead of spawned per link — with a pid check so a `fork` can never hand a job to a
worker that only exists in the parent.

**P3 — encoding off the hot paths.** Data-flow events carry a `Capture` — raw bytes, or a
measured-but-unread file range — and `Serialize` does the redaction, base64 and FNV hash, which
means all of it runs on the flusher thread. Binary hex previews are table-driven.

**P4 — transport.** One connection, kept open across every batch of a shell's life
(`Connection: keep-alive`, responses drained to their `Content-Length` so the socket stays
framed, one retry on a connection the host has since closed). Plus NASH.md §5.4's per-batch
preview budget, which was specified but never implemented: 1 MiB of payload per batch, spent in
arrival order, beyond which events keep their counts and hashes and give up their previews.

**Regression guard.** `overhead.py` reports CPU (rusage) beside wall for every case, adds
`--stream-mb` throughput cases (one and two links) gated on CPU, defaults `--bash` to
`/usr/bin/bash.real`, and refuses a baseline that reports `NUCLEIC_NASH=1` (probed with a
scrubbed environment, since the harness itself usually runs *under* nash).

### Measured (aarch64 Linux container, release build, observed → spool)

| Case | before | after |
|---|---|---|
| 256 MB through `a\|b` | +0.2% wall / **+11.9% CPU** | −22.4% wall / **−11.3% CPU** |
| 256 MB through `a\|b\|c` | +29.8% wall / **+100.9% CPU** | −35.8% wall / **+11.4% CPU** |
| 300 tiny pipelines | 0.220 s wall / 0.328 s CPU | 0.209 s wall / 0.309 s CPU |

The two-link stream case — the one that doubled CPU — now costs about a tenth of that, and both
stream cases run *faster than bash* in wall time, because the tee's extra pipe adds pipelining
the kernel now moves for free. The pipeline-storm case improves ~5% from the thread pool; most of
what remains there is brush's spawn baseline, which is the separate workstream noted above.

### Deviations from the plan

- **P3.2 measures at exit, reads at flush.** Moving the redirect read-back wholesale to the
  flusher is wrong, not merely stale: `echo a > f; echo b >> f` flushes as one batch, and by then
  the truncating write's read-back would report both lines. The command path keeps one `stat`
  (which also answers "is this a regular file at all"), fixing the range; everything
  else — open, read, hash, redact, encode — moved. Guarded by
  `deferred_readback_reads_the_measured_range`.
- **Hashes are uniformly over the captured prefix.** `cmdsub` used to hash its *entire* output,
  which meant `x=$(cat 32MB)` hashed 32 MB on the interpreter thread. Carrying those bytes to the
  flusher to preserve that would have cost more than the hash. Pipes already hashed the prefix;
  files never exceed it. NASH.md §5.4 now says so.
- **P1.4 (UI coalescing) not done, and not needed yet.** Batching already reduced the viewer's
  invalidations to one per POST — at most two per second per shell, which SwiftUI absorbs
  without a rate limiter. Worth revisiting only if a profile says otherwise.
