nucleic/docs/NASH_STREAM_PERF_PLAN.md

Nash stream performance: findings & optimization plan

Investigated 2026-07-29; P1–P4 and the regression guard implemented 2026-07-29. Companion to NASH.md (§5 data-flow taps, §6 transport, §9 host ingest) and MAIN_THREAD_PERFORMANCE_PLAN.md.

Status: all four phases shipped. What landed, and where it deviates from the plan below, is recorded in §4 Outcome.

Nash's observation stream was suspected of significantly affecting performance. This doc records what was measured, where the cost actually is, and a prioritized plan to reduce it.

TL;DR: the dominant cost is not the shell-side tee — it is the host app's per-event, main-thread ingest. The shell side mostly costs CPU (not wall time) plus a small per-pipeline setup tax; the host side turns every shell event from every concurrent agent into an unstructured Task, a MainActor hop, a full copy-on-write ring copy, an @Observable invalidation, and an os_log line — individually cheap, multiplied by thousands of events per minute during builds.

---

1. How the stream works (as shipped)

• Capture (guest, brush-core fork). When a gate is installed, every pipeline link a | b gets two pipes instead of one plus a freshly spawned nash-pipe-tee thread that copies all bytes through userspace in 32 KiB chunks, mirroring a 64 KiB prefix (third_party/brush/brush-core/src/interp.rs, nash_spawn_pipe_tee). Redirects hand the child the real fd and read back ≤64 KiB post-exit; cmdsub output is hashed/previewed from memory.
• Eventing (guest, shell/nash-observe). Events flow through a bounded channel (drop-not-block) to a flusher thread batching 200 events / 500 ms; batches are one JSON body POSTed over a unix socket or TCP with Connection: close, falling back to a JSONL disk spool. Redaction + base64 of previews happens at event-construction time — i.e. on the interpreter thread (cmdsub), the tee thread (pipes), or the on_exit command path (redirect read-back).
• Ingest (host). MCPApprovalServer.handleShellEvent parses the batch and fans it out one handler call per event (MCPApprovalServer.swift:3304). Each registered handler wraps the call in an unstructured Task (ClaudeCodeBackend.swift:936 and three sibling sites), which awaits @MainActor AppStore.observeShellEvent (AppStore.swift:5660) → NashEventLog.record (@Observable, 1000-row ring per session) → per-event CMDTRACE os_log, plus the observeCommand classify path for non-git execs.

Note for anyone re-measuring: the agent sandbox's /bin/bash is nash with require_observation=1 posting to the host. Benchmarks must use /usr/bin/bash.real as the baseline and a root container (edit /etc/nucleic/nash.conf) to get a truly unobserved nash. In-sandbox comparisons silently benchmark nash against itself.

2. Measurements

Best-of-3 wall / CPU (user+sys) ms, Linux (aarch64) container, release build at 20c2c31c. "off" = gate inactive (no tee, no events); "obs" = observation to a local spool (deployed capture config minus network).

Case	bash.real	nash off	nash obs	obs vs bash
512 MiB through a|b	755 / 544	1129 / 744	1030 / 918	+36% wall / +69% CPU
512 MiB through a|b|c	1036 / 912	943 / 880	564 / 1078	−46% wall / +18% CPU
2 GiB /dev/zero → wc -c	1010 / 1665	909 / 1433	1226 / 1774	+21% wall / +7% CPU
300 tiny pipelines	109 / 152	132 / 190	148 / 232	+36% wall / +53% CPU
1000 bare execs	252 / 268	339 / 401	370 / 444	+47% wall / +66% CPU
300 > file redirects	28 / 8	41 / 14	63 / 29	+125% wall (0.12 ms/redirect)
50 × < 32MiB reads	191 / 190	212 / 214	200 / 209	~noise

Readings:

• The tee costs CPU, not necessarily wall. On idle cores the extra copier thread actually adds pipelining (the 3-stage case got faster). But the deployed reality is many agents sharing one box: the +20–70% CPU and doubled context switches come out of everyone's wall clock under contention.
• Per-unit costs: ~0.13 ms per observed pipeline (2 pipes + thread spawn + events), ~+0.03 ms per observed command, ~0.12 ms per write-redirect read-back. Small numbers that build-system shell storms (configure scripts, npm/make recursion) multiply by thousands.
• Brush baseline, not the stream: nash unobserved is already ~+35% per bare exec vs bash. That is brush's spawn path, a separate workstream — worth tracking, but disabling observation would not recover it.
• Host-side volume: this modest benchmark alone produced ~9,000 events (2.8 MB spooled JSON). Each pipe/redirect event can carry ~85 KB of base64 preview; a 200-event batch can approach 17 MB.
• Anomaly (coverage, not perf): x=$(cat 32MB-file) emitted no cmdsub event — the on_cmdsub hook (expansion.rs:1005) appears to miss the assignment path. Also nash was ~25× faster than bash on the huge-cmdsub case (bash's own buffer growth is the bottleneck there). File separately.

3. Plan

P1 — Batch-preserving host ingest (biggest win; no protocol change)

1. Fan out batches, not events. Change ShellReportHandler to take [ShellReportCall] and make handleShellEvent deliver the whole parsed batch: one Task, one MainActor hop per batch (≤200 events) instead of per event. This also fixes a latent correctness smell — per-event unstructured Tasks do not preserve batch order, so exec-start/exec/pipe rows can land out of order today.
2. Batch append in NashEventLog. Add record(batch:): one pass, one ring trim, one @Observable invalidation per batch. Independently, fix the per-event copy — var bucket = eventsBySession[id] ?? []; bucket.append(…); eventsBySession[id] = bucket forces a full CoW copy of the 1000-row ring (thousands of retain/releases) per event; eventsBySession[id, default: []].append(…) mutates in place via _modify.
3. Tame CMDTRACE. Demote per-event .notice lines to .debug (keep .error for policy events); os_log formatting with public interpolation is measurable at this volume.
4. Coalesce UI updates (optional, after 1–2): buffer feed-visible mutations and flush at ~10 Hz during bursts — the panel cannot meaningfully render 500 rows/sec anyway.

P2 — Cheaper tee (guest, brush-core)

1. splice(2) passthrough on Linux. Capture the 64 KiB prefix with ordinary reads, then switch the loop to splice src→dst: zero userspace copies for the bulk of the stream, and total_bytes comes free from splice's return value. This removes essentially all tee CPU past the first 64 KiB. (NASH.md §5.2b already anticipates a "splice-style passthrough".)
2. Portable fallback (macOS): grow the copy buffer 32 KiB → 128–256 KiB, and on Linux also F_SETPIPE_SZ the pipes up — 4–8× fewer syscalls/context switches per MB even without splice.
3. Cut per-pipeline setup: reuse tee threads via a small pool (or at minimum spawn with a reduced stack) to shave the ~0.13 ms/pipeline that shell-heavy builds multiply.

P3 — Move encoding off the hot paths (guest, nash-observe)

1. Carry raw captured bytes in events; run redact + base64 + FNV on the flusher thread just before serialization instead of on the interpreter/tee/exit paths.
2. Move redirect read-back (metadata/open/read ≤64 KiB) from on_exit — the shell's command path — to the flusher. It is post-hoc file I/O by design; nothing requires it inline. (Bounded staleness is acceptable and already inherent.)
3. Replace the format!-per-byte hex encoding of binary previews with a lookup table (micro, but free).

P4 — Transport polish (guest ↔ host)

1. Keep the unix-socket/TCP connection alive across batches (drop Connection: close), or at least reuse one connection per flusher lifetime.
2. Consider a per-batch preview-byte budget so a pipeline-storm batch does not ship 200 × 85 KB; drop previews (keep counts/hashes) beyond the budget and mark the events, mirroring the existing dropped doctrine.

Regression guard

Extend shell/corpus/overhead.py:

• add a CPU-time column (rusage) alongside wall — the tee's cost is invisible to a wall-only gate on idle hardware;
• add a stream-throughput case (hundreds of MB through 1–2 links) so P2 is guarded;
• default --bash to bash.real and refuse to run if the baseline shell reports NUCLEIC_NASH=1, so the harness can never silently benchmark nash against itself.

Out of scope, filed separately

• on_cmdsub coverage gap for substitutions in assignments (x=$(…)).
• Brush's ~+35% bare-exec baseline vs bash (spawn-path work, independent of observation).

---

4. Outcome

What shipped

P1 — batch-preserving host ingest. ShellReportHandler takes [ShellReportCall]; handleShellEvent delivers one parsed batch per POST, through one Task and one MainActor hop, to ConflictArbiter.observeShellEvents(sessionID:calls:) → AppStore.observeShellEvents. NashEventLog.record(sessionID:calls:…) lands the whole batch in a single in-place mutation of the session ring (withRing, which threads _modify through the @Observable property and the dictionary subscript), with one trim and one invalidation for the batch instead of per event — and no copy-on-write of the 1000-row buffer, which the old read-modify-write did on every event. HostShellEventListener's app-scoped sink and the VM spool drain are batch-shaped too. Per-event CMDTRACE lines dropped to .debug; fallback, dropped and policy — the rare ones — kept their levels.

P2 — cheaper tee. After the 64 KiB prefix, a tapped link is moved with splice(2) on Linux (pipe-to-pipe in the kernel, byte count from the same call), falling back to a 128 KiB read/write loop on macOS or on EINVAL/ENOSYS/EPERM. Both pipes are grown with F_SETPIPE_SZ where permitted, and copier threads are pooled per process (≤4 idle, 512 KiB stacks) instead of spawned per link — with a pid check so a fork can never hand a job to a worker that only exists in the parent.

P3 — encoding off the hot paths. Data-flow events carry a Capture — raw bytes, or a measured-but-unread file range — and Serialize does the redaction, base64 and FNV hash, which means all of it runs on the flusher thread. Binary hex previews are table-driven.

P4 — transport. One connection, kept open across every batch of a shell's life (Connection: keep-alive, responses drained to their Content-Length so the socket stays framed, one retry on a connection the host has since closed). Plus NASH.md §5.4's per-batch preview budget, which was specified but never implemented: 1 MiB of payload per batch, spent in arrival order, beyond which events keep their counts and hashes and give up their previews.

Regression guard. overhead.py reports CPU (rusage) beside wall for every case, adds --stream-mb throughput cases (one and two links) gated on CPU, defaults --bash to /usr/bin/bash.real, and refuses a baseline that reports NUCLEIC_NASH=1 (probed with a scrubbed environment, since the harness itself usually runs under nash).

Measured (aarch64 Linux container, release build, observed → spool)

Case	before	after
256 MB through a|b	+0.2% wall / +11.9% CPU	−22.4% wall / −11.3% CPU
256 MB through a|b|c	+29.8% wall / +100.9% CPU	−35.8% wall / +11.4% CPU
300 tiny pipelines	0.220 s wall / 0.328 s CPU	0.209 s wall / 0.309 s CPU

The two-link stream case — the one that doubled CPU — now costs about a tenth of that, and both stream cases run faster than bash in wall time, because the tee's extra pipe adds pipelining the kernel now moves for free. The pipeline-storm case improves ~5% from the thread pool; most of what remains there is brush's spawn baseline, which is the separate workstream noted above.

Deviations from the plan

• P3.2 measures at exit, reads at flush. Moving the redirect read-back wholesale to the flusher is wrong, not merely stale: echo a > f; echo b >> f flushes as one batch, and by then the truncating write's read-back would report both lines. The command path keeps one stat (which also answers "is this a regular file at all"), fixing the range; everything else — open, read, hash, redact, encode — moved. Guarded by deferred_readback_reads_the_measured_range.
• Hashes are uniformly over the captured prefix. cmdsub used to hash its entire output, which meant x=$(cat 32MB) hashed 32 MB on the interpreter thread. Carrying those bytes to the flusher to preserve that would have cost more than the hash. Pipes already hashed the prefix; files never exceed it. NASH.md §5.4 now says so.
• P1.4 (UI coalescing) not done, and not needed yet. Batching already reduced the viewer's invalidations to one per POST — at most two per second per shell, which SwiftUI absorbs without a rate limiter. Worth revisiting only if a profile says otherwise.