19 KiB
narOS kernel gate — kernel-level observation and gating of agent file I/O
Status: EXPLORATION — nothing here is locked. This doc maps the design space for
enforcing harness file locks (LOCKING.md) below user space, proposes a concrete
mechanism (owning the sandbox guest kernel + a small in-kernel naros_gate LSM), and
sequences it. It deliberately amends one NAROS.md non-goal: "building our own kernel"
stops being a non-goal for the surfaces where Nucleic already supplies the kernel
binary (the container sandbox and the Linux VM). narOS still owns no kernel where it
can't (the Cloudflare runner).
Related docs: LOCKING.md (the lock model being enforced), NASH.md §4.2 (the user-space gate this backs up), NAROS.md (§2.3 kernel identity — partially retired by this), CONTAINER_ISOLATION.md (the vminitd patch/ship pipeline this reuses), VSOCK_CONTROL_PLANE.md, LINUX_VM.md.
1. Problem and threat model
Every enforcement point Nucleic has today is user-space and cooperative:
| Layer | Where | What it stops |
|---|---|---|
| PreToolUse edit gate | ClaudeCodeBackend.swift:1471-1503 → AppStore.arbitrate (AppStore.swift:4202) |
Edit/Write/MultiEdit/NotebookEdit on locked paths — the only hard gate |
check_conflict MCP tool |
MCPApprovalServer.swift:1330 |
Nothing — advisory probe the agent calls voluntarily |
| WorktreeMutationGuard | WorktreeMutationGuard.swift:24 |
A known list of raw-shell mutators (sed -i, tee, dd of=, redirects, …) — deliberately conservative heuristic |
nash Gate |
third_party/brush/brush-core/src/gate.rs:99 |
Nothing yet — verdict-shaped but allow-all (nash-observe/src/lib.rs:367); /shell-gate reserved, unbuilt (NASH.md §4.2) |
The bypass surface is structural, not a bug list: any file write that isn't one of four
structured tools and isn't on the mutation guard's list goes through ungated — a
three-line python script, an unlisted utility, an interpreter heredoc, a build step
with a side effect. Reads are gated nowhere. And the agent user holds NOPASSWD:ALL
sudo by design (NAROS.md §6.1) — so any guest user-space enforcement added later
(a fanotify daemon, an auditd rule, a detachable BPF program, an in-guest wrapper) is
also removable by the thing it polices. nash forcing narrows the practical gap
enormously, but nash is still a process the agent can decline to use (a static binary
that never execs a shell sees no nash).
Threat model for this doc: an agent with root in the guest, doing arbitrary
syscalls (including io_uring), trying to read or write a path the harness has locked
or protected. Out of scope: an agent that compromises the guest kernel (see §8 for
the one design that survives even that, kept on the shelf), and the macOS surfaces
(NASH.md §7.5 territory).
The conclusion the threat model forces: the only tamper-proof place Nucleic controls that sits in the file-I/O path is the guest kernel — and on the two surfaces that matter most, Nucleic already supplies that kernel binary and can swap it freely.
2. What Nucleic actually controls, per surface
| Surface | Kernel | File path to project tree | Kernel-gate verdict |
|---|---|---|---|
Sandbox containers / linux_container |
Host-supplied vmlinux-arm64: Kata prebuilt via scripts/fetch-kernel.sh, or GHCR nucleic-kernel OCI artifact (ensureKernel, ContainerEngine.swift:319; kernel-image.yml); cmdline host-owned (ContainerEngine.swift:304-305); full kernel build toolchain already vendored (third_party/containerization/kernel/{build.sh,config-arm64}) |
Apple in-VMM virtiofs share at the identical host path (ContainerEngine.swift:579) |
Fully feasible — swap one blob, no image/rootfs change |
| Linux VM | Host-supplied raw Image via VZLinuxBootLoader (MacVMEngine+LinuxConfig.swift:34-41); today Ubuntu's kernel, but the URL is a setting |
virtiofs tag nucleicshare, bind-mounted to host paths |
Feasible — needs a matching modules story; aligns with narOS N4 |
| Cloudflare runner | Cloudflare's Firecracker host kernel (-cloudflare-firecracker) — immutable to us |
Container-local ephemeral disk (git clone) | Not possible. Ceiling here = nash Gate + /shell-gate |
| macOS VM / host Mac | macOS | virtiofs / local | Out of scope (Linux-only, as narOS) |
Two facts make the container surface unusually cheap to convert:
- The kernel is already a swappable, published artifact.
ensureKernelresolution order ends at a GHCR OCI pull keyed byProjectSandbox.kernelImage;kernel-image.ymlpublishes the blob. Shipping a self-built kernel is: build →oras pushat a new tag → bump one pin.NUCLEIC_KERNEL_PATHis the dev override, and the old Kata tag is a one-line rollback — the same lever culture aslegacyShell/vminitReference. - Session identity already exists in-kernel. CONTAINER_ISOLATION.md patch #9 puts
every session exec in its own cgroup (
/container/<id>/exec-<execID>). A kernel hook can therefore attribute any task to a session by walking its cgroup — exactly the granularityLockManagerlocks are held at, including inside the shared control container.
3. Mechanism survey
What could observe/gate reads and writes at or below the syscall boundary, judged
against: (a) survives guest root, (b) supports dynamic policy (locks come and go
mid-session), (c) covers all I/O paths (incl. io_uring, mmap), (d) cost to build
and carry.
| Mechanism | Verdict | Why |
|---|---|---|
fanotify permission events (FAN_OPEN_PERM, pre-content) |
Observation: good. Enforcement: rejected | Daemon lives in guest user space — root kills it or unmarks the mount. Fine as a no-kernel-rebuild observation fallback (CONFIG_FANOTIFY=y already), useless as a wall |
| seccomp user-notify on open/write syscalls | Rejected | Filters are irremovable once applied (good) but path-argument TOCTOU is unsolvable in general, io_uring ops bypass the syscall filter entirely, supervisor death = guest-wide DoS, and per-syscall round-trips are slow |
| Landlock | Rejected | Self-sandboxing and monotonic — a process can only tighten its own rules, never lift them, so dynamic lock release can't be expressed; agent-spawned processes must opt in |
| BPF-LSM | Dogfood tier only | Right hooks, fast iteration, no kernel patch (config + lsm= change only) — but guest root holds CAP_BPF/CAP_SYS_ADMIN and can detach programs. Great for K1 observation; not a wall |
| Mount topology (don't share locked paths; RO subtree remounts) | Rejected as primary | Sound even against guest kernel compromise, but virtiofs shares are per-directory and locks are per-file and dynamic; remount churn mid-session, coarse granularity. Kept as a coarse backstop for statically-known secrets |
| Host-side virtiofs interposition | Shelf (§8) | The only design outside the guest's TCB entirely — but the virtiofs server is Apple's, in-VMM, no backend API; replacing it means our own FUSE-over-vsock file server, a big perf/complexity bet |
| Custom in-kernel LSM in a self-built kernel | Proposed | Below every I/O path (LSM hooks fire for io_uring and mmap too), not detachable (built-in, no module unload), policy can be sealed against guest root (§4.4), dynamic by construction, and the delivery pipeline already exists |
4. Proposal: the narOS kernel + naros_gate
4.1 K0 — own the kernel (no behavior change)
Replace the fetched Kata blob with a self-built narOS kernel: same LTS line Kata
tracks (≥ 6.12, VZ-compatible), config seeded from the vendored
third_party/containerization/kernel/config-arm64, built by the vendored build.sh in
CI, published through the existing kernel-image.yml → nucleic-kernel GHCR channel.
Nucleic patch content at K0: none — K0 is purely proving the build/publish/boot loop and
taking ownership of the config (the fetched Kata kernel's config isn't even ours today;
the cmdline requests lsm=…,landlock,apparmor that the vendored config doesn't build).
Free win: CONFIG_LOCALVERSION="-naros" gives uname -r = 6.x.y-naros for real on
this surface — the naros-identity preload shim (NAROS.md §2.3) stays only for surfaces
on foreign kernels (runner), and its passthrough caveats vanish where the narOS kernel
boots.
Cost owned deliberately: kernel CVE/rebuild cadence stops being Kata's problem. Mitigate
by staying on the same LTS line Kata publishes (their bumps signal ours), and by the
narOS channel machinery — edge kernel builds ride the weekly narOS cadence, stable
moves with the app pin.
4.2 K1 — observation first (report-only, fail-open)
Mirror nash's locked "hooks now, enforce later" (NASH.md §4.2 — same posture, one layer
down). naros_gate compiles in but only reports: opens/writes/renames/unlinks on
policy-watched paths (and nothing else — no firehose) into a ring buffer; vminitd
drains it over the existing vsock control plane; a new report route lands beside
git/gh/shell in MCPApprovalServer and feeds the same observeShellEvent-style seam →
feed rows + ConflictCoordinator ground truth.
This is immediately useful before any gating exists: the lock system currently infers write activity from tool calls and shell parsing; K1 gives it the actual write set — including everything the mutation guard misses — so lock-release confidence and "session A touched locked path P" detection stop being heuristic. It also produces the data to size K2's semantics (how often would a deny have fired, and on what).
Scoping for cost: hooks check only tasks inside session cgroups, only on the virtiofs
superblock(s), and only against a usually-tiny policy table (a session holds locks on a
handful of paths). Everything else exits the hook in a few branches. Gate target: same
bar as nash — <3% on the corpus replay, expected far lower since file_permission
fast-paths on an inode flag set at lookup.
4.3 K2 — gating
Flip watched entries from report to enforce. Semantics, mapped onto LOCKING.md:
- Policy entry:
(path prefix, mode mask R/W, holder cgroup(s), report|enforce). Written by the host wheneverLockManagergrants/releases — the kernel table is a projection of the host's authoritative table, never a second brain. - Non-holder write to a locked path →
EPERMimmediately + an event. No in-kernel parking/queueing: the fair waiter queue, deadlock detection, and arbitration stay inLockManagerwhere they are; a denied agent gets a normal permission error plus (via the event → feed → nash/check_conflictguidance) an explanation and the existing park-and-retry path. Deny-fast avoids wedging builds on a kernel-held sleep and keeps kernel policy stateless. - Reads ungated for locks (locks never restricted reads) — but the mode mask means
read-gating is a policy row, not a feature: locked-out secrets (
.env, credential files), append-only audit paths, or a future "quarantine this diff from other sessions" all reuse the same hook. That's the "other potential future features" headroom: the kernel grows a mechanism once; features are policy rows. - Fail posture: fail-open by absence (no policy table → allow everything, exactly
like
ConflictCoordinatortoday) but fail-closed per entry (an enforce row denies until released — a lost release update means an over-blocked agent, never an under-blocked one). - Kill switches, in the house style: a
legacyKernelsettings lever repoints the pin at the last Kata tag; per-bootnaros_gate.mode=offon the cmdline; per-session opt-out by simply not pushing policy.
4.4 Policy channel and sealing
v1 (K1/K2): host → vminitd RPC (SetFilePolicy, one new proto beside the existing exec
plumbing) → vminitd writes the table to /sys/kernel/security/naros_gate/policy.
Honest caveat: guest root could also write that file. This still closes the entire
practical bypass class (agents write files with python, not by attacking PID 1), and
it ships without cryptography.
v2 (K3, sealing): make the kernel accept only host-authored policy. The host generates a
per-boot secret, passes it on the kernel cmdline (host composes the cmdline —
ContainerEngine.swift:304), and naros_gate consumes and scrubs it from
/proc/cmdline during early init, before any user space exists. Policy blobs are then
HMAC'd by the host with monotonic sequence numbers; vminitd (or anything — even the
agent) is a dumb relay; the kernel verifies. Guest root can drop updates (DoS: a
missed release leaves it over-blocked — self-punishing) but cannot forge, replay, or
roll back one. The LSM can emit a keyed receipt per applied update so the host knows its
projection is current. At v2 the gate is sound against everything short of a kernel
exploit.
4.5 What naros_gate actually is
Deliberately small: one out-of-tree LSM (~500–800 lines of C, single file + Kconfig),
registered in CONFIG_LSM/lsm=, no module (unloadable ⇒ pointless). Hooks:
file_open (mode-masked check at open), file_permission (inode-flagged fast path for
read/write on already-open fds — this is what catches mmap-adjacent and long-lived-fd
cases and what io_uring funnels through), path_rename/path_unlink/path_link
(the rename-over-a-locked-file dodge), sb_umount (can't unmount the share to confuse
path matching). Path matching is prefix-on-virtiofs-mount, mirroring
ConflictDetector.pathsOverlap componentwise semantics so kernel and host never
disagree about what a lock covers. Rebases across LTS bumps are trivial at this size;
the LSM hook API is the most stable extension surface the kernel has.
5. Per-surface adoption
- Sandbox containers /
linux_containerfirst (K0–K2): kernel is one blob, no modules to ship (naros-init loads none), rollback is a pin. This covers the surface where concurrent-session locking actually bites today. - Linux VM (K3-era): boot the narOS kernel via the existing
linuxKernelURLsetting; needs a module set for the desktop flavor (virtio-gpu etc.), so headless (narOS N4) adopts first — natural alignment, since N4 already rebuilds the guest rootfs. Spool-drain observation (NASH.md §6.2) gains the same kernel event stream vianucleic-linux-agent. - Cloudflare runner: permanent documented gap. No kernel control under Firecracker.
The runner's ceiling is the user-space stack — nash forced by narOS N3 plus, when
enforcement is wanted there, activating the reserved
/shell-gateverdict path (NASH.md §4.2). Single-session runners also don't contend for locks, so the gap is mostly theoretical for the lock feature specifically.
6. Milestones
| Phase | Deliverable | Exit criteria |
|---|---|---|
| K0 — own the kernel | CI-built narOS kernel (vendored toolchain, our config, -naros LOCALVERSION), published via kernel-image.yml; pin flip + legacyKernel rollback lever |
Boots every ContainerSandboxTests surface; parity sweep clean; rollback exercised; two releases cut through the channel |
| K1 — observe | naros_gate report-only; SetFilePolicy RPC; ring-buffer → vminitd → vsock → feed route; lock table projected as watch-only rows |
Kernel-observed write set visible in feed; zero false gating (report-only); overhead <3% on corpus replay in-image; miss-rate of WorktreeMutationGuard quantified from real dogfood |
| K2 — gate | Enforce rows for locked paths (EPERM + event); host projection wired into LockManager grant/release; kill switches |
A scripted bypass suite (python write, unlisted utility, io_uring open, rename-over, static binary) is denied on locked paths and allowed after release; one clean dogfood week with enforcement on |
| K3 — seal + VM | Cmdline-key HMAC policy channel with scrub + receipts; narOS kernel on the headless Linux VM | Forged/replayed policy rejected in test; guest-root tamper test can only DoS; VM surface streams the same events |
| K4 — features on the mechanism | First non-lock policy rows (secret-path read denial, append-only audit) | Feature ships as policy rows only — no kernel change required |
7. Testing
- Bypass suite (the point of the whole thing): in-image tests that attempt every
known dodge — interpreter one-liners, unlisted mutators, redirects via exotic
spellings,
io_uringopen/write,mmapafter open, rename/link-over, raw-syscall static binary, sudo'd everything — against a locked path. K1 asserts they're seen; K2 asserts they're denied; both assert unlocked paths are untouched. - Parity/regression: full nash corpus replay + parity sweep on the narOS kernel at every phase (a kernel swap can break things nash never could); ContainerSandboxTests for the RPC, event route, and lever.
- Attribution: shared control container with two session execs — holder writes
freely, sibling is denied, cgroup-walk attribution matches
LockManager's holder. - Sealing (K3): forge, replay, rollback, and drop tests against the HMAC channel;
/proc/cmdlinescrub asserted from guest root. - Performance: corpus replay delta + a hot-loop microbench (
read(2)on flagged vs unflagged inodes) with budgets asserted in CI.
8. Kept on the shelf
- Host-side file service (our own FUSE-over-vsock/virtiofs server replacing Apple's
in-VMM share): the only design whose enforcement survives guest-kernel compromise,
because the bytes never enter the guest's TCB ungated. Rejected for now on
perf/complexity (reimplementing a file server Apple gives us for free, in the hot path
of every build), but it composes with — and could later subsume —
naros_gate's policy table unchanged. Revisit if the threat model ever includes kernel exploits. - BPF-LSM variant of K1: if K0 stalls, the observation tier can ship on the stock
Kata kernel rebuilt only with
CONFIG_BPF_LSM+lsm=…,bpf, with vminitd loading the program. Loses tamper-resistance (irrelevant for report-only) and adds a BTF/toolchain dependency; kept as the fallback path to K1's data without committing to K2's carrier. - fanotify observation without any kernel change at all: weakest, but zero build cost; only worth it if both K0 paths stall.
9. Risks
| Risk | Mitigation |
|---|---|
| Kernel ownership burden (CVEs, LTS bumps) | Track Kata's line (their bump = our signal); tiny out-of-tree patch surface; edge/stable channel machinery; legacyKernel pin rollback keeps the Kata kernel bootable forever |
| Gate denies legitimate work (false positives wedge a session) | K1 report-only phase sizes this with real data before any deny ships; deny-fast + feed explanation + existing arbitrate/park path for recovery; per-session opt-out |
| Kernel/host lock-table divergence | Kernel table is a projection, never authoritative; prefix semantics copied from ConflictDetector.pathsOverlap; K3 receipts confirm applied seq |
| Perf regression on hot file I/O | Inode-flag fast path, cgroup + superblock scoping, CI-asserted budgets, corpus replay gate |
| vminitd relay compromised (v1 window) | Acknowledged in §4.4; closed by K3 sealing; until then the gate still beats every existing layer (which share the same trust in guest user space and are weaker) |
| Apple containerization framework drift (cmdline, VZ kernel expectations) | Kernel stays VZ-compatible on the same LTS line the framework's own make fetch-default-kernel uses; vendored framework pins the contract |