Files
nucleic/docs/NAROS_KERNEL_GATE.md
T

19 KiB
Raw Blame History

narOS kernel gate — kernel-level observation and gating of agent file I/O

Status: EXPLORATION — nothing here is locked. This doc maps the design space for enforcing harness file locks (LOCKING.md) below user space, proposes a concrete mechanism (owning the sandbox guest kernel + a small in-kernel naros_gate LSM), and sequences it. It deliberately amends one NAROS.md non-goal: "building our own kernel" stops being a non-goal for the surfaces where Nucleic already supplies the kernel binary (the container sandbox and the Linux VM). narOS still owns no kernel where it can't (the Cloudflare runner).

Related docs: LOCKING.md (the lock model being enforced), NASH.md §4.2 (the user-space gate this backs up), NAROS.md (§2.3 kernel identity — partially retired by this), CONTAINER_ISOLATION.md (the vminitd patch/ship pipeline this reuses), VSOCK_CONTROL_PLANE.md, LINUX_VM.md.


1. Problem and threat model

Every enforcement point Nucleic has today is user-space and cooperative:

Layer Where What it stops
PreToolUse edit gate ClaudeCodeBackend.swift:1471-1503AppStore.arbitrate (AppStore.swift:4202) Edit/Write/MultiEdit/NotebookEdit on locked paths — the only hard gate
check_conflict MCP tool MCPApprovalServer.swift:1330 Nothing — advisory probe the agent calls voluntarily
WorktreeMutationGuard WorktreeMutationGuard.swift:24 A known list of raw-shell mutators (sed -i, tee, dd of=, redirects, …) — deliberately conservative heuristic
nash Gate third_party/brush/brush-core/src/gate.rs:99 Nothing yet — verdict-shaped but allow-all (nash-observe/src/lib.rs:367); /shell-gate reserved, unbuilt (NASH.md §4.2)

The bypass surface is structural, not a bug list: any file write that isn't one of four structured tools and isn't on the mutation guard's list goes through ungated — a three-line python script, an unlisted utility, an interpreter heredoc, a build step with a side effect. Reads are gated nowhere. And the agent user holds NOPASSWD:ALL sudo by design (NAROS.md §6.1) — so any guest user-space enforcement added later (a fanotify daemon, an auditd rule, a detachable BPF program, an in-guest wrapper) is also removable by the thing it polices. nash forcing narrows the practical gap enormously, but nash is still a process the agent can decline to use (a static binary that never execs a shell sees no nash).

Threat model for this doc: an agent with root in the guest, doing arbitrary syscalls (including io_uring), trying to read or write a path the harness has locked or protected. Out of scope: an agent that compromises the guest kernel (see §8 for the one design that survives even that, kept on the shelf), and the macOS surfaces (NASH.md §7.5 territory).

The conclusion the threat model forces: the only tamper-proof place Nucleic controls that sits in the file-I/O path is the guest kernel — and on the two surfaces that matter most, Nucleic already supplies that kernel binary and can swap it freely.

2. What Nucleic actually controls, per surface

Surface Kernel File path to project tree Kernel-gate verdict
Sandbox containers / linux_container Host-supplied vmlinux-arm64: Kata prebuilt via scripts/fetch-kernel.sh, or GHCR nucleic-kernel OCI artifact (ensureKernel, ContainerEngine.swift:319; kernel-image.yml); cmdline host-owned (ContainerEngine.swift:304-305); full kernel build toolchain already vendored (third_party/containerization/kernel/{build.sh,config-arm64}) Apple in-VMM virtiofs share at the identical host path (ContainerEngine.swift:579) Fully feasible — swap one blob, no image/rootfs change
Linux VM Host-supplied raw Image via VZLinuxBootLoader (MacVMEngine+LinuxConfig.swift:34-41); today Ubuntu's kernel, but the URL is a setting virtiofs tag nucleicshare, bind-mounted to host paths Feasible — needs a matching modules story; aligns with narOS N4
Cloudflare runner Cloudflare's Firecracker host kernel (-cloudflare-firecracker) — immutable to us Container-local ephemeral disk (git clone) Not possible. Ceiling here = nash Gate + /shell-gate
macOS VM / host Mac macOS virtiofs / local Out of scope (Linux-only, as narOS)

Two facts make the container surface unusually cheap to convert:

  1. The kernel is already a swappable, published artifact. ensureKernel resolution order ends at a GHCR OCI pull keyed by ProjectSandbox.kernelImage; kernel-image.yml publishes the blob. Shipping a self-built kernel is: build → oras push at a new tag → bump one pin. NUCLEIC_KERNEL_PATH is the dev override, and the old Kata tag is a one-line rollback — the same lever culture as legacyShell/vminitReference.
  2. Session identity already exists in-kernel. CONTAINER_ISOLATION.md patch #9 puts every session exec in its own cgroup (/container/<id>/exec-<execID>). A kernel hook can therefore attribute any task to a session by walking its cgroup — exactly the granularity LockManager locks are held at, including inside the shared control container.

3. Mechanism survey

What could observe/gate reads and writes at or below the syscall boundary, judged against: (a) survives guest root, (b) supports dynamic policy (locks come and go mid-session), (c) covers all I/O paths (incl. io_uring, mmap), (d) cost to build and carry.

Mechanism Verdict Why
fanotify permission events (FAN_OPEN_PERM, pre-content) Observation: good. Enforcement: rejected Daemon lives in guest user space — root kills it or unmarks the mount. Fine as a no-kernel-rebuild observation fallback (CONFIG_FANOTIFY=y already), useless as a wall
seccomp user-notify on open/write syscalls Rejected Filters are irremovable once applied (good) but path-argument TOCTOU is unsolvable in general, io_uring ops bypass the syscall filter entirely, supervisor death = guest-wide DoS, and per-syscall round-trips are slow
Landlock Rejected Self-sandboxing and monotonic — a process can only tighten its own rules, never lift them, so dynamic lock release can't be expressed; agent-spawned processes must opt in
BPF-LSM Dogfood tier only Right hooks, fast iteration, no kernel patch (config + lsm= change only) — but guest root holds CAP_BPF/CAP_SYS_ADMIN and can detach programs. Great for K1 observation; not a wall
Mount topology (don't share locked paths; RO subtree remounts) Rejected as primary Sound even against guest kernel compromise, but virtiofs shares are per-directory and locks are per-file and dynamic; remount churn mid-session, coarse granularity. Kept as a coarse backstop for statically-known secrets
Host-side virtiofs interposition Shelf (§8) The only design outside the guest's TCB entirely — but the virtiofs server is Apple's, in-VMM, no backend API; replacing it means our own FUSE-over-vsock file server, a big perf/complexity bet
Custom in-kernel LSM in a self-built kernel Proposed Below every I/O path (LSM hooks fire for io_uring and mmap too), not detachable (built-in, no module unload), policy can be sealed against guest root (§4.4), dynamic by construction, and the delivery pipeline already exists

4. Proposal: the narOS kernel + naros_gate

4.1 K0 — own the kernel (no behavior change)

Replace the fetched Kata blob with a self-built narOS kernel: same LTS line Kata tracks (≥ 6.12, VZ-compatible), config seeded from the vendored third_party/containerization/kernel/config-arm64, built by the vendored build.sh in CI, published through the existing kernel-image.ymlnucleic-kernel GHCR channel. Nucleic patch content at K0: none — K0 is purely proving the build/publish/boot loop and taking ownership of the config (the fetched Kata kernel's config isn't even ours today; the cmdline requests lsm=…,landlock,apparmor that the vendored config doesn't build).

Free win: CONFIG_LOCALVERSION="-naros" gives uname -r = 6.x.y-naros for real on this surface — the naros-identity preload shim (NAROS.md §2.3) stays only for surfaces on foreign kernels (runner), and its passthrough caveats vanish where the narOS kernel boots.

Cost owned deliberately: kernel CVE/rebuild cadence stops being Kata's problem. Mitigate by staying on the same LTS line Kata publishes (their bumps signal ours), and by the narOS channel machinery — edge kernel builds ride the weekly narOS cadence, stable moves with the app pin.

4.2 K1 — observation first (report-only, fail-open)

Mirror nash's locked "hooks now, enforce later" (NASH.md §4.2 — same posture, one layer down). naros_gate compiles in but only reports: opens/writes/renames/unlinks on policy-watched paths (and nothing else — no firehose) into a ring buffer; vminitd drains it over the existing vsock control plane; a new report route lands beside git/gh/shell in MCPApprovalServer and feeds the same observeShellEvent-style seam → feed rows + ConflictCoordinator ground truth.

This is immediately useful before any gating exists: the lock system currently infers write activity from tool calls and shell parsing; K1 gives it the actual write set — including everything the mutation guard misses — so lock-release confidence and "session A touched locked path P" detection stop being heuristic. It also produces the data to size K2's semantics (how often would a deny have fired, and on what).

Scoping for cost: hooks check only tasks inside session cgroups, only on the virtiofs superblock(s), and only against a usually-tiny policy table (a session holds locks on a handful of paths). Everything else exits the hook in a few branches. Gate target: same bar as nash — <3% on the corpus replay, expected far lower since file_permission fast-paths on an inode flag set at lookup.

4.3 K2 — gating

Flip watched entries from report to enforce. Semantics, mapped onto LOCKING.md:

  • Policy entry: (path prefix, mode mask R/W, holder cgroup(s), report|enforce). Written by the host whenever LockManager grants/releases — the kernel table is a projection of the host's authoritative table, never a second brain.
  • Non-holder write to a locked path → EPERM immediately + an event. No in-kernel parking/queueing: the fair waiter queue, deadlock detection, and arbitration stay in LockManager where they are; a denied agent gets a normal permission error plus (via the event → feed → nash/check_conflict guidance) an explanation and the existing park-and-retry path. Deny-fast avoids wedging builds on a kernel-held sleep and keeps kernel policy stateless.
  • Reads ungated for locks (locks never restricted reads) — but the mode mask means read-gating is a policy row, not a feature: locked-out secrets (.env, credential files), append-only audit paths, or a future "quarantine this diff from other sessions" all reuse the same hook. That's the "other potential future features" headroom: the kernel grows a mechanism once; features are policy rows.
  • Fail posture: fail-open by absence (no policy table → allow everything, exactly like ConflictCoordinator today) but fail-closed per entry (an enforce row denies until released — a lost release update means an over-blocked agent, never an under-blocked one).
  • Kill switches, in the house style: a legacyKernel settings lever repoints the pin at the last Kata tag; per-boot naros_gate.mode=off on the cmdline; per-session opt-out by simply not pushing policy.

4.4 Policy channel and sealing

v1 (K1/K2): host → vminitd RPC (SetFilePolicy, one new proto beside the existing exec plumbing) → vminitd writes the table to /sys/kernel/security/naros_gate/policy. Honest caveat: guest root could also write that file. This still closes the entire practical bypass class (agents write files with python, not by attacking PID 1), and it ships without cryptography.

v2 (K3, sealing): make the kernel accept only host-authored policy. The host generates a per-boot secret, passes it on the kernel cmdline (host composes the cmdline — ContainerEngine.swift:304), and naros_gate consumes and scrubs it from /proc/cmdline during early init, before any user space exists. Policy blobs are then HMAC'd by the host with monotonic sequence numbers; vminitd (or anything — even the agent) is a dumb relay; the kernel verifies. Guest root can drop updates (DoS: a missed release leaves it over-blocked — self-punishing) but cannot forge, replay, or roll back one. The LSM can emit a keyed receipt per applied update so the host knows its projection is current. At v2 the gate is sound against everything short of a kernel exploit.

4.5 What naros_gate actually is

Deliberately small: one out-of-tree LSM (~500800 lines of C, single file + Kconfig), registered in CONFIG_LSM/lsm=, no module (unloadable ⇒ pointless). Hooks: file_open (mode-masked check at open), file_permission (inode-flagged fast path for read/write on already-open fds — this is what catches mmap-adjacent and long-lived-fd cases and what io_uring funnels through), path_rename/path_unlink/path_link (the rename-over-a-locked-file dodge), sb_umount (can't unmount the share to confuse path matching). Path matching is prefix-on-virtiofs-mount, mirroring ConflictDetector.pathsOverlap componentwise semantics so kernel and host never disagree about what a lock covers. Rebases across LTS bumps are trivial at this size; the LSM hook API is the most stable extension surface the kernel has.

5. Per-surface adoption

  1. Sandbox containers / linux_container first (K0K2): kernel is one blob, no modules to ship (naros-init loads none), rollback is a pin. This covers the surface where concurrent-session locking actually bites today.
  2. Linux VM (K3-era): boot the narOS kernel via the existing linuxKernelURL setting; needs a module set for the desktop flavor (virtio-gpu etc.), so headless (narOS N4) adopts first — natural alignment, since N4 already rebuilds the guest rootfs. Spool-drain observation (NASH.md §6.2) gains the same kernel event stream via nucleic-linux-agent.
  3. Cloudflare runner: permanent documented gap. No kernel control under Firecracker. The runner's ceiling is the user-space stack — nash forced by narOS N3 plus, when enforcement is wanted there, activating the reserved /shell-gate verdict path (NASH.md §4.2). Single-session runners also don't contend for locks, so the gap is mostly theoretical for the lock feature specifically.

6. Milestones

Phase Deliverable Exit criteria
K0 — own the kernel CI-built narOS kernel (vendored toolchain, our config, -naros LOCALVERSION), published via kernel-image.yml; pin flip + legacyKernel rollback lever Boots every ContainerSandboxTests surface; parity sweep clean; rollback exercised; two releases cut through the channel
K1 — observe naros_gate report-only; SetFilePolicy RPC; ring-buffer → vminitd → vsock → feed route; lock table projected as watch-only rows Kernel-observed write set visible in feed; zero false gating (report-only); overhead <3% on corpus replay in-image; miss-rate of WorktreeMutationGuard quantified from real dogfood
K2 — gate Enforce rows for locked paths (EPERM + event); host projection wired into LockManager grant/release; kill switches A scripted bypass suite (python write, unlisted utility, io_uring open, rename-over, static binary) is denied on locked paths and allowed after release; one clean dogfood week with enforcement on
K3 — seal + VM Cmdline-key HMAC policy channel with scrub + receipts; narOS kernel on the headless Linux VM Forged/replayed policy rejected in test; guest-root tamper test can only DoS; VM surface streams the same events
K4 — features on the mechanism First non-lock policy rows (secret-path read denial, append-only audit) Feature ships as policy rows only — no kernel change required

7. Testing

  1. Bypass suite (the point of the whole thing): in-image tests that attempt every known dodge — interpreter one-liners, unlisted mutators, redirects via exotic spellings, io_uring open/write, mmap after open, rename/link-over, raw-syscall static binary, sudo'd everything — against a locked path. K1 asserts they're seen; K2 asserts they're denied; both assert unlocked paths are untouched.
  2. Parity/regression: full nash corpus replay + parity sweep on the narOS kernel at every phase (a kernel swap can break things nash never could); ContainerSandboxTests for the RPC, event route, and lever.
  3. Attribution: shared control container with two session execs — holder writes freely, sibling is denied, cgroup-walk attribution matches LockManager's holder.
  4. Sealing (K3): forge, replay, rollback, and drop tests against the HMAC channel; /proc/cmdline scrub asserted from guest root.
  5. Performance: corpus replay delta + a hot-loop microbench (read(2) on flagged vs unflagged inodes) with budgets asserted in CI.

8. Kept on the shelf

  • Host-side file service (our own FUSE-over-vsock/virtiofs server replacing Apple's in-VMM share): the only design whose enforcement survives guest-kernel compromise, because the bytes never enter the guest's TCB ungated. Rejected for now on perf/complexity (reimplementing a file server Apple gives us for free, in the hot path of every build), but it composes with — and could later subsume — naros_gate's policy table unchanged. Revisit if the threat model ever includes kernel exploits.
  • BPF-LSM variant of K1: if K0 stalls, the observation tier can ship on the stock Kata kernel rebuilt only with CONFIG_BPF_LSM + lsm=…,bpf, with vminitd loading the program. Loses tamper-resistance (irrelevant for report-only) and adds a BTF/toolchain dependency; kept as the fallback path to K1's data without committing to K2's carrier.
  • fanotify observation without any kernel change at all: weakest, but zero build cost; only worth it if both K0 paths stall.

9. Risks

Risk Mitigation
Kernel ownership burden (CVEs, LTS bumps) Track Kata's line (their bump = our signal); tiny out-of-tree patch surface; edge/stable channel machinery; legacyKernel pin rollback keeps the Kata kernel bootable forever
Gate denies legitimate work (false positives wedge a session) K1 report-only phase sizes this with real data before any deny ships; deny-fast + feed explanation + existing arbitrate/park path for recovery; per-session opt-out
Kernel/host lock-table divergence Kernel table is a projection, never authoritative; prefix semantics copied from ConflictDetector.pathsOverlap; K3 receipts confirm applied seq
Perf regression on hot file I/O Inode-flag fast path, cgroup + superblock scoping, CI-asserted budgets, corpus replay gate
vminitd relay compromised (v1 window) Acknowledged in §4.4; closed by K3 sealing; until then the gate still beats every existing layer (which share the same trust in guest user space and are weaker)
Apple containerization framework drift (cmdline, VZ kernel expectations) Kernel stays VZ-compatible on the same LTS line the framework's own make fetch-default-kernel uses; vendored framework pins the contract