Nucleic — Runtime & Persistence Architecture (v0)

Where the contracts meet. BACKEND_PROTOCOL defines the event/approval model, SYNC_PROTOCOL projects it to the phone, WORKTREE_MANAGER owns the git side. This doc defines the object graph, the single-writer event pipeline, the persistence schema, and the concurrency model that wire them together on the Mac host.

Status: design draft. Targets Swift 6 strict concurrency.

1. Object graph

                         ┌──────────────── AppStore (@MainActor, @Observable) ───────────────┐
                         │  projects, session list, selection, settings                       │
                         └───────┬───────────────────────────────────────────────┬───────────┘
                                 │ owns N                                          │ owns 1 each
                                 ▼                                                 ▼
                   ┌──────── SessionController (actor) ────────┐         WorktreeManager (actor)
   one per live    │  the hub for ONE session:                 │         ProcessHost (actor)
   session         │   • AgentBackend (adapter + child proc)   │         SyncServer (actor)
                   │   • Worktree handle                       │         NotificationService
                   │   • TranscriptWriter (assigns canon seq)  │         Database (GRDB pool)
                   │   • ApprovalCoordinator                   │
                   │   • derived Session status                │
                   │   • fan-out to subscribers ──────────────┼──▶ UI (MainActor view models)
                   └───────────────────────────────────────────┘  └▶ SyncServer ─▶ iPhone clients

The SessionController is the only writer of a session's state. Everyone else (UI, SyncServer, DB) subscribes. This single-writer rule is what makes ordering, persistence, and multi-client sync consistent without locks scattered everywhere.

2. The event pipeline (single writer, fan-out readers)

AgentBackend.start() ──AsyncThrowingStream<AgentEvent>──▶ SessionController.ingest(event)
                                                                │
   (also injected: setup-script logs, approvalResolved echoes, lifecycle markers)
                                                                ▼
                                          1. TranscriptWriter.append → assign CANONICAL seq
                                          2. update derived Session.status (state machine)
                                          3. persist metadata delta to GRDB (status, lastSeq, diffstat)
                                          4. yield to in-process subscribers (UI)
                                          5. SyncServer.broadcast(event, perSubscriberVerbosity)
                                          6. if status ∈ {awaitingApproval, awaitingInput}
                                                 → NotificationService.notify

2.1 Canonical `seq` (reconciles BACKEND_PROTOCOL §3)

The backend adapter stamps a provisional per-adapter seq for its own stream. But a session also emits synthetic events the adapter never saw — setup-script output, approvalResolved echoes, interrupted markers. So the canonical, session-wide seq is (re)assigned by TranscriptWriter at append time, the single serialization point. That canonical seq is what the transcript stores, the UI renders, and the sync cursor uses. (Amends BACKEND_PROTOCOL §3.1: the adapter's seq is an ordering hint within its own stream; the authority is the writer.)

2.2 Backpressure

ingest is an actor-serialized async call; the backend stream is consumed in a single task. If a slow consumer (a phone on cellular) can't keep up, that's absorbed in SyncServer's per-subscriber buffering + verbosity coalescing (SYNC_PROTOCOL §5.5) — never by blocking the writer. The transcript and UI always get the full firehose.

3. Key actors

Actor / type	Isolation	Responsibility
`AppStore`	`@MainActor`, `@Observable`	Top-level UI state: projects, session summaries, selection, settings. Spawns/owns `SessionController`s.
`SessionController`	`actor` (one per session)	Orchestrates a session end-to-end; the only writer of its state; fan-out hub.
`TranscriptWriter`	`actor` (one per session)	Append-only JSONL, assigns canonical `seq`, fsync policy.
`TranscriptReader`	value/service	Reads JSONL for resume + history paging (by `seq` offset).
`ApprovalCoordinator`	`actor` (one per session)	Holds outstanding `ApprovalRequest`s; bridges the backend's blocked approval waiter (Claude MCP call / Codex JSON-RPC request) to UI+sync; enforces first-responder-wins.
`WorktreeManager`	`actor`	Git/worktree ops; per-repo structural lock (WORKTREE_MANAGER §1).
`MergeQueue`	`actor`	Autoship: serializes finished sessions' merges per target branch; FIFO + per-session dedupe; reports status (§3.1).
`ProcessHost`	`actor`	Spawns/monitors child processes, line-buffered stdio, signal delivery, lifecycle.
`ContainerManager`	`actor`	Sandbox lifecycle policy: per-container busy ref-counts, idle timers, teardown, shared-control-container naming. Drives `ContainerEngine`.
`ContainerEngine`	`actor`	In-process container runtime on Apple's `containerization` framework (the mechanism `ContainerManager` drives): one shared VM manager (bundled kernel + vminitd), image store, `vmnet` network, and a registry of live `LinuxContainer`s. Daemonless → VMs are bounded by the app process; `reconcile` is on-disk GC. An agent run is an `exec` whose vsock stdio is adapted to `ProcessHandle` (`ContainerizedProcessHandle`), so backends drive it exactly like a host spawn.
`SyncServer`	`actor`	`SecureChannel`s to clients, subscription registry, per-subscriber broadcast/throttle.
`NotificationService`	`@MainActor` facade	Local notifications now; APNs later (SYNC_PROTOCOL §6).
`Database`	GRDB `DatabasePool`	WAL-mode SQLite; metadata only (transcripts live in JSONL).

UI never touches an AgentBackend or git directly — it sends intents to the SessionController (sendInput, respondToApproval, requestIntegrate) and observes state.

3.1 Autoship & the merge queue

Autoship (the Ship toggle, beside Auto) lets a session merge its own branch into the project's default branch the moment its work is done — no human babysitting the merge.

Trigger (disposition-gated). When a turn's disposition settles .completed, an autoship session enqueues a ShipRequest onto the MergeQueue (AppStore.shipIfCompleted, the same signal that frees a finished session's sandbox). Two callers reach it, because they are the same declaration that the work is finished: the model's own classification (AppStore.classifyDisposition), and the user's Mark Done from the sidebar or the phone (AppStore.markSessionDone, reached over the wire by ClientMsg.markSessionDone → markSessionDoneRemote) — the manual override for a chat the classifier left stuck on "Awaiting Input". It never fires mid-turn or while the agent is still asking a question: shipIfCompleted re-reads the live session and bails unless it is idle at .awaitingInput with .completed as its last disposition, so a Mark Done that setDisposition rejected (a running, blocked, or terminal chat) can't enqueue a phantom merge.
Implies Auto. Shipping requires an autonomous run, so enabling Ship enables Auto; disabling Auto disables Ship (SessionController.setAutoShip / setAuto).
Per-target serialization. MergeQueue processes requests FIFO, strictly one-at-a-time per (projectID, target) key (different repos/branches still run in parallel). Combined with WorktreeManager's per-repo lock this means two sessions never merge into the same branch concurrently. A re-fired .completed can't double-ship: the queue dedupes per session.
Merge against the live tip, never force. Each request runs SessionController.integrate (squash), which checks out the current target tip and merges — so a session branched from an older tip is merged into the post-other-merges branch, never overwriting newer work. A conflict is aborted (squash resets hard to HEAD, since --squash sets no MERGE_HEAD) and returned as .conflicted([paths]) with the branch intact and the target tree clean.
Conflict → stop, don't loop. On .conflicted/.failed the queue's status observer turns the session's Ship flag off and surfaces an error banner; a human (or a fresh agent turn) resolves it. No blind retries.
Visibility. Status (queued/merging/merged/conflicted/failed) is fanned out on MergeQueue.updates(), mirrored into AppStore.shipStatuses and shown as a header pill.

4. Persistence schema (GRDB / SQLite, WAL)

Metadata + indices in SQLite; event bodies stay in transcript JSONL (BACKEND_PROTOCOL §6). SQLite stores what must be queried (lists, status, resume pointers, pairing).

-- Registered repos
CREATE TABLE project (
  id            TEXT PRIMARY KEY,         -- uuid
  name          TEXT NOT NULL,
  root_path     TEXT NOT NULL,
  default_branch TEXT NOT NULL,
  default_backend TEXT,                   -- 'claudeCode' | 'codex'
  worktree_base TEXT,                     -- override; null → default sibling dir
  setup_script  TEXT,
  setup_policy  TEXT NOT NULL DEFAULT 'block',  -- 'block' | 'warn'
  approval_defaults JSON,
  created_at    DATETIME NOT NULL
);

-- One agent run
CREATE TABLE session (
  id            TEXT PRIMARY KEY,         -- OUR SessionID
  project_id    TEXT NOT NULL REFERENCES project(id),
  backend       TEXT NOT NULL,
  backend_session_id TEXT,                -- native id for --resume
  title         TEXT NOT NULL,
  status        TEXT NOT NULL,            -- state machine value
  worktree_path TEXT,
  branch        TEXT,
  base_sha      TEXT,
  model         TEXT,
  last_seq      INTEGER NOT NULL DEFAULT 0,
  transcript_path TEXT NOT NULL,
  native_transcript_path TEXT,
  diff_files    INTEGER, diff_added INTEGER, diff_removed INTEGER,
  ahead         INTEGER, behind INTEGER,
  created_at    DATETIME NOT NULL,
  updated_at    DATETIME NOT NULL
);
CREATE INDEX idx_session_project ON session(project_id, updated_at);

-- Outstanding approvals (resolved ones live in the transcript; this is the live queue)
CREATE TABLE approval (
  id            TEXT PRIMARY KEY,
  session_id    TEXT NOT NULL REFERENCES session(id),
  tool_call_id  TEXT,
  tool_name     TEXT NOT NULL,
  risk          TEXT NOT NULL,
  input         JSON NOT NULL,
  created_at    DATETIME NOT NULL,
  resolved_at   DATETIME,                 -- null while pending
  decision      JSON, decided_by TEXT
);
CREATE INDEX idx_approval_pending ON approval(session_id) WHERE resolved_at IS NULL;

-- "Always allow" rules, per session (BACKEND_PROTOCOL §4)
CREATE TABLE always_rule (
  id TEXT PRIMARY KEY, session_id TEXT NOT NULL REFERENCES session(id),
  scope TEXT NOT NULL, tool_name TEXT, pattern TEXT, created_at DATETIME NOT NULL
);

-- Paired devices for sync (SYNC_PROTOCOL §4)
CREATE TABLE device (
  id            TEXT PRIMARY KEY,         -- device identity fingerprint
  label         TEXT NOT NULL,
  public_key    BLOB NOT NULL,            -- pinned identity key
  scope         TEXT NOT NULL DEFAULT 'approve',  -- 'view'|'approve'|'control'
  push_token    TEXT,
  paired_at     DATETIME NOT NULL,
  last_seen_at  DATETIME
);

Private keys live in Keychain / Secure Enclave, never in SQLite. Transcript files live under Application Support/Nucleic/sessions/<id>/transcript.jsonl.

5. Approval bridging (the tricky bit)

The two backends block a waiter when a tool is gated; the ApprovalCoordinator is what they await:

Claude:  MCP `approve` tool invoked ─┐
Codex:   JSON-RPC requestApproval ───┤
                                     ▼
              ApprovalCoordinator.await(request) -> Decision   (suspends)
                       │ emits AgentEvent.approvalRequested (into the pipeline §2)
                       │ persists row in `approval`
                       ▼
        UI and/or iPhone resolve ──▶ ApprovalCoordinator.resolve(id, decision, by:)
                       │ first responder wins; others get .alreadyResolved
                       ▼
              continuation resumes ─▶ adapter maps Decision → native reply (BACKEND_PROTOCOL §4.1)
                       └ emits approvalResolved into the pipeline (echo, dismiss everywhere)

always_rule is consulted before surfacing: a matching rule auto-resolves without ever emitting approvalRequested to humans (matching the normalized "always allow" semantics even where the CLI re-asks every time).

6. Resume wiring (start-up & on-demand)

On launch, WorktreeManager.reconcile (WORKTREE_MANAGER §8) settles git vs DB; sessions left active become interrupted.
Opening an interrupted/finished session: TranscriptReader streams the JSONL into the UI (rebuilds view state + last_seq) — no CLI involved, the transcript is canonical.
If the user resumes the agent: SessionController calls AgentBackend.resume(ResumeSpec) with the stored backend_session_id; new events continue from last_seq + 1.
If the JSONL is missing/corrupt: degraded import from native_transcript_path.

7. Concurrency rules (Swift 6)

All long-lived mutable state lives in actors; UI state is @MainActor. No shared mutable globals.
Cross-actor payloads (AgentEvent, Decision, ApprovalRequest, summaries) are Sendable value types — already true in the contracts.
One consuming task per backend stream; SessionController serializes ingest. Fan-out to UI is via AsyncStream (or observation of an @Observable snapshot updated on @MainActor).
ProcessHost owns the only references to child-process file handles; stdio is line-buffered to avoid NDJSON deadlocks (BACKEND_PROTOCOL note 7).
GRDB writes go through the DatabasePool writer; reads use snapshots — no manual locking.

8. Open questions

UI observation transport — @Observable snapshot diffing vs. an explicit AsyncStream<SessionStateDelta> per open session. Leaning @Observable for the summary list + a scoped event stream for the open transcript (avoids re-diffing huge transcripts).
fsync policy — per-line fsync (durable, slower) vs. batched + fsync on status transitions. Leaning batched, fsync at turn boundaries and on approval persistence.
Transcript compaction — long sessions' JSONL can grow large; do we snapshot/segment (e.g. one file per N events) for faster paging, and prune partial-delta events after a message finalizes?
Single shared SessionController vs. per-session process supervision — confirm the crash of one child process can never take down sibling sessions (process isolation via ProcessHost says yes; verify in the M2 parallel milestone).

15 KiB Raw Blame History