nucleic/docs/RUNTIME_ARCHITECTURE.md

# Nucleic — Runtime & Persistence Architecture (v0)

Where the contracts meet. [BACKEND_PROTOCOL](BACKEND_PROTOCOL.md) defines the event/approval
model, [SYNC_PROTOCOL](SYNC_PROTOCOL.md) projects it to the phone, [WORKTREE_MANAGER](WORKTREE_MANAGER.md)
owns the git side. This doc defines the **object graph, the single-writer event pipeline, the
persistence schema, and the concurrency model** that wire them together on the Mac host.

**Status:** design draft. Targets Swift 6 strict concurrency.

---

## 1. Object graph

```
                         ┌──────────────── AppStore (@MainActor, @Observable) ───────────────┐
                         │  projects, session list, selection, settings                       │
                         └───────┬───────────────────────────────────────────────┬───────────┘
                                 │ owns N                                          │ owns 1 each
                                 ▼                                                 ▼
                   ┌──────── SessionController (actor) ────────┐         WorktreeManager (actor)
   one per live    │  the hub for ONE session:                 │         ProcessHost (actor)
   session         │   • AgentBackend (adapter + child proc)   │         SyncServer (actor)
                   │   • Worktree handle                       │         NotificationService
                   │   • TranscriptWriter (assigns canon seq)  │         Database (GRDB pool)
                   │   • ApprovalCoordinator                   │
                   │   • derived Session status                │
                   │   • fan-out to subscribers ──────────────┼──▶ UI (MainActor view models)
                   └───────────────────────────────────────────┘  └▶ SyncServer ─▶ iPhone clients
```

The **SessionController is the only writer** of a session's state. Everyone else (UI,
SyncServer, DB) *subscribes*. This single-writer rule is what makes ordering, persistence, and
multi-client sync consistent without locks scattered everywhere.

---

## 2. The event pipeline (single writer, fan-out readers)

```
AgentBackend.start() ──AsyncThrowingStream<AgentEvent>──▶ SessionController.ingest(event)
                                                                │
   (also injected: setup-script logs, approvalResolved echoes, lifecycle markers)
                                                                ▼
                                          1. TranscriptWriter.append → assign CANONICAL seq
                                          2. update derived Session.status (state machine)
                                          3. persist metadata delta to GRDB (status, lastSeq, diffstat)
                                          4. yield to in-process subscribers (UI)
                                          5. SyncServer.broadcast(event, perSubscriberVerbosity)
                                          6. if status ∈ {awaitingApproval, awaitingInput}
                                                 → NotificationService.notify
```

### 2.1 Canonical `seq` (reconciles BACKEND_PROTOCOL §3)

The backend adapter stamps a *provisional* per-adapter `seq` for its own stream. But a session
also emits **synthetic** events the adapter never saw — setup-script output, `approvalResolved`
echoes, `interrupted` markers. So the **canonical, session-wide `seq` is (re)assigned by
`TranscriptWriter` at append time**, the single serialization point. That canonical `seq` is
what the transcript stores, the UI renders, and the sync cursor uses. (Amends BACKEND_PROTOCOL
§3.1: the adapter's `seq` is an ordering hint within its own stream; the authority is the
writer.)

### 2.2 Backpressure

`ingest` is an actor-serialized async call; the backend stream is consumed in a single task.
If a slow consumer (a phone on cellular) can't keep up, that's absorbed in `SyncServer`'s
per-subscriber buffering + verbosity coalescing (SYNC_PROTOCOL §5.5) — never by blocking the
writer. The transcript and UI always get the full firehose.

---

## 3. Key actors

| Actor / type | Isolation | Responsibility |
| --- | --- | --- |
| `AppStore` | `@MainActor`, `@Observable` | Top-level UI state: projects, session summaries, selection, settings. Spawns/owns `SessionController`s. |
| `SessionController` | `actor` (one per session) | Orchestrates a session end-to-end; the only writer of its state; fan-out hub. |
| `TranscriptWriter` | `actor` (one per session) | Append-only JSONL, assigns canonical `seq`, fsync policy. |
| `TranscriptReader` | value/service | Reads JSONL for resume + history paging (by `seq` offset). |
| `ApprovalCoordinator` | `actor` (one per session) | Holds outstanding `ApprovalRequest`s; bridges the backend's blocked approval waiter (Claude MCP call / Codex JSON-RPC request) to UI+sync; enforces first-responder-wins. |
| `WorktreeManager` | `actor` | Git/worktree ops; per-repo structural lock (WORKTREE_MANAGER §1). |
| `MergeQueue` | `actor` | Autoship: serializes finished sessions' merges per target branch; FIFO + per-session dedupe; reports status (§3.1). |
| `ProcessHost` | `actor` | Spawns/monitors child processes, line-buffered stdio, signal delivery, lifecycle. |
| `ContainerManager` | `actor` | Sandbox lifecycle *policy*: per-container busy ref-counts, idle timers, teardown, shared-control-container naming. Drives `ContainerEngine`. |
| `ContainerEngine` | `actor` | In-process container runtime on Apple's `containerization` framework (the mechanism `ContainerManager` drives): one shared VM manager (bundled kernel + vminitd), image store, `vmnet` network, and a registry of live `LinuxContainer`s. Daemonless → VMs are bounded by the app process; `reconcile` is on-disk GC. An agent run is an `exec` whose vsock stdio is adapted to `ProcessHandle` (`ContainerizedProcessHandle`), so backends drive it exactly like a host spawn. |
| `SyncServer` | `actor` | `SecureChannel`s to clients, subscription registry, per-subscriber broadcast/throttle. |
| `NotificationService` | `@MainActor` facade | Local notifications now; APNs later (SYNC_PROTOCOL §6). |
| `Database` | GRDB `DatabasePool` | WAL-mode SQLite; metadata only (transcripts live in JSONL). |

UI never touches an `AgentBackend` or `git` directly — it sends intents to the
`SessionController` (`sendInput`, `respondToApproval`, `requestIntegrate`) and observes state.

### 3.1 Autoship & the merge queue

Autoship (the **Ship** toggle, beside **Auto**) lets a session merge its own branch into
the project's default branch the moment its work is done — no human babysitting the merge.

- **Trigger (model-gated).** When a turn's disposition is classified `.completed`
  (`AppStore.classifyDisposition`, the same signal that frees a finished session's sandbox),
  an autoship session enqueues a `ShipRequest` onto the `MergeQueue`. It never fires mid-turn
  or while the agent is asking a question — only when the model says the work is finished.
- **Implies Auto.** Shipping requires an autonomous run, so enabling Ship enables Auto;
  disabling Auto disables Ship (`SessionController.setAutoShip` / `setAuto`).
- **Per-target serialization.** `MergeQueue` processes requests FIFO, strictly one-at-a-time
  per `(projectID, target)` key (different repos/branches still run in parallel). Combined with
  `WorktreeManager`'s per-repo lock this means two sessions never merge into the same branch
  concurrently. A re-fired `.completed` can't double-ship: the queue dedupes per session.
- **Merge against the live tip, never force.** Each request runs `SessionController.integrate`
  (squash), which checks out the *current* target tip and merges — so a session branched from
  an older tip is merged into the post-other-merges branch, never overwriting newer work. A
  conflict is **aborted** (squash resets hard to `HEAD`, since `--squash` sets no `MERGE_HEAD`)
  and returned as `.conflicted([paths])` with the branch intact and the target tree clean.
- **Conflict → stop, don't loop.** On `.conflicted`/`.failed` the queue's status observer turns
  the session's Ship flag off and surfaces an error banner; a human (or a fresh agent turn)
  resolves it. No blind retries.
- **Visibility.** Status (`queued`/`merging`/`merged`/`conflicted`/`failed`) is fanned out on
  `MergeQueue.updates()`, mirrored into `AppStore.shipStatuses` and shown as a header pill.

---

## 4. Persistence schema (GRDB / SQLite, WAL)

Metadata + indices in SQLite; **event bodies stay in transcript JSONL** (BACKEND_PROTOCOL §6).
SQLite stores what must be *queried* (lists, status, resume pointers, pairing).

```sql
-- Registered repos
CREATE TABLE project (
  id            TEXT PRIMARY KEY,         -- uuid
  name          TEXT NOT NULL,
  root_path     TEXT NOT NULL,
  default_branch TEXT NOT NULL,
  default_backend TEXT,                   -- 'claudeCode' | 'codex'
  worktree_base TEXT,                     -- override; null → default sibling dir
  setup_script  TEXT,
  setup_policy  TEXT NOT NULL DEFAULT 'block',  -- 'block' | 'warn'
  approval_defaults JSON,
  created_at    DATETIME NOT NULL
);

-- One agent run
CREATE TABLE session (
  id            TEXT PRIMARY KEY,         -- OUR SessionID
  project_id    TEXT NOT NULL REFERENCES project(id),
  backend       TEXT NOT NULL,
  backend_session_id TEXT,                -- native id for --resume
  title         TEXT NOT NULL,
  status        TEXT NOT NULL,            -- state machine value
  worktree_path TEXT,
  branch        TEXT,
  base_sha      TEXT,
  model         TEXT,
  last_seq      INTEGER NOT NULL DEFAULT 0,
  transcript_path TEXT NOT NULL,
  native_transcript_path TEXT,
  diff_files    INTEGER, diff_added INTEGER, diff_removed INTEGER,
  ahead         INTEGER, behind INTEGER,
  created_at    DATETIME NOT NULL,
  updated_at    DATETIME NOT NULL
);
CREATE INDEX idx_session_project ON session(project_id, updated_at);

-- Outstanding approvals (resolved ones live in the transcript; this is the live queue)
CREATE TABLE approval (
  id            TEXT PRIMARY KEY,
  session_id    TEXT NOT NULL REFERENCES session(id),
  tool_call_id  TEXT,
  tool_name     TEXT NOT NULL,
  risk          TEXT NOT NULL,
  input         JSON NOT NULL,
  created_at    DATETIME NOT NULL,
  resolved_at   DATETIME,                 -- null while pending
  decision      JSON, decided_by TEXT
);
CREATE INDEX idx_approval_pending ON approval(session_id) WHERE resolved_at IS NULL;

-- "Always allow" rules, per session (BACKEND_PROTOCOL §4)
CREATE TABLE always_rule (
  id TEXT PRIMARY KEY, session_id TEXT NOT NULL REFERENCES session(id),
  scope TEXT NOT NULL, tool_name TEXT, pattern TEXT, created_at DATETIME NOT NULL
);

-- Paired devices for sync (SYNC_PROTOCOL §4)
CREATE TABLE device (
  id            TEXT PRIMARY KEY,         -- device identity fingerprint
  label         TEXT NOT NULL,
  public_key    BLOB NOT NULL,            -- pinned identity key
  scope         TEXT NOT NULL DEFAULT 'approve',  -- 'view'|'approve'|'control'
  push_token    TEXT,
  paired_at     DATETIME NOT NULL,
  last_seen_at  DATETIME
);
```

Private keys live in **Keychain / Secure Enclave**, never in SQLite. Transcript files live
under `Application Support/Nucleic/sessions/<id>/transcript.jsonl`.

---

## 5. Approval bridging (the tricky bit)

The two backends block a waiter when a tool is gated; the `ApprovalCoordinator` is what they
await:

```
Claude:  MCP `approve` tool invoked ─┐
Codex:   JSON-RPC requestApproval ───┤
                                     ▼
              ApprovalCoordinator.await(request) -> Decision   (suspends)
                       │ emits AgentEvent.approvalRequested (into the pipeline §2)
                       │ persists row in `approval`
                       ▼
        UI and/or iPhone resolve ──▶ ApprovalCoordinator.resolve(id, decision, by:)
                       │ first responder wins; others get .alreadyResolved
                       ▼
              continuation resumes ─▶ adapter maps Decision → native reply (BACKEND_PROTOCOL §4.1)
                       └ emits approvalResolved into the pipeline (echo, dismiss everywhere)
```

`always_rule` is consulted *before* surfacing: a matching rule auto-resolves without ever
emitting `approvalRequested` to humans (matching the normalized "always allow" semantics even
where the CLI re-asks every time).

---

## 6. Resume wiring (start-up & on-demand)

1. On launch, `WorktreeManager.reconcile` (WORKTREE_MANAGER §8) settles git vs DB; sessions
   left `active` become `interrupted`.
2. Opening an interrupted/finished session: `TranscriptReader` streams the JSONL into the UI
   (rebuilds view state + `last_seq`) — **no CLI involved**, the transcript is canonical.
3. If the user resumes the agent: `SessionController` calls `AgentBackend.resume(ResumeSpec)`
   with the stored `backend_session_id`; new events continue from `last_seq + 1`.
4. If the JSONL is missing/corrupt: degraded import from `native_transcript_path`.

---

## 7. Concurrency rules (Swift 6)

- All long-lived mutable state lives in `actor`s; UI state is `@MainActor`. No shared mutable
  globals.
- Cross-actor payloads (`AgentEvent`, `Decision`, `ApprovalRequest`, summaries) are `Sendable`
  value types — already true in the contracts.
- One consuming task per backend stream; `SessionController` serializes ingest. Fan-out to UI
  is via `AsyncStream` (or observation of an `@Observable` snapshot updated on `@MainActor`).
- `ProcessHost` owns the only references to child-process file handles; stdio is line-buffered
  to avoid NDJSON deadlocks (BACKEND_PROTOCOL note 7).
- GRDB writes go through the `DatabasePool` writer; reads use snapshots — no manual locking.

---

## 8. Open questions

1. **UI observation transport** — `@Observable` snapshot diffing vs. an explicit
   `AsyncStream<SessionStateDelta>` per open session. Leaning `@Observable` for the summary
   list + a scoped event stream for the open transcript (avoids re-diffing huge transcripts).
2. **fsync policy** — per-line fsync (durable, slower) vs. batched + fsync on status
   transitions. Leaning batched, fsync at turn boundaries and on approval persistence.
3. **Transcript compaction** — long sessions' JSONL can grow large; do we snapshot/segment
   (e.g. one file per N events) for faster paging, and prune partial-delta events after a
   message finalizes?
4. **Single shared `SessionController` vs. per-session process supervision** — confirm the
   crash of one child process can never take down sibling sessions (process isolation via
   `ProcessHost` says yes; verify in the M2 parallel milestone).