nucleic/docs/BACKEND_PROTOCOL.md

Nucleic — Agent Backend Protocol (v0)

The contract every coding-agent CLI is normalized into. Both backends (Claude Code, Codex) and the entire Mac↔iPhone sync layer depend on this, so it is specified tightly here and the per-CLI quirks are pushed down into adapters.

Status: design draft. CLI-specific field shapes below are validated-against-docs but some are version-dependent and flagged ⚠︎ validate. The normalized types (our side) are the stable contract; adapters absorb CLI churn.

---

1. Design goals & invariants

1. One event model, two (or N) CLIs. Adapters translate native output → AgentEvent. Nothing above the adapter knows whether it's talking to Claude or Codex.
2. The local transcript is the source of truth. Every AgentEvent is appended to our own JSONL as it is produced. The UI renders from that, never from re-reading the CLI's native transcript. Native session files are used only to drive resume and as a recovery import.
3. Capabilities, not assumptions. Backends differ (interactive approvals, input rewrite, partial streaming, sandbox). Each backend declares a BackendCapabilities; the UI adapts. We never assume a feature is present.
4. Total per-session ordering. The adapter stamps every event with a monotonic seq. Consumers (UI, sync, persistence) rely on seq, not wall-clock time.
5. Unknown-field tolerant. Adapters decode leniently and pass anything unrecognized through as .raw rather than crashing. CLI JSON is explicitly experimental (esp. Codex).

---

2. The protocol surface (Swift)

protocol AgentBackend: Sendable {
    static var id: BackendID { get }                 // .claudeCode | .codex
    var capabilities: BackendCapabilities { get }

    /// Begin a fresh run in a prepared worktree. Returns the live event stream.
    func start(_ run: RunSpec) -> AsyncThrowingStream<AgentEvent, Error>

    /// Reattach to an existing backend session (native resume).
    func resume(_ resume: ResumeSpec) -> AsyncThrowingStream<AgentEvent, Error>

    /// Queue a follow-up user turn (and/or injected context) on a running session.
    func send(_ input: AgentInput) async throws

    /// Answer an outstanding approval. No-op throws if backend lacks interactive approvals.
    func respond(to approvalID: ApprovalID, _ decision: Decision) async throws

    /// Cooperative interrupt of the current turn (keeps session resumable).
    func interrupt() async

    /// Terminate the process/connection and release resources.
    func shutdown() async
}

2.1 Capabilities

struct BackendCapabilities: Sendable {
    var interactiveApprovals: Bool         // can answer an approval mid-run?
    var allowAlwaysScopes: Set<AlwaysScope> // which "always allow" scopes are real
    var canModifyToolInput: Bool           // can we rewrite args on allow?
    var partialMessageStreaming: Bool      // token-level deltas available?
    var emitsThinking: Bool                // reasoning/thinking surfaced?
    var emitsFileChangeEvents: Bool        // native file_change events (vs. inferred)
    var nativeResume: Bool                 // --resume / resume <id>
    var sandboxModes: [SandboxMode]        // [] if not a first-class concept
    var followUpWhileRunning: Bool         // can `send` mid-turn, or only between turns
}

Capability	Claude Code (stream-json + MCP)	Codex (app-server)	Codex (exec --json)
interactiveApprovals	✅	✅	❌ (fail-closed)
canModifyToolInput	✅ (updatedInput)	❌ (policy amendments only, not command rewrite)	❌
partialMessageStreaming	✅ (--include-partial-messages)	✅ (item/agentMessage/delta, …/outputDelta)	🟡 item updates
followUpWhileRunning	🟡	✅ (turn/steer)	❌
emitsThinking	✅ (thinking blocks)	✅ (reasoning item)	✅ (reasoning item)
emitsFileChangeEvents	❌ (infer from Edit/Write)	✅ (file_change)	✅ (file_change)
nativeResume	✅	✅	✅
sandboxModes	[] (permission model)	read-only/workspace-write/full	same

Decision: the default Codex adapter is codex app-server (JSON-RPC) so interactive approvals work. exec --json is a secondary "unattended" adapter (CodexExecBackend) used only when the user opts into a fixed policy; its capability set advertises interactiveApprovals = false and the UI warns at session start.

2.2 Run / resume / input specs

struct RunSpec: Sendable {
    let sessionID: SessionID            // OUR id (uuid), not the backend's
    let worktree: WorktreePath
    let prompt: AgentInput
    let model: String?                  // nil → backend default
    let approvalPolicy: ApprovalPolicy  // interactive | fixed(rules)
    let sandbox: SandboxMode?           // Codex only; ignored by Claude
    let mcpConfigPath: URL?
    let appendSystemPrompt: String?
    let extraEnv: [String: String]
    let extraArgs: [String]             // escape hatch, adapter-validated
}

struct ResumeSpec: Sendable {
    let sessionID: SessionID
    let backendSessionID: String        // native id captured at start
    let worktree: WorktreePath
    let fork: Bool                      // start a new native id, keep our transcript
}

struct AgentInput: Sendable {
    enum Part { case text(String); case context(label: String, body: String) }
    let parts: [Part]
}

---

3. AgentEvent — the normalized event model

Every event carries an envelope; the kind holds the payload.

struct AgentEvent: Sendable, Codable {
    let sessionID: SessionID
    let seq: UInt64                 // monotonic per session, assigned by adapter
    let at: Date                    // adapter receive time
    let backend: BackendID
    let nativeType: String?         // original CLI type, for debugging/forensics
    let kind: Kind
}

extension AgentEvent {
    enum Kind: Sendable, Codable {
        case sessionStarted(SessionStarted)
        case assistantText(TextChunk)      // partial or complete assistant prose
        case thinking(TextChunk)           // reasoning / thinking
        case toolCallStarted(ToolCall)     // id + name known; input may be empty/partial
        case toolCallInputDelta(ToolInputDelta) // only when partialMessageStreaming
        case toolCallCompleted(ToolCall)   // full input finalized
        case toolResult(ToolResult)        // output of a tool call
        case fileChange(FileChange)         // add/update/delete a path
        case approvalRequested(ApprovalRequest)
        case approvalResolved(ApprovalResolved) // echoed for the transcript
        case usage(Usage)
        case turnCompleted(TurnCompleted)
        case runFinished(RunFinished)
        case error(AgentError)
        case raw(Raw)                       // unrecognized native event, passthrough
    }
}

3.1 Payloads

struct SessionStarted: Codable {
    let backendSessionID: String       // ← capture for resume
    let model: String
    let cwd: String
    let toolNames: [String]
    let nativeTranscriptPath: String?  // ~/.claude/projects/… or ~/.codex/sessions/…
}

struct TextChunk: Codable {
    let messageID: String              // groups deltas of one assistant message
    let text: String                   // delta if isPartial, else full text
    let isPartial: Bool
}

struct ToolCall: Codable {
    let toolCallID: String             // native tool id (toolu_… / item id)
    let name: String                   // "Bash", "Edit", "command_execution", …
    let input: JSONValue               // best-known input; complete on …Completed
    let parentToolCallID: String?      // sub-agent / nested
}

struct ToolInputDelta: Codable { let toolCallID: String; let partialJSON: String }

struct ToolResult: Codable {
    let toolCallID: String
    let content: JSONValue             // text or structured
    let isError: Bool
}

struct FileChange: Codable {
    enum Kind: String, Codable { case add, update, delete, rename }
    let path: String
    let kind: Kind
    let toolCallID: String?            // the call that produced it, if known
    let diffHint: String?              // unified diff if the CLI provides one
}

struct Usage: Codable {
    let inputTokens: Int?
    let cachedInputTokens: Int?
    let outputTokens: Int?
    let reasoningTokens: Int?
    let costUSD: Double?               // nil if backend doesn't report it
}

struct TurnCompleted: Codable { let stopReason: String?; let usage: Usage? }

struct RunFinished: Codable {
    enum Outcome: String, Codable { case completed, interrupted, errored, maxTurns }
    let outcome: Outcome
    let finalText: String?
    let totalUsage: Usage?
    let durationMs: Int?
}

struct AgentError: Codable {
    let recoverable: Bool              // e.g. api_retry vs fatal
    let message: String
    let native: JSONValue?
}

struct Raw: Codable { let native: JSONValue }

JSONValue is our standard recursive JSON enum (Codable, Sendable, Equatable).

3.2 Tool-call lifecycle (normalized)

toolCallStarted ──▶ [toolCallInputDelta]* ──▶ toolCallCompleted
                                                    │
                          (if gated) approvalRequested ──▶ approvalResolved
                                                    │
                                               toolResult
                                        (and/or fileChange × N)

• Adapters that don't stream partial input simply emit toolCallStarted with full input and immediately toolCallCompleted (no deltas).
• fileChange may arrive without a preceding tool call on backends that emit it natively (Codex). For Claude we synthesize fileChange from Edit/Write/MultiEdit tool calls (emitsFileChangeEvents = false signals the UI it's inferred, not authoritative).

---

4. Approvals — the normalized round-trip

This is the highest-divergence area; the model is designed around the asymmetry in §1.

struct ApprovalRequest: Codable, Identifiable {
    let id: ApprovalID                 // OUR uuid
    let sessionID: SessionID
    let toolCallID: String?            // links to the gated tool call
    let toolName: String
    let input: JSONValue
    let title: String                  // human summary ("Run: rm -rf build/")
    let risk: Risk                     // heuristic classification (below)
    let suggested: Decision?           // backend hint, if any
    let createdAt: Date
}

enum Risk: String, Codable { case readOnly, write, execute, network, destructive, unknown }

enum AlwaysScope: String, Codable {
    case session                       // this tool, this session
    case toolName                      // any call to this tool, this session
    case toolNameWithPattern           // tool + argument pattern (e.g. Bash(git *))
}

enum Decision: Sendable, Codable {
    case allow(updatedInput: JSONValue? = nil)
    case allowAlways(AlwaysScope)
    case deny(reason: String?)
    case cancelRun                      // deny + stop the whole run
}

struct ApprovalResolved: Codable {
    let id: ApprovalID
    let decision: Decision
    let decidedBy: String              // "mac-ui" | "iphone:<device>" — first responder wins
    let decidedAt: Date
}

Risk classification is computed by the adapter from toolName + input (e.g. Bash with rm/git push/curl → destructive/network; Read/Glob → readOnly). It drives UI emphasis and any auto-policy; it never decides on its own in interactive mode.

allowAlways semantics depend on backend support:

• Where the backend has a native "for session" answer (Codex acceptForSession), use it.
• Where it doesn't (Claude calls the permission tool every time), the adapter caches the rule locally and auto-answers subsequent matching requests without surfacing them. Either way, the UI behavior is identical — that's the point of normalizing here.

4.1 Approval mapping per backend

Both columns are source-confirmed (ADAPTERS §1.2 / §2.4). Claude's reply is the JSON object stringified inside an MCP text content block; Codex's is a JSON-RPC result.

Normalized Decision	Claude (--permission-prompt-tool reply, JSON-stringified)	Codex app-server (requestApproval result)
.allow()	{"behavior":"allow","updatedInput":<orig>}	{"decision":"accept"}
.allow(updatedInput:)	{"behavior":"allow","updatedInput":<new>}	n/a — canModifyToolInput=false; UI hides it
.allowAlways(.session)	adapter-cached auto-allow + behavior:allow	{"decision":"acceptForSession"}
.deny(reason:)	{"behavior":"deny","message":<reason>}	{"decision":"decline"}
.cancelRun	behavior:deny + interrupt()	{"decision":"cancel"}

✅ Confirmed: Claude keys are behavior/message/updatedInput (not decision/reason), with updatedInput echoed even when unchanged; the input the tool receives is {tool_name, input, tool_use_id?}. Codex's v2 decision enum is accept/acceptForSession/decline/cancel. The deprecated v1 path (execCommandApproval/applyPatchApproval, approved/denied/abort) is not used. item/permissions/requestApproval replies with a granted-subset + scope, not a decision (ADAPTERS §2.4).

4.2 CodexExecBackend (no interactive approvals)

When a session runs under codex exec, respond throws Unsupported. Instead the run is created with a fixed ApprovalPolicy.fixed(rules) mapped to --sandbox + --ask-for-approval. A would-be approval becomes a fail-closed toolResult(isError) plus an .error(recoverable) event so the UI can show "blocked by policy (non-interactive)". Surface this clearly at session creation so users don't expect to be prompted.

---

5. Lifecycle / status state machine

Session.status is derived from the event stream:

            start()                 approvalRequested
  idle ───────────────▶ running ───────────────────▶ awaitingApproval
                          ▲ │                              │ approvalResolved
        approvalResolved  │ │ turnCompleted (no follow-up) │
                          │ ▼                              ▼
                       awaitingInput ◀───────────────── running
                          │
          runFinished     │  error(recoverable:false)
  running ──────────▶ finished        running ───────────▶ error

• awaitingApproval and awaitingInput are the two "needs a human" states that trigger notifications (Mac + iPhone).
• error(recoverable:true) (e.g. Claude api_retry, Codex -32001 overload) does not leave running; the adapter handles backoff and emits an info-level .error.

---

6. Persistence — our transcript is canonical (locked decision)

Each session writes ~/Library/Application Support/Nucleic/sessions/<sessionID>/transcript.jsonl:

• Line 0: SessionHeader — our sessionID, backend, backendSessionID, worktree, model, nativeTranscriptPath, createdAt, schema version.
• Lines 1…N: one AgentEvent per line, in seq order.

Metadata (status, counts, last activity, ahead/behind) lives in GRDB and points at this file.

Resume flow: (1) read our JSONL → rebuild UI state and the last known seq; (2) call backend.resume(ResumeSpec) with the stored backendSessionID; (3) continue appending new events from seq+1. We never depend on the CLI to re-emit history. If our file is lost we can import from nativeTranscriptPath as a degraded fallback.

---

7. Backend adapter mapping reference

7.1 Claude Code adapter

• Spawn: claude -p --output-format stream-json --input-format stream-json --verbose [--include-partial-messages] --permission-prompt-tool mcp__nucleic__approve --mcp-config <generated> [--model] [--append-system-prompt] [--resume <id>] [--add-dir] with cwd = worktree.

• Approvals: Nucleic hosts an in-process MCP server exposing approve. Claude calls it per gated tool with {tool_name, input, description} ⚠︎; the call blocks until the UI/phone resolves, then returns the mapped reply (§4.1).

• Event mapping:

  Native stdout (type)	→ AgentEvent.kind
  system/init	sessionStarted (capture session_id)
  assistant message w/ text block	assistantText(isPartial:false)
  assistant message w/ thinking block	thinking
  assistant message w/ tool_use block	toolCallStarted + toolCallCompleted
  user message w/ tool_result	toolResult
  stream_event (partial)	assistantText/thinking/…(isPartial:true) / toolCallInputDelta
  system/api_retry	error(recoverable:true)
  result	runFinished + usage

  fileChange synthesized from Edit/Write/MultiEdit/NotebookEdit tool calls. ⚠︎ Without --include-partial-messages the canonical stream emits full assistant/user message events (not stream_event); the adapter must handle both shapes.

7.2 Codex adapter (app-server, default)

• Spawn: codex app-server (stdio JSON-RPC). One-time initialize, then per-run thread/turn RPCs; --cd/cwd = worktree, model + sandbox via params/config.
• Approvals: server→client requests item/commandExecution/requestApproval, item/fileChange/requestApproval, item/permissions/requestApproval → approvalRequested; reply per §4.1. mcpServer/elicitation/request also bridged.
• Event mapping (thread/turn/item taxonomy):

  Native	→ AgentEvent.kind
  thread.started	sessionStarted (thread_id = backendSessionID)
  turn.started	(status only)
  item agent_message	assistantText
  item reasoning	thinking
  item command_execution started/updated/completed	toolCallStarted→toolResult (+ exit code)
  item file_change	fileChange (per changes[])
  item mcp_tool_call	toolCallStarted/toolCallCompleted/toolResult
  item web_search	toolCall* (name=web_search)
  turn.completed	turnCompleted + usage
  turn.failed / error	error

7.3 Codex adapter (exec, unattended fallback)

• Spawn: codex exec --json [-o last.txt] [--sandbox …] [--ask-for-approval never|untrusted] [-m] [-C <wd>]; resume via codex exec resume <id>.
• Same item taxonomy as app-server for events; no approval channel (§4.2).

---

8. Versioning & validation

• Protocol carries a schemaVersion; transcript header records it.
• Adapters pin a tested CLI version range and tolerate unknown fields (decode → .raw).
• Every ⚠︎ validate item must be confirmed against a captured real stream from the pinned CLI version before that adapter ships. Recommended: a fixtures/ set of recorded native streams + golden normalized AgentEvent output, used in adapter unit tests.

9. Open questions

1. Claude permission-tool reply keys — confirm behavior/message/updatedInput vs decision/reason against the shipping claude; adjust adapter only.
2. Codex input rewrite — can app-server amendments express arbitrary command edits, or do we set canModifyToolInput=false for Codex and hide "edit & allow" in the UI?
3. Follow-up mid-turn — does each backend accept a queued user turn while running, or only at awaitingInput? Sets followUpWhileRunning.
4. Partial streaming cost — default --include-partial-messages on (snappier UI, more events to persist/sync) vs off (coarser, cheaper). Likely on for local, throttled for iPhone sync.