nucleic/nucleic
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b900f713a1543b6587b043725c737d8ba288bb0b
nucleic/docs/CARBON_SHARDING.md
T

65 KiB
Raw Blame History

Nucleic — Carbon Sharding (logical carbon copies across the Covalence mesh)

Status (2026-07-13): implementation-grade spec; nothing built. Every Nucleic Control project's repo data — git history the origin doesn't have, plus each live session's uncommitted working-tree and index state — is decomposed into small, content-addressed, encrypted, immutable shards and replicated to a deterministic replica set across the Covalence mesh. The result is a logical carbon copy: no member holds a checked-out clone of another's work, but any key-holding member can materialize one from shards. If a device dies or loses its disk, the data stays fully recoverable from replicas until the device reconnects (it back-fills) or is replaced (the replacement back-fills).

This doc supersedes AGENT_RESUSCITATION.md §7 (the deferred "mesh worktree-checkpoint") and is the local-first analogue of CLOUD_RUNTIME.md §3.5's R2 checkpoint. Resuscitation consumes this layer (its §6.3 step 4 and §6.2 election preference); this layer never bypasses the resuscitation ownership fence. Design decisions below were resolved with the project owner on 2026-07-13 (§0); code seams were verified by a 3-lane reconnaissance and are cited inline.

Amended 2026-07-21 (owner sign-off): (1) Carbon and Covalence are mandatory — the master toggle and per-project opt-out are removed (D17); the enrollment repo-shape gate remains an honest inability, never a preference. (2) Carbon subsumes the locking system — LOCKING.md's LockManager moves into Carbon and becomes mesh-native (§17), and every lock release is fenced on Carbon propagation: no device observes a lock as released until it holds the updated project files via Carbon (§17.4).

0. Locked decisions (do not re-litigate; change only with owner sign-off)

# Decision Choice Spec section
D1 Wave-1 streams All three: history + worktreeSnapshot + nativeTranscript. (history is mandatory for correctness — snapshot packs are thin against the branch tip, which usually exists nowhere but the owner.) §3
D2 Replication model R-replica placement, not full replication §5
D3 R and degradation R = 2 replicas beyond the owner; fewer eligible members ⇒ replicate to what exists and warn ("reduced redundancy" / "not protected"); never block the session §5.3
D4 Placement algorithm Hybrid: rendezvous hashing for the base set plus pinned always-on members (runners default-pinned; user can pin Macs) always included §5.1–5.2
D5 Relay policy Steady-state deltas allowed over the relay by default, ≤ 4 MiB per manifest payload; bulk backfill LAN/tailnet-only; per-project opt-out §8.4
D6 At-rest crypto Encrypted from day one: per-project key, deterministic (SIV-equivalent) AEAD, shard ID = SHA-256 of ciphertext, dedup preserved, mirrors verify without decrypting §4
D7 Key escrow iCloud Keychain (synchronizable item) escrow of each project key at enrollment; Linux runners receive keys via the credential mesh, never iCloud §4.4
D8 Snapshot cadence 20 s while dirty + exponential backoff to 2 min on identical trees + always at turn boundaries §6.4
D9 localOnly projects Enroll and shard the full graph (first history manifest = entire object graph), bounded by the per-project disk cap; oversize ⇒ "not protected" warning, not silent skip §3.2
D10 Index fidelity Two trees per snapshot: worktree tree and index tree, so restore reproduces staged-vs-unstaged exactly §6.2–6.3, §9.1
D11 GC authority Mirror-autonomous (pure local rules, no delete verbs on the wire), retention K = 8 snapshot manifests per session §7.3
D12 Sequencing Parallel tracks with the resuscitation plan: Carbon phases 1–3 have no fence dependency; integration happens at Carbon phase 4 §15
D13 Platforms Mac + Linux runner from day one; iOS is a viewer, every new wire case inert on phones §8.6, §16
D14 R2 / off-mesh sink Design the ShardSink interface, do not build the R2 implementation in this effort §11
D15 Wave-1 UI Full surface: durability chip, Settings pane, and the per-session snapshot/restore browser §12
D16 Origin-trust mode Default: negatives may assume origin holds previously-fetched SHAs; per-project paranoid mode shards everything and trusts origin for nothing §6.6, §13
D17 Mandatory status (2026-07-21) Carbon and Covalence are mandatory: no master toggle, no per-project opt-out (repo-shape / disk-cap remain honest inabilities); the Covalence mesh stack always runs §12, §17.6
D18 Locking (2026-07-21) The LOCKING.md system moves into Carbon: per-domain single-writer lock ledger stream arbitrated by a fenced lock authority; LOCKING's acquisition/queue/grant semantics stay normative §17
D19 Release fence (2026-07-21) A lock release names the Carbon heads carrying the landed files (autoship/worktree: base-branch history head; nvrsion: trunk history head); no device observes "released" until it holds them §3.2, §17.4

1. Thesis and substrate (read once, then build)

Carbon's job changes from moving a copy on request (SessionCarbonCopy, SessionCarbonCopy.swift:11-17) to maintaining copies continuously. Three facts make it cheap:

  1. Git is already a content-addressed store. A snapshot taken with a throwaway index (git write-tree) yields only changed objects; unchanged files dedup to the same blob SHA and never travel twice. We shard the incremental pack, never the worktree — that is what satisfies "no constant git ops, no syncing of entire worktrees."
  2. The origin remote is the durable base for cloned projects. Carbon ships only the delta over origin (kilobytes–megabytes), except for localOnly projects (D9) where it ships everything.
  3. The replication pattern exists. The transcript mirror (TranscriptMirrorStore.swift, TranscriptSyncCoordinator.swift:62-79) is a pull-based, holder-probing replicator. Carbon reuses that shape with content addressing (reconciliation = set difference) and placement (D2).

Substrate facts an implementor must know before writing code:

  • Chunking/flow-control precedent: TransferChunk.maxDataBytes = 2 MiB under the 16 MiB frame cap, stop-and-wait window 4, per-item resume cursors (TransferMessages.swift:327-363, WireFraming.swift:8-10, SessionTransferCoordinator.swift:31-33).
  • Wire extensibility rules: independent per-subsystem format versions (TranscriptSyncMessages.swift:16-20); capability bits decode-default-false (WireMessages.swift:130-295); ClientMsg throws on unknown tags (MessageEnvelope.swift:435-438) so every new client verb must be gated on the receiver's advertised capability; every new HostMsg needs a SyncClient.Event case + branches in all four exhaustive switches (two are iOS-only, compiled solely under xcodebuild).
  • Git seams: WIP-capture recipe finalize(.auto) (WorktreeManager.swift:449-476); commits land only at ship/integrate (SessionController.swift:1667), transfer quiesce, reclaim (WorktreeManager.swift:913); only createPR pushes (:673-703); bundle restore recipe createFromTransfer (:834-865); repo-shape gate (shallow/partial/LFS block, mirrored here) (TransferGit.swift:38-67); project identity triple UUID / root commit / normalized remote (TransferGit.swift:72-106).
  • Topology: full mesh of paired Macs/runners (PeerClient.swift:60-68,577-599), relay as E2EE fallback; realistic mesh size 2–6 members.
  • Nothing content-addressed exists today; transcript fetch has no flow control (fire-and-stream, ConnectionHandler.swift:646-658) — shard fetch must NOT copy that; it copies the transfer ack window instead.

2. Vocabulary (used precisely throughout)

  • Member — a paired Mac or runner on the mesh. Phones are never members for Carbon.
  • Eligible member — a member advertising canMirrorCarbon that holds (or can receive) the project key.
  • Owner — the single host currently running a session (single-writer, MESH_TRANSFER.md invariant #1). Only owners produce.
  • Replica set — the deterministic set of eligible members assigned to hold a project's shards (§5). The owner holds its own data implicitly and is not counted in R.
  • Shard — immutable encrypted blob ≤ 2 MiB, ID = SHA-256 hex of its ciphertext.
  • Manifest — immutable encrypted CBOR document describing one snapshot event; ID = SHA-256 hex of its ciphertext.
  • Head — the newest manifest of a stream: (manifestID, ownerGeneration, snapshotSeq).
  • Mesh-durable — a head is mesh-durable when ≥ min(R, |replica set|) replica-set members have gossiped holding it (§5.3).
  • Lock ledger — the per-domain lock stream (§17.2): the append-only, single-writer record of lock events; a member's current lock table is a pure fold of it.
  • Lock authority — the single host arbitrating a lock domain (§17.3), fenced by authorityGeneration under the §10 head-advance discipline.
  • Release fence — the rule that a released ledger event is observed by a device only once that device holds the event's requiredHeads — the landed project files (§17.4).

3. Streams — what gets sharded per project

Every stream is a chain of manifests. Three stream kinds (D1, all wave 1):

3.1 worktreeSnapshot (per session)

The uncommitted state of a live session worktree: two trees (D10) — the working files as they sit on disk, and the git index (staged state) — captured by §6, shipped as a thin incremental pack. This is the headline stream; RPO = one cadence interval.

3.2 history (per session, plus one per-project base chain)

Git objects the origin does not have, without which snapshot packs cannot be applied:

  • Per session: commits on nucleic/<slug> from baseSHA to the branch tip, produced every time the tip moves (§6.6). Required for restore whenever the branch tip was never pushed — i.e. almost always.
  • Per project (sessionID = nil) — the project-base chain. Two shapes:
    • localOnly projects (no origin remote, created by the from-scratch createProject path, AppStore.swift:1744-1820): the first manifest packs the entire reachable graph (D9); subsequent manifests pack deltas as local base branches move.
    • Cloned projects (added 2026-07-21 for D19): deltas of the local base branches (and the nvrsion trunk nucleic/trunk, when nvrsion is active) ahead of origin — typically just the merge/land commits produced at ship/integrate/promote and at each nvrsion land (§6.6). Previously this chain did not exist for cloned projects (origin was the base); the lock-release fence (§17.4) requires the landed base/trunk content to be replicable across the mesh without waiting for an origin push, so it now does.

3.3 nativeTranscript (per session)

The agent CLI's own resume file (<backendSessionID>.jsonl, today only shipped inside a live transfer, TransferChannel.swift:116-132). Append-only: each manifest's payload is the byte range since the previous manifest (payloadKind = .jsonlSegment). Restoring the full concatenation + the backendSessionID gives true claude --resume after resuscitation.

3.4 lock (per lock domain) (added 2026-07-21, D18 — not wave-1; ships with §15 phase 7)

The append-only lock ledger of §17.2. Not a file stream — it carries no project content; its payloads are small CBOR lock events (payloadKind = lockEvent, usually one shard) and its released events reference content heads (§17.4). Ledger manifests reuse the §6.3 schema with stream = "lock", sessionID = nil, branch = the domain ref, and ownerGeneration carrying the authorityGeneration (§17.3) — the §10 head-advance rule applies verbatim. Single writer: the domain's lock authority.

Not sharded: the Nucleic transcript (already mirrored byte-faithfully by the existing mirror — do not duplicate), the GRDB database (gossip + transfer records already carry its durable content), credentials (never replicated — MESH_TRANSFER invariant), and .gitignored files (rebuildable, unbounded). Enrollment gate: TransferGitProbe.repoShape — shallow, partial/promisor, or LFS repos do not enroll; the UI must show "not mesh-protected: ", never silently skip (§13).

4. Cryptography (D6, D7)

4.1 Keys

  • Per project, at enrollment, the enrolling host generates a random 32-byte project master key K_p.
  • Derived subkeys via HKDF-SHA256 (swift-crypto, works on Darwin + Linux):
    • K_enc = HKDF(K_p, info: "nucleic.carbon.enc.v1", 32 bytes)
    • K_siv = HKDF(K_p, info: "nucleic.carbon.siv.v1", 32 bytes)
  • K_p never leaves key custody paths; only subkeys are used by the store.

4.2 Deterministic AEAD (SIV-equivalent, buildable with swift-crypto)

swift-crypto has no AES-SIV, so use the standard derived-nonce construction, which gives the same guarantee (same plaintext ⇒ same ciphertext; nonce reuse impossible across distinct plaintexts):

nonce      = HMAC-SHA256(K_siv, plaintext)[0..<12]
sealed     = AES-256-GCM.seal(plaintext, key: K_enc, nonce: nonce)
blob       = nonce ‖ sealed.ciphertext ‖ sealed.tag
shardID    = SHA256(blob).hexString        // lowercase, 64 chars

Decrypt: split blob, open with K_enc/nonce, GCM tag authenticates. Properties the design relies on:

  • Determinism ⇒ identical plaintext shards dedup to one stored blob and one wire transfer, and shardID is derivable by re-encryption (the producer never needs a plaintext→ID map).
  • Verify-without-decrypt ⇒ any holder (including a future untrusted ShardSink) validates a shard by re-hashing blob and comparing to shardID. Required on write and on every read before serving (§7.2).
  • Accepted leak: equality of identical plaintext within one project is visible to non-key-holders. All replica members hold the key anyway; documented in §13.

Manifests are encrypted with the exact same construction (manifestID = SHA256(ciphertext)). Replica members decrypt manifests in memory to index/GC; a dumb sink never needs to.

4.3 Distribution

The project key rides the existing credential mesh (COVALENCE_RUNNER §6 — the machinery that already places agent credentials on eligible hosts), as a new credential kind carbonProjectKey(projectUUID). Delivered to every eligible member — note: to be counted in a replica set a member must hold the key, so key delivery is a placement precondition (§5.1). A member missing the key answers shard requests normally (blobs are opaque) but reports keyMissing in its head gossip so the owner can surface it.

4.4 Escrow (D7)

At enrollment the Mac writes K_p to the user's iCloud Keychain as a synchronizable item (kSecAttrSynchronizable = true, service "nucleic.carbon.projectKey", account = projectUUID). Recovery path for total-device-loss: new Mac signs into iCloud → pairs → reads escrowed keys → decrypts shards back-filled from any surviving holder or future sink. Linux runners never touch iCloud (they get keys via §4.3). If Keychain write fails, enrollment still proceeds but the durability chip shows "key not escrowed" (§12).

5. Placement (D2, D3, D4)

5.1 Inputs, computed identically by every member with no coordination

eligible(project) = members m where:
    m.capabilities.canMirrorCarbon           // Macs + runners only; never phones
  ∧ m holds (or is queued to receive) carbonProjectKey(project)
  ∧ m is not the project's current owner-of-record for this computation
pinned(project)   = { m ∈ eligible : m.alwaysOn }   // runners default alwaysOn=true;
                                                    // user may pin Macs in Settings

Membership comes from the existing mesh-roster gossip (PeerClient roster, PeerClient.swift:840-843,908-912). alwaysOn is a new roster flag, additive/decode-default false.

5.2 The replica set (hybrid rendezvous, D4)

score(m)     = SHA256(projectUUID_bytes ‖ m.hostID_bytes)  // interpret as big-endian UInt256
base         = top-R of (eligible \ pinned) by descending score, R = 2
replicaSet   = pinned ∪ base

Deterministic: every member computes the same set from the same roster; no placement map is gossiped or persisted (cache it in placement.json, §7.1, but it is always recomputable). Membership change ⇒ recompute:

  • New winner ⇒ it back-fills (§8.5).
  • Ex-member ⇒ keeps its shards for a 14-day grace (free extra redundancy), then its normal GC (§7.3) reclaims them. There is no "evict now" message (D11).

5.3 Durability accounting and degradation (D3)

The owner tracks, per stream head, which replica-set members' gossiped heads include it. A head is mesh-durable when count ≥ min(2, |replicaSet|). Surfacing (never blocking):

Condition Chip state
head durable on ≥2 replicas protected ×N (N = holders incl. owner)
exactly 1 replica holds it reduced redundancy (amber)
replicaSet empty (mesh of one, or no key-holders) not protected (red) + first-run nudge "pair a second device or add a runner"
enrolled but repo-shape-blocked / over disk cap / key not escrowed not protected: <reason>

6. Producer — CarbonSnapshotter (owner-side only)

An actor owned by AppStore, one instance per host, driving all enrolled projects. Mirrors NEVER produce. All git work for a session must serialize with WorktreeManager structural ops (run through the same actor) and must skip any session present in transferringSessions (AppStore.swift:1181) or mid-integrate.

6.1 Per-tick pipeline for one dirty session worktree

Every command runs with cwd = worktree path; <wt> below. Fixed committer identity Nucleic Carbon <[email protected]> via GIT_AUTHOR_*/GIT_COMMITTER_* env (do NOT use the user's identity or the Managed-Git signing path — snapshot commits are plumbing, never shown, never pushed).

  1. Dirty check (cheap, always first): git status --porcelain --no-renames; empty output ⇒ record no-op, apply backoff (§6.4), stop. Enroll-time: set core.untrackedCache=true on the repo.
  2. Index tree (D10): indexTree = git write-tree against the session's real index.
    • If it fails with unmerged entries (exit ≠ 0, stderr mentions "unmerged"): set indexTreeSHA = nil, set manifest flag indexUnavailable = true, continue (working files still captured; restore then leaves everything unstaged).
    • write-tree does not mutate tracked content (it may refresh the cache-tree extension — harmless).
  3. Worktree tree (throwaway index — never touch the session's real index): 
    export GIT_INDEX_FILE=$(mktemp -u)           # unique path, not created yet
git read-tree <branchTipSHA>
git add -A                                    # tracked mods + untracked; .gitignore respected
worktreeTree=$(git write-tree)
rm -f $GIT_INDEX_FILE; unset GIT_INDEX_FILE
    branchTipSHA = git rev-parse refs/heads/<branch> read at step start; if the tip moves mid-snapshot (a ship raced us), abort this tick — the tip-move trigger (§6.6) will re-enter.
  4. Skip identical: if worktreeTree == previous.worktreeTreeSHA AND indexTree == previous.indexTreeSHA ⇒ no-op (backoff), stop.
  5. Hidden anchor commits (keep objects reachable so repo git gc never collects them; give pack negatives stable anchors; invisible to porcelain): 
    wtCommit  = git commit-tree <worktreeTree> -p <prevWtCommit> -p <branchTipSHA> \
              -m "carbon wt snapshot seq=<seq>"
idxCommit = git commit-tree <indexTree>    -p <prevIdxCommit> \
              -m "carbon idx snapshot seq=<seq>"          # only when indexTree ≠ nil
git update-ref refs/nucleic/carbon/<sessionID>/wt  <wtCommit>
git update-ref refs/nucleic/carbon/<sessionID>/idx <idxCommit>
    First snapshot of a session: omit the prev* parents.
  6. Thin incremental pack (exactly the new objects): 
    git pack-objects --revs --thin --delta-base-offset --stdout <<EOF
<wtCommit>
<idxCommit>                # if present
^<prevWtCommit>            # if present
^<prevIdxCommit>           # if present
^<branchTipSHA>
EOF
    The ^branchTipSHA negative is safe because the history stream (§6.6) guarantees the tip's objects are independently sharded — this coupling is the reason D1 requires both streams. Typical agent-cadence packs: KBs to a few hundred KB.
  7. Shard + manifest + notify: split the pack into ≤ 2 MiB plaintext slices → encrypt each (§4.2) → CarbonShardStore.put each blob → build + encrypt + put the manifest (§6.3) → advance the local head → send carbonHeadAdvanced to the replica set (§8.2).

6.2 What a snapshot captures / does not capture

Captured: tracked modifications, deletions, renames, untracked-unignored files, file modes, symlinks (as git stores them), the staged/unstaged split (via the two trees). Not captured: .gitignored files, .git internals beyond the trees, filesystem xattrs/ACLs, in-flight editor buffers. State this in the Settings pane footnote verbatim.

6.3 Manifest schema (canonical CBOR v1, then encrypted per §4.2)

Canonical encoding = RFC 8949 §4.2 core deterministic encoding (sorted map keys, definite lengths). Field order/names are frozen by tests.

CarbonManifest v1 {
  v:                1
  project:          { uuid, rootCommitSHA, normalizedRemote?, cloneURL?, localOnly: Bool }
  stream:           "worktreeSnapshot" | "history" | "nativeTranscript" | "lock"   // "lock": §3.4, §17
  sessionID:        String?              // nil only for the localOnly project-base history chain
  branch:           String?              // session streams
  baseSHA:          String?              // session streams; the diff anchor, preserved verbatim
  ownerDeviceID:    String
  ownerGeneration:  UInt64               // the resuscitation fence stamp (§10)
  snapshotSeq:      UInt64               // monotonic per (stream, sessionID, ownerGeneration), starts 1
  parentManifest:   String?              // manifestID of the previous manifest in this chain
  branchTipSHA:     String?              // worktreeSnapshot: tip at capture; history: the new tip
  wtCommitSHA:      String?              // worktreeSnapshot only
  worktreeTreeSHA:  String?              //   "
  idxCommitSHA:     String?              //   "  (nil when indexUnavailable)
  indexTreeSHA:     String?              //   "
  indexUnavailable: Bool                 // default false
  payloadKind:      "gitPack" | "jsonlSegment" | "lockEvent"
  segmentOffset:    UInt64?              // jsonlSegment: byte offset of this segment in the file
  backendSessionID: String?              // nativeTranscript stream
  shards:           [ { id: String, byteCount: UInt64 } ]   // ordered; plaintext concatenation = payload
  createdAtMs:      UInt64               // producer wall clock, informational only — never used for ordering
}

6.4 Cadence state machine (D8)

Per enrolled session: interval = 20 s. After each tick that no-ops at step 1 or 4: interval = min(interval × 2, 120 s). Any of the following resets interval = 20 s and schedules an immediate tick: turn boundary (the seams finalize(.auto) was built for), a sendInput, a worktree file event if a watcher is available. Turn boundaries always snapshot even if the last tick was < 20 s ago. Producer pauses entirely while the session is transferring/integrating and while the app is quiescing for shutdown.

6.5 nativeTranscript producer

On each turn boundary (piggyback the §6.4 trigger): stat the native file; if size > lastOffset, read [lastOffset, size), slice to ≤ 2 MiB plaintext shards, manifest with payloadKind = jsonlSegment, segmentOffset = lastOffset. If size < lastOffset (CLI rewrote/compacted the file): emit a rebase manifest — segmentOffset = 0, whole file, parentManifest = nil (chain restart); mirrors GC the orphaned old chain normally.

6.6 history producer and origin-trust (D16)

Trigger: the branch tip moved — hook the existing onCommit callback that finalize already fires (WorktreeManager.swift:449-476) plus the ship/integrate/reclaim sites. (2026-07-21:) base-branch and nvrsion-trunk tip moves at the integrate/promote/interceptor-landing sites and at each nvrsion land also produce project-base chain manifests (§3.2) — emitted synchronously at the landing site, before any lock-release publication, because those heads are the release fence (§17.4). Pack:

positives:  <newTipSHA>
negatives:  ^<prevHistoryTipSHA>                      // if a prior history manifest exists
            ^<originSafeSHA>                          // see below; omitted in paranoid mode / localOnly

originSafeSHA selection (this is the correctness-critical part — read twice):

  1. Candidate = baseSHA if git merge-base --is-ancestor <baseSHA> <lastKnownOriginDefault> succeeds, where lastKnownOriginDefault = the remote-tracking ref as of the last actual fetch/clone/push (never fetch here — no new network ops).
  2. Otherwise candidate = lastKnownOriginDefault itself (the pack then includes the local main-ahead-of-origin delta — bigger, still correct).
  3. Paranoid mode (per-project toggle, D16) or localOnly: no origin negative at all; everything is sharded; the disk cap governs. Default off.

Honest limit: if origin history is force-push-rewritten after our last fetch and the old objects are GC'd server-side, a restore that needs them can fail (§13). Detection: any real fetch that observes a non-fast-forward of the default branch ⇒ producer emits a re-seed manifest (full pack, parentManifest = nil) for each affected chain.

7. Store — CarbonShardStore (every member)

7.1 Layout

Actor, sibling of TranscriptMirrorStore, rooted beside the mirror root:

<root>/carbon/<projectUUID>/
  objects/<aa>/<shardID>            # aa = first two hex chars; ciphertext blobs; write-once
  manifests/<aa>/<manifestID>       # ciphertext CBOR; write-once
  heads.json                        # this member's head per stream + durability ack cache (plaintext:
                                    #   contains only IDs/seqs/generations — no content, no secrets)
  placement.json                    # cached replica-set computation; always recomputable

Writes: temp file in the same directory + rename() (atomic), fsync the file before rename. A put of an existing ID is a no-op success (idempotent). Eviction of a project = directory drop, mirroring TranscriptMirrorStore.removeSource (:112-116).

7.2 Verify on write AND on read (load-bearing, not optional)

put: re-hash the blob; mismatch ⇒ reject, log, never store. get for serving a peer or a restore: re-hash before returning; mismatch ⇒ quarantine the file (objects/quarantine/), reply notHeld, schedule a re-fetch. A bit-rotted mirror must degrade to "doesn't have it," never poison the mesh. Required tests in §16.

7.3 Retention + GC (D11 — mirror-autonomous, pure local rules)

Run per project on the existing sweep cadence (piggyback the 60 s auto-archive timer, AppStore.swift:6481-6493, but a full GC pass at most hourly). Rules, in order:

  1. Retained manifests =
    • last K = 8 manifests of every worktreeSnapshot chain, per (sessionID, highest observed ownerGeneration) — lower-generation chains are retained only until rule 3;
    • the entire history and nativeTranscript chains of every live session;
    • the entire project-base history chain while the project is enrolled.
  2. Mark: decrypt retained manifests (members hold the key), mark every referenced shard.
  3. Session end: a session gossiped shipped/archived/tombstoned starts a grace timer (default = the archived-worktree cleanup interval already in Settings, AppStore.swift:6548-6569); on expiry drop its three chains from "retained."
  4. Ex-replica-member: if this member left a project's replica set (§5.2), start a 14-day grace, then treat the whole project as unretained.
  5. Sweep: delete unmarked blobs whose mtime is > 48 h old (the 48 h floor protects shards that landed before their manifest).
  6. Producer-side: on session archive, delete refs/nucleic/carbon/<sessionID>/* so the repo's own git gc can reclaim snapshot objects.

Disk budget: per-project cap (default 2 GiB, Settings). At cap: stop accepting/producing new manifests for that project, surface not protected: disk cap — never silently drop old data first (rollback depth is the product promise; the user raises the cap or opts out).

7.4 Store API (as built, phase 1 — Sources/NucleicCore/Carbon/CarbonShardStore.swift)

actor CarbonShardStore {
  // One store spans every project, so keys resolve per project (credential vault / escrow),
  // not via a single injected CarbonCrypto. A nil key ⇒ blobs stay opaque: serving works,
  // manifest decode throws .keyMissing, GC retains everything (safe).
  init(root: URL, keyProvider: @Sendable (UUID) -> CarbonCrypto.ProjectKey?)
  func putShard(ciphertext: Data, expectedID: String, projectUUID: UUID) throws  // verifies, atomic, idempotent
  func shard(id: String, projectUUID: UUID) -> Data?                   // verifies; quarantines on rot
  func hasShard(id: String, projectUUID: UUID) -> Bool
  func putManifest(ciphertext: Data, expectedID: String, projectUUID: UUID) throws
  func manifestBlob(id: String, projectUUID: UUID) -> Data?            // what a mirror serves — no key needed
  func manifest(id: String, projectUUID: UUID) throws -> CarbonManifest?   // decrypts + decodes
  func heads(projectUUID: UUID) -> [CarbonStreamKey: CarbonHead]
  func advanceHead(_ head: CarbonHead) throws                          // enforces §10 fencing rules
  func missingShards(for manifest: CarbonManifest) -> [String]
  func collectGarbage(project: UUID, policy: CarbonRetention) throws -> CarbonGCReport
  func evict(project: UUID) throws
}

8. Replication — wire vocabulary + CarbonSyncCoordinator

8.1 Wire messages (new file Sources/NucleicProtocol/Sync/CarbonMessages.swift)

CarbonFormat.version = 1, independent of SyncProtocol.version (the TranscriptSyncFormat pattern). Capability bits: WireClientCapabilities.canMirrorCarbon (Macs/runners only), WireCapabilities.canServeCarbon. Every client verb below is sent only to peers advertising the capability (unknown ClientMsg tags disconnect old peers — hard rule). Roster addition: alwaysOn: Bool (decode-default false).

struct CarbonStreamKey { projectUUID, stream, sessionID? }
struct CarbonHead      { key: CarbonStreamKey, manifestID, ownerGeneration, snapshotSeq,
                         keyMissing: Bool }                    // keyMissing: §4.3

ClientMsg.fetchCarbonHeads(projectUUID: UUID?)                 // nil = all enrolled
HostMsg.carbonHeads([CarbonHead])                              // full state, idempotent

HostMsg.carbonHeadAdvanced(CarbonHead)                         // owner push on every manifest

ClientMsg.fetchCarbonManifests(ids: [String])                  // ≤ 64 per request
HostMsg.carbonManifestData(id: String, blob: Data)             // one per manifest; blob = ciphertext
HostMsg.carbonManifestUnavailable(id: String, reason)          // notHeld | internalError

ClientMsg.fetchCarbonShards(ids: [String])                     // ≤ 256 per request
HostMsg.carbonShardData(id: String, blob: Data)                // one frame per shard (≤2MiB+ε)
HostMsg.carbonShardUnavailable(id: String, reason)
ClientMsg.carbonShardAck(ids: [String])                        // window ack, see §8.3

Checklist per new case (copy of the COVALENCE_RUNNER §11.4 recipe): envelope tag + SyncClient.Event case + branches in ConnectionHandler.messageLoop, SyncClient.messageLoop, iOS RemoteStore (×2), iOS HostConnection — the iOS ones compile only under xcodebuild and every case there is inert.

8.2 Steady-state flow (push-notify, pull-fetch)

  1. Owner writes manifest → sends carbonHeadAdvanced to reachable replica-set members.
  2. A replica member receiving a head it lacks: fetchCarbonManifests([manifestID]) → decrypt → walk parentManifest links until it reaches a manifest it holds (or chain start) → fetchCarbonShards(missing) from the sender; on shardUnavailable/disconnect, probe other holders (owner first, then replica-set members claiming the head) — the TranscriptSyncCoordinator.attempt fallthrough discipline (:96-111).
  3. Member persists shards → manifests → then advances its head (this order makes the gossiped head an honest "I hold everything reachable" claim = the durability ack, §5.3).
  4. On (re)connect to any peer: exchange fetchCarbonHeads both ways — heads are state, not events; missed pushes are irrelevant.

Concurrency: per-member coordinator caps — maxConcurrentChains = 3 (the transcript coordinator's number), one in-flight fetch per stream key (reserve synchronously before awaiting, the inFlight guard pattern).

8.3 Flow control

Shard fetch uses the transfer window, not transcript fire-and-stream: server sends up to 4 un-acked carbonShardData frames, then waits for carbonShardAck; client acks every frame (batched ids allowed). A shard is one frame (2 MiB ciphertext + overhead ≪ 16 MiB cap). Server inserts Task.yield() between frames (the ConnectionHandler streaming discipline).

8.4 Transport policy (D5)

  • Steady-state (carbonHeadAdvanced + resulting manifest/shard pulls): any transport, including the relay, iff the manifest's total shards[].byteCount ≤ 4 MiB (default, Settings). Larger payloads over the relay: the mirror defers that chain until a LAN/tailnet path exists and the durability chip shows reduced redundancy (awaiting direct link).
  • Bulk backfill (§8.5): LAN/tailnet candidates only — verbatim the transcript-sync restriction (TranscriptSyncCoordinator.swift:59-62).
  • Per-project opt-out of relay entirely (Settings).

8.5 Backfill (new replica member, returning device, replaced device)

Same coordinator, no special mode: exchange heads → walk chains → fetch missing, oldest chain first, LAN/tailnet only, throttled to maxConcurrentChains. A replaced device is just a new member that happens to hold nothing; base git history comes from origin via normal createProject clone, so backfill volume is deltas only (except localOnly). Redundancy is restored when its heads match; the chip returns to protected.

8.6 Platforms (D13)

Producer + mirror + restore all compile and run on macOS and Linux (nucleicd) from day one. Runners ship alwaysOn = true by default (⇒ pinned into every replica set they're eligible for, §5.2). iOS: no Carbon store, no key custody, every wire case inert; phones may display durability state carried on existing session gossip.

9. Restore — CarbonRestore

9.1 Materialize a session worktree from shards (the algorithm)

Preconditions: caller holds the project key; for resuscitation, caller has already claimed the ownership fence (AGENT_RESUSCITATION §4) — restore itself is ownership-agnostic and also serves the manual restore browser (§12) and device-replacement flows.

  1. Ensure project. Resolve by identity triple (UUID / root commit / normalized remote, AppStore.swift:9802-9813 pattern); absent ⇒ createProject(cloneURL) from the manifest's project.cloneURL; localOnly ⇒ git init an empty repo (the base chain supplies everything).
  2. Select manifests. Newest mesh-durable worktreeSnapshot manifest M for the session (or a user-chosen older one, restore browser); its history chain head; its nativeTranscript head. Verify chain integrity by walking parentManifest to a chain start; any missing manifest ⇒ fetch (§8.2) before proceeding.
  3. Apply packs, oldest→newest, history chain first, then worktreeSnapshot chain: for each manifest with payloadKind == gitPack: fetch/read shards in order, decrypt, concatenate → git index-pack --fix-thin --stdin into the project repo's objects. Chain order guarantees thin bases exist (§6.1 step 6 negatives are always prior-chain positives or origin objects present from step 1). An index-pack failure here is terminal for this manifest ⇒ retry once from a different holder (corrupt-blob suspicion), then fall back to the newest older snapshot manifest whose pack applies, surfacing "restored to older snapshot
  4. Branch + worktree. git branch <branch> <M.branchTipSHA> (must not already exist — if it does and points elsewhere, abort: another live copy may exist; surface, don't force); git worktree add <path> <branch> (path allocation via the existing importer discipline, SessionTransferImporter path-component validation); preserve M.baseSHA as the diff anchor.
  5. Re-dirty the tree (exact working files): git read-tree -u -m <M.branchTipSHA> <M.worktreeTreeSHA> — two-tree merge updates working files including deletions; untracked-at-snapshot files materialize (they are tracked in worktreeTree).
  6. Replay the index (D10): if M.indexTreeSHA != nil: git read-tree <M.indexTreeSHA> (index only), then git update-index --refresh (best effort; non-zero exit here is fine — it just reports files whose stat differs). Result: git status shows staged = tip→indexTree, unstaged = indexTree→worktree, untracked = files in worktreeTree but not indexTree — byte- and stage-identical to the dead host at snapshot time. If indexTreeSHA == nil (indexUnavailable), skip: everything shows unstaged; the manifest flag drives an honest note in the UI.
  7. Native memory: concatenate the nativeTranscript chain segments (honor segmentOffset; a rebase manifest restarts the file) to the CLI's expected path; keep backendSessionID ⇒ true claude --resume. Chain absent/incomplete ⇒ drop the id, use the existing fresh-context fallback (SessionTransferImporter.swift:368-390).

Post-conditions to assert in tests (§16): git status --porcelain=v2 output equals the producer's at capture; file bytes equal; branch tip and baseSHA equal.

9.2 Resuscitation integration (AGENT_RESUSCITATION §6.3 step 4 replacement)

The elected resuscitator, after claiming the fence, runs §9.1 with M = newest mesh-durable snapshot whose ownerGeneration ≤ the generation it observed (never a stale-generation manifest, §10). Election preference (feeds AGENT_RESUSCITATION §6.2): among candidates, prefer replica-set members holding the newest mesh-durable head, then lowest hostID. RPO = one cadence interval; UI copy: "recovered to within ~20 seconds."

9.3 Returning partitioned owner

On reconnect it back-fills as a mirror. For sessions where it observes a higher ownerGeneration: its own unshipped local snapshots are stale by fencing — manifests stamped with the old generation are retained as chain ancestors but can never advance a head (§10), and every member ignores them for durability/restore selection. Discarded, not merged; no CRDT (MESH_TRANSFER invariant #1).

9.4 Restore browser (D15)

Per-session UI: list retained snapshot manifests (time, seq, changed-file count = cheap git diff-tree --name-only <tip> <worktreeTree> | wc -l at display time, durability state). Actions:

  • "Open as new worktree" (safe, default): §9.1 into a fresh worktree + new session branch fork — never touches the live session.
  • "Roll back this session's worktree" (destructive): only for the session's current owner on its live worktree; snapshot first (so rollback is itself undoable), then steps 5–6 onto the existing worktree. Confirm dialog states exactly what will be overwritten.

10. Consistency & fencing (composes with AGENT_RESUSCITATION §4 — never weakens it)

  • Single producer per stream: only the session owner snapshots; only a project's enrolling hosts produce base-history manifests. Mirrors are read-only. Shards/manifests are immutable and content-addressed ⇒ replication can never conflict; the only ordered structure is the manifest chain.
  • Head-advance rule (enforced in advanceHead, everywhere, no exceptions): a head may only advance to (gen', seq') if gen' > gen, or gen' == gen ∧ seq' > seq. A manifest whose ownerGeneration is lower than the highest generation this member has observed for the session (via SessionSummary ownership gossip) is stored if referenced as an ancestor but never advances a head and is never selected for restore.
  • Heads are CRDT-free by construction: single writer per (stream, generation), merge = max on (generation, seq) — the same monotonic-scalar discipline as the runner-pool epoch (pool.ts:72-74) and the session fence. Deliberately not consensus.
  • createdAtMs is informational only. Ordering NEVER uses wall clocks.

11. ShardSink (D14 — design only, do not build)

protocol ShardSink {          // an off-mesh shard holder, e.g. R2 via nucleic-edge
  func put(blobs: [(id: String, ciphertext: Data)]) async throws
  func has(ids: [String]) async throws -> Set<String>
  func get(id: String) async throws -> Data
}

Because shards/manifests are encrypted and self-verifying (§4.2), a sink needs zero trust and zero keys; it is simply another holder the coordinator can push to / restore from. The R2 implementation belongs to the CLOUD_RUNTIME effort; §3.5's checkpoint set should adopt these IDs rather than invent a parallel format. Total-device-loss recovery then = iCloud key escrow (§4.4) + sink shards.

12. Settings & UI (D15) — Settings ▸ Covalence ▸ Carbon

  • No master toggle, no per-project opt-out — Carbon is mandatory (D17, 2026-07-21). Every project enrolls; the only non-enrolled states are honest inabilities (repo shape, disk cap), surfaced per §5.3/§13, never a preference. (The former nucleic.carbon.enabled toggle and per-project opt-out are removed; §17.6 explains why locking requires this.)
  • Per-project: paranoid mode (D16), disk cap (default 2 GiB), relay policy + 4 MiB threshold.
  • Cadence nucleic.carbon.snapshotSeconds (default 20, clamp 5…300).
  • Pinned members editor (surface alwaysOn; runners pre-checked).
  • Read-only: per-project replica set + per-member sync lag; per-session durability chip (states table §5.3); key-escrow status.
  • The restore browser (§9.4) on every session's detail pane.
  • All copy rules from §13 (never promise losslessness; always name the reason a thing is unprotected).

13. Failure handling (implementors: treat this table as normative)

Failure Behavior
write-tree unmerged index indexUnavailable = true, single-tree snapshot, amber note
Branch tip moved mid-snapshot Abort tick; tip-move trigger re-enters
pack-objects non-zero Log + retry next tick; 3 consecutive failures ⇒ chip not protected: snapshot failing + notification
Disk cap reached (producer or mirror) Stop new manifests for that project; chip not protected: disk cap; never evict old data to make room
Shard fails verify on read Quarantine, reply notHeld, schedule re-fetch from another holder
Replica member missing project key Serves blobs it has, reports keyMissing = true; owner surfaces "replica cannot restore (no key)"
No replica ack for a head after 10 min of connectivity Chip degrades per §5.3; no retries beyond the coordinator's normal cadence
Relay payload over threshold Defer chain until direct path; chip reduced redundancy (awaiting direct link)
index-pack --fix-thin fails at restore Retry from another holder once; then fall back to newest older applicable snapshot; surface "restored to older snapshot"
Origin force-push GC'd assumed-present objects Restore may fail for affected chains; producer emits re-seed manifests on next non-fast-forward fetch observation; per-project paranoid mode (D16) eliminates the class
iCloud Keychain escrow write fails Enroll anyway; chip key not escrowed; retry on next launch
All key-holders + escrow lost Data unrecoverable — by design; documented in Settings footnote
Mesh of one Chip not protected + pair-a-device nudge; producer still runs (local snapshots power the restore browser)

Honesty rules for all user-facing copy: RPO is the cadence ("your last ~20 seconds may be re-derived by the agent, not byte-recovered"); deterministic encryption leaks intra-project equality to non-key-holders (invisible in practice — replicas hold keys); repo-shape-excluded projects say so; ignored-file churn is the user's .gitignore problem and the snapshotter logs the top offenders to help.

14. New files & types (build checklist)

File Contents
Sources/NucleicProtocol/Sync/CarbonMessages.swift §8.1 verbs, CarbonFormat, CarbonHead, CarbonStreamKey, capability bits
Sources/NucleicCore/Carbon/CarbonCrypto.swift §4 key derivation, seal/open, ID computation; Keychain escrow (Darwin-only compiled)
Sources/NucleicCore/Carbon/CarbonShardStore.swift §7 actor
Sources/NucleicCore/Carbon/CarbonManifest.swift §6.3 model + canonical CBOR codec
Sources/NucleicCore/Carbon/CarbonSnapshotter.swift §6 producer actor (all three streams)
Sources/NucleicCore/Carbon/CarbonPlacement.swift §5 pure functions (roster in, replica set out)
Sources/NucleicCore/Carbon/CarbonSyncCoordinator.swift §8 reconciler
Sources/NucleicCore/Carbon/CarbonRestore.swift §9 algorithm
Sources/NucleicCore/Carbon/ShardSink.swift §11 protocol only
Sources/NucleicCore/Carbon/CarbonLockLedger.swift §17.2 event model, fold, compaction snapshots
Sources/NucleicCore/Carbon/CarbonLockManager.swift LOCKING's LockManager relocated (§17.7): authority arbitration + ledger append/fold collaborators
Sources/NucleicProtocol/Sync/CarbonMessages.swift (same file as row 1) + §17.5 lock verbs
GRDB migration (next version slot) enrollment/opt-out flags, paranoid mode, disk caps — additive columns, the v23/v24 pattern (GRDBMetadataStore.swift:203-238)

15. Phasing (parallel track to AGENT_RESUSCITATION §10 per D12; integration at phase 4)

Each phase lands only with its acceptance criteria green.

  1. Crypto + store + manifest. CarbonCrypto, CarbonShardStore, CarbonManifest, placement pure-functions. Accept: §16 store/crypto/placement tests green on macOS + Linux; a scripted round-trip (encode→encrypt→put→get→decrypt→decode) is byte-stable across platforms.
  2. Producer + local restore. CarbonSnapshotter (all three streams), hidden refs, CarbonRestore against the local store only. Accept: kill-a-worktree fixture restores byte- and stage-identical (porcelain=v2 equality) with no network; producer provably never mutates user-visible git state (status/branch/index hashes unchanged around a tick).
  3. Wire + replication. CarbonMessages, capabilities, CarbonSyncCoordinator, transport policy, durability accounting. Accept: 3-member loopback (the SessionTransferTests style) converges through partition/rejoin; relay threshold honored; iOS builds via xcodebuild with inert cases.
  4. Resuscitation integration (requires the resuscitation fence, its §10 items 1–2). Restore-from-shards replaces §6.3 step 4; election preference; stale-generation manifest rules end-to-end. Accept: the resuscitation loopback test recovers uncommitted + staged state, RPO ≤ cadence, and a returning partitioned owner's late snapshots never win.
  5. Backfill + GC hardening + key escrow. Replaced-device flow, ex-member grace, disk caps, Keychain escrow + recovery flow. Accept: a fresh member reaches protected from zero; GC never collects reachable data (property test); escrow round-trip on a second Mac.
  6. Surfaces. Chip, Settings pane, restore browser (§9.4), notifications. Accept: every §13 row has a visible, worded surface; restore browser's "open as new worktree" and "roll back" both covered by UI tests.
  7. Carbon Locks (added 2026-07-21; requires phases 1–3, parallel to 4–6). Ledger stream
    • fold (§17.2), authority + generation fencing (§17.3), the release fence in both the autoship/worktree and nvrsion release flows (§17.4), project-base history chains for cloned projects (§3.2), relocation of LockManager → CarbonLockManager (§17.7), lock wire verbs (§17.5), lock-viewer re-source. Accept: §17.9 suites green; the LOCKING §10 matrix passes unchanged against CarbonLockManager on a single host (degenerate case); 3-member contention loopback — never a double grant, release invisible without content, a partitioned straggler keeps seeing "held".

16. Tests (required deliverables, per house discipline)

  • Crypto: determinism (same plaintext ⇒ same blob/ID), tamper detection (any flipped bit fails open), cross-platform vector fixtures (encrypt on macOS, decrypt on Linux, and vice versa), HKDF vectors frozen.
  • Store: idempotent put; atomic write (kill mid-put leaves no partial visible); verify-on- read quarantines and falls through; GC property test — never collects anything reachable from a retained manifest or live head; eviction = directory drop.
  • Placement: pure-function table tests — pinned always included; R winners stable under irrelevant roster changes; minimal reshuffle on join/leave; empty/one-member meshes degrade per §5.3.
  • Producer: captures tracked+untracked+deletes+modes; respects .gitignore; identical trees ⇒ no manifest; index tree captures staged hunks; unmerged-index fallback; hidden refs survive git gc; user-visible git state untouched (before/after porcelain + HEAD + index checksum equal); thin pack applies on a receiver holding only the negatives; history originSafeSHA cases 1/2/3 each exercised; native rebase manifest on file shrink.
  • Restore: full §9.1 on real temp repos — porcelain=v2 equality incl. staged/unstaged split, deletions, untracked; older-snapshot fallback; localOnly restore with no origin; branch-collision abort; fresh-context fallback when native chain absent.
  • Replication: 3-member convergence with partition/rejoin; stale-generation manifests never advance heads or get selected; durability ack only after shards+manifests persisted; window flow control (server never exceeds 4 unacked); relay size threshold; bulk backfill restricted to LAN candidates; keyMissing surfaced.
  • Locks: the §17.9 suite (ledger fold determinism, release fence under autoship + nvrsion, authority election/fencing, cross-host contention, repo-shape gate).
  • Cross-platform: protocol + core suites green on Linux and Darwin; iOS xcodebuild with every new wire case inert.

17. Carbon Locks — the locking system, mesh-native (added 2026-07-21)

What changed: the file-locking system specified in LOCKING.md moves into Carbon and its state becomes mesh state. Locking's semantics — all-or-nothing acquisition, the waiter queue with demotion, update-on-grant + re-ground, cascade, and the landing detection of LOCKING §4.4 — are unchanged and stay normative in LOCKING.md. What Carbon adds is where the state lives (a replicated lock ledger), who can contend (every mesh device, not just the host that happens to run the sessions), and when release is visible (fenced on the landed files having propagated). This is possible only because Carbon and Covalence are mandatory (D17): a lock subsystem cannot sit behind a feature flag.

17.1 Why Carbon owns locks

A lock is a claim about data integrity — "this file's next content is being produced by exactly one session, and everyone else must see that content before acting on the file." That is a data-layer statement, and Carbon is the data storage/integrity layer: it already owns the authoritative replicated record of every project's content (manifest chains) and the machinery to move that content (replication, heads, durability accounting). Housing locks anywhere else would force the lock and the content it guards to travel separately and reunite — the exact race the release fence (§17.4) exists to close.

17.2 The lock ledger (stream lock)

Per lock domain — (project identity, domainRef), where domainRef is the LOCKING §2 rootRef for worktree projects, or the trunk branch for nvrsion projects — Carbon keeps one append-only lock ledger: a manifest chain (stream lock, §3.4) of lock events:

CarbonLockEvent (CBOR, the manifest payload — small, usually one shard)
  kind:            acquired | queued | released | retargeted | authorityChanged | snapshot
  sessionID:       String?
  deviceID:        String                      // the host running the session
  paths:           [String]
  requiredHeads:   [CarbonHead]?               // `released` only — the release fence (§17.4)
  ledgerSeq:       UInt64                      // per-domain, monotonic within one authority generation

Single writer per chain (§10 discipline): the domain's lock authority (§17.3). Every mesh member replicates the ledger like any other stream; a member's current lock table is a pure fold of the chain (compacted periodically by a snapshot event so replay stays bounded). The LOCKING §8 lock/queue viewer re-sources from the local fold on Macs/runners; phones hold no Carbon store (D13) and keep rendering the existing sync-protocol lock snapshots (LOCKING §8 iOS parity), now sourced from their host's ledger fold — read-only, as ever.

17.3 The lock authority

Exactly one host arbitrates a domain at a time — acquisition, queue order, grant, demotion all run there, preserving LOCKING §4's single-actor semantics (the LockManager actor simply runs on the authority; §17.7). Concretely:

  • Who: the project's owner-of-record host — for a control project, the host running its shared control container. Bootstrap: the first host to open a session in the domain appends authorityChanged(gen+1) and becomes the authority.
  • Fencing: authorityGeneration rides ownerGeneration on every ledger manifest (§3.4); the §10 head-advance rule applies verbatim — a stale authority's events are stored as chain ancestors but never advance the head. Same discipline as the resuscitation fence; deliberately not consensus.
  • Remote sessions: a session on another host acquires/releases via the §17.5 verbs, routed to the authority; arbitrate on the remote host becomes a thin proxy. The agent-facing contract (LOCKING §4.6) is unchanged — the suspension just spans the wire.
  • Authority loss: if the authority dies, the AGENT_RESUSCITATION §6.2 election (preferring replica-set members holding the newest ledger head) elects a successor, which folds the ledger, bumps the generation, and resumes arbitration. In-flight waiter continuations on the dead host died with it (so did its agents); their sessions re-acquire — exactly the LOCKING §6.2 restart argument, mesh-wide.
  • Unreachable ≠ dead: a partitioned remote session's acquire waits, surfaced as "waiting for lock authority" — never fail-open (fail-open would permit a concurrent edit, the one thing the system exists to prevent). A mesh of one degenerates to today's fully local behavior with zero added latency (§17.4).

17.4 The release fence — no device sees "released" before it has the files

Requirement (load-bearing): before a lock releases, Carbon updates the project files across the mesh; a device must not observe the lock as released until it holds all of the updated files via Carbon.

Release is therefore two-stage:

  1. Publication (authority-side). When the landing is certain, the authority appends released(paths, requiredHeads) to the ledger. requiredHeads names the Carbon heads that carry the landed content, per release flow:

    Release flow Landing signal (unchanged) requiredHeads
    Standard autoship / worktree (LOCKING §4.4: mediated merge, interceptor event, hasLanded poll) the work landed in the session's parentRef the project-base history head (§3.2) whose pack covers the parent tip now containing the work — emitted synchronously at the landing site (§6.6), before the release is appended; plus the session-branch history head when the parent is another session's branch (nested child)
    nvrsion (NVRSION §4: host-certain release at turn-end / idle / file-switch) the trunk commit(s) already landed per edit the trunk history head (§3.2) covering the last landed commit touching paths — emitted per land, so it always exists before any release fires

  2. Observation (per-device). A device treats the lock as released iff its local CarbonShardStore holds every requiredHead — manifests and all referenced shards (the honest-head rule, §8.2 step 3) — and, for a device with a local clone of the project, the coordinator has applied the heads' gitPack payloads (index-pack) and fast-forwarded the named local branch ref to the manifest's branchTipSHA. Until then the device's fold renders the lock as releasing — syncing files: a local session's acquire of those paths queues exactly as if the lock were still held, and the UI says why.

Properties:

  • The landing host releases immediately. It holds requiredHeads by construction (it produced them), so single-host operation — and a mesh of one — has zero added latency. Today's behavior is the degenerate case, not a special case.
  • A granted remote session always has the content. Its device could not have observed the release (and so could not be granted) without holding the landed objects, so LOCKING §4.5 update-on-grant always finds its merge source present locally. The fence is what makes a mesh-wide grant sound, not merely visible.
  • Non-fast-forward on ref apply (the local base diverged from the manifest's tip — e.g. the user committed to dev on that host out-of-band): never force; hold the lock at releasing, surface the divergence like an update-on-grant conflict (LOCKING §4.5), let a human resolve, then the fold resumes.
  • Fail toward safety on readers, progress on the writer. A device that cannot fetch (partition, relay-deferred bulk) simply keeps seeing the lock held — correct, since it provably lacks the content. The authority and the landing host are never blocked by a straggler.

17.5 Wire additions (extends §8.1; same rules — capability-gated, additive)

ClientMsg.carbonLockAcquire(domain, sessionID, paths: [String])     // remote session → authority
ClientMsg.carbonLockRelease(domain, sessionID, paths: [String]?)    // remote → authority; nil = all
HostMsg.carbonLockDecision(domain, sessionID, outcome)              // granted | queued | cancelled
HostMsg.carbonLockLedgerAdvanced(CarbonHead)                        // authority push; pull-fetch per §8.2

Ledger manifests/shards ride the existing fetch verbs unchanged. All four cases follow the §8.1 checklist (envelope tag, SyncClient.Event, the four exhaustive switches); the iOS cases are inert (phones hold no Carbon store and never acquire, D13) — the phone lock viewer keeps riding the sync-protocol lock snapshots (LOCKING §8), sourced from its host's fold.

Ownership is enforced per verb (2026-07-27). sessionID arrives as data on the wire; it is not a capability, and requireControl() is a pairing-scope check, not a session one. So the authority binds each remote session to the authenticated deviceID that acquired it (remoteLockSessions) and checks that binding on every verb:

  • acquire refuses a session id that names a session running locally on the authority (those arbitrate in-process and never route over the wire), and refuses one already claimed by a different device while that device still holds locks — a stale claim holding nothing stays re-claimable so a transferred session can arbitrate from its new host.
  • release is honored only from the owning device, and only for the project it was acquired under. It also takes the same formatVersion gate acquire already had.

Refusal always leaves the lock held (fail closed, §17.3): the owning host's own landing-driven release retries and the manual force-release remains the escape hatch. Without this, any control-scope peer could free any session's locks — including a session editing locally on the authority — by naming its id. Note that requiredHeads on a release is a liveness hint (when each device observes the release, §17.4), never a proof that anything landed; trusting it off the wire is only safe because the release itself is authorized here.

17.6 Mandatory status (D17) and the enrollment gate

Carbon has no master toggle and no per-project opt-out (§12 amended); Covalence's mesh stack likewise always runs (the old covalenceEnabled gate is removed — COVALENCE_QUEUE §1). This is what lets locking live here: behind a feature flag, flipping the flag would change correctness, not preference. What remains honest and unchanged:

  • The repo-shape gate (§3) is an inability, not an opt-out. A shallow/partial/LFS repo cannot be sharded, so its lock domain cannot be mesh-fenced: locking for it stays single-host exactly as today (LOCKING applies locally; no ledger), the chip says not mesh-protected: <reason>, and Covalence dispatch of its sessions to other hosts is refused — running them elsewhere without mesh locks would reintroduce the unlocked race.
  • Paranoid mode, disk caps, relay thresholds stay per-project knobs — they tune transport and trust, never whether Carbon runs.
  • Locking remains Nucleic-Control-only (LOCKING §4.4 scope note) — unchanged by the move.

17.7 What moves, what retires

LOCKING.md component Disposition
LockManager actor (acquire / queue / demotion / grant / re-ground) Moves to Sources/NucleicCore/Carbon/CarbonLockManager.swift, semantics intact; runs on the domain authority; gains ledger append + fold collaborators
Landing detection (mediated, interceptor, hasLanded poll) Kept — now the publication trigger (§17.4 stage 1)
Release effect (releaseAll → waiters granted) Split: publication (authority) + per-device observation (the fence)
Launch reconstruction (LOCKING §6) Kept on the authority for its own sessions; cross-checked against the folded ledger (the ledger wins on conflict — it is the authoritative record; divergence is logged and corrected with follow-up events)
nvrsion release governor (NVRSION §4) Kept; its release calls now publish with trunk requiredHeads
lockQueueSnapshot / lock viewer Re-sourced from the local ledger fold on Macs/runners; phones keep the sync-protocol snapshots (LOCKING §8), fed from their host's fold

17.8 Failure handling (extends §13; normative)

Failure Behavior
Device cannot fetch requiredHeads Lock renders releasing — syncing files; local acquires queue; no timeout-release, ever
Authority host dies §6.2-style election; successor folds the ledger, bumps the generation; the dead host's waiters re-acquire
Stale authority partitioned back Its ledger events never advance the head (§10); its local grants are void — those sessions re-acquire against the live authority
Release published, then the landing host dies before any replica holds requiredHeads If a replica got the shards, normal durability recovers; if none did, the content is lost with the host and the lock stays releasing on other devices — surfaced after 10 min as release stranded: content lost with <host> with a manual force-release (re-opens the file at its pre-landing content; destructive, confirm dialog)
Fold divergence from the authority's live state The ledger wins; the authority logs and appends corrective events
Non-fast-forward applying a base-branch head Hold at releasing; surface as a conflict (§17.4); never force

17.9 Tests (extends §16)

  • Ledger: fold determinism (same chain ⇒ same table on every member); compaction-snapshot equivalence; stale-generation events ignored by the fold.
  • Fence: a release is invisible on a member until manifests + shards are held AND packs applied — exercised under both the autoship/worktree flow and nvrsion; the landing host releases immediately; a granted remote session finds its update-on-grant merge source present (soundness).
  • Authority: remote acquire/queue/grant round-trip; authority death → election → generation bump → re-acquire; a partitioned stale authority's grants are void.
  • 3-member loopback: two hosts contending on one file — never both granted; release propagates only with content; a partitioned straggler keeps seeing "held".
  • Gate: a repo-shape-blocked project keeps single-host locking and refuses cross-host dispatch.
Reference in New Issue View Git Blame Copy Permalink