nucleic/docs/COVALENCE_QUEUE.md

Nucleic — Covalence: the mesh session work queue

Status (2026-07-12): implemented + tested on top of the existing mesh-dispatch stack (MeshDispatchScorer / RunnerPresence casts / the durable mesh_dispatch queue) and the session-transfer engine (mesh P5). All wire changes are field-additive — no new ClientMsg/HostMsg tags, no new capability flags, SyncProtocol.version stays 1. Builds green: macOS (NucleicApp, nucleicd), iOS (xcodebuild), Linux (nucleicd + NucleicProtocolTests in a swift:6.3-bookworm container). Suites: MeshDispatchScorerTests (rebalance contracts), WireMessageTests (Covalence wire + legacy tolerance), MeshDispatchQueueTests (never-lands-locally), SessionTransferTests (covalence carry + mirror adoption).

1. What it is

Amended 2026-07-21: Covalence is mandatory. The mesh stack — pairing, presence, casts, transfer, Carbon replication — always runs; the covalenceEnabled master gate is removed (a mesh of one is simply quiet). "Here" vs "Covalence" remains a per-chat run-target choice: mandatory means the subsystem, not the routing. This is a precondition for Carbon-native mesh locking (CARBON_SHARDING §17.6), which cannot sit behind a feature flag.

The composer on every device gains a run target: Here (the local device — unchanged behavior) or Covalence (the mesh work queue). A Covalence job runs each turn on the optimal host currently available on the user's Covalence mesh, under two invariants:

1. Covalence never selects the issuing device. "Covalence" means anywhere on the mesh but here — the device the job was issued from is excluded at dispatch AND from every later rebalance. (On iOS the non-Covalence option reads "This Mac" — the project's owning host — because a phone can't run sessions; every Covalence candidate is already "not here".)
2. One host owns the session at a time (the locked single-writer invariant), but ownership is fluid between turns: when a strictly better host opens after a turn, the session moves there via the ordinary 2-phase-commit session transfer.

2. Dispatch (the queue's front door)

• Mac: NewChatComposer's Run-on menu → AppStore.dispatchChatToMesh — ranks runnerPresenceByHost peers (self excluded) with MeshDispatchScorer.rank, routes with the correlated idempotent startChat(requestID:) ack, clones-on-dispatch when only a canCreateProjects host is free, and parks the job durably (mesh_dispatch, GRDB v28) when no host can take it — retried automatically on presence change / peer connect.
• iOS: StartChatComposer → RemoteStore.dispatchChatToMesh — same shared scorer, same acknowledged dispatch, over the phone's live host connections.
• The request stamps StartChatRequest.covalenceOriginDeviceID (additive field): the issuing device. The target persists it on the session (session.covalence_origin_device_id, GRDB v29) — non-nil is what makes a session Covalence-managed.

3. Between-turn rebalancing (the queue's scheduler)

The session's current owner is the scheduler (only it holds the controller, so there is no cross-device claim race). Two triggers arm a debounced sweep (scheduleCovalenceRebalanceSweep):

• a turn ends (ingestUI's .runFinished, beside the presence republish), and
• a runner.presence cast changes — "a more optimal host opened" while a Covalence session sits idle here.

rebalanceCovalenceSession gates, in order: not already moving; awaitingInput (never mid-turn); nobody is typing a follow-up (the composer.typing cast — casting doubles as the user- intent veto); not an autoship chat (its post-turn merge pipeline must not race a transfer); per-session cooldown (120 s). Then the shared, pure MeshDispatchScorer.rebalanceTarget decides:

• candidates = presence-known peers, minus this host, minus covalenceOriginDeviceID, holders only (a rebalance never clones), canReceiveSessionTransfer only;
• move only when the best candidate beats the owner's load ratio by more than rebalanceMargin (0.15): at the default capacity denominator (8) a one-session imbalance never moves anyone, two does — and since the move itself shifts one session, a just-over- margin gap collapses to ~0 after the move. No ping-pong by construction.

The move is AppStore.moveSession — the existing transfer engine (quiesce, git bundle with haveSHAs, 2PC, tombstone-before-commit, relaunch recovery). SessionTransferRecord. covalenceOriginDeviceID (additive) rides the offer, so the new owner inherits the management duty and keeps rebalancing/excluding the origin.

4. Casting keeps the transcript everywhere (cheap moves)

Small signals ride casts; the transcript itself rides the full-transcript mesh sync (canSyncTranscripts → TranscriptMirrorStore), which already keeps active sessions mirrored on every Mac. Covalence leverages that at transfer time — the transcript analogue of haveSHAs:

• The importer, at offer time, asks its environment for a mirror complete through the offer's lastSeq (TransferImportEnvironment.mirroredTranscript, default nil; AppStore.mirroredTranscriptForTransfer consults mirrorStore.heldSeq). Events carry canonical source-assigned seqs and are host-invariant, so a current mirror IS the transcript.
• It adopts the mirror into staging, marks the item complete, and the accept claims it via TransferAccept.alreadyHaveItems (additive; an old destination never claims, an old source ignores it and streams — the chunk path acks redundant chunks as duplicates).
• The coordinator skips streaming claimed items, so a between-turn Covalence move usually streams only the git bundle (itself prerequisite-based). Header rewriting at staging is unchanged; native (agent-memory) transcripts still stream — they aren't mirrored.

5. Wire + persistence delta (the §11.4 recipe, all-additive)

Change	Where
StartChatRequest.covalenceOriginDeviceID: String?	Dashboard.swift (decode/encode-if-present)
SessionTransferRecord.covalenceOriginDeviceID: String?	TransferMessages.swift (custom init(from:))
TransferAccept.alreadyHaveItems: [TransferItemKind]	TransferMessages.swift (decode-default [], resumeFrom also decode-defaulted)
MeshDispatchScorer.Candidate.canReceiveSessionTransfer + rebalanceTarget/loadRatio/rebalanceMargin	MeshDispatch.swift (pure, platform-neutral)
Session.covalenceOriginDeviceID	Session.swift + GRDB v29 (covalence_origin_device_id TEXT, nullable)
TransferImportEnvironment.mirroredTranscript(…)	TransferChannel.swift (defaulted extension — non-mirror hosts unchanged)

No exhaustive-switch churn anywhere (no new envelope tags), which is what kept this change compact.

6. Honesty notes / limits

• Load signal is coarse: RunnerPresence.activeSessionCount / maxConcurrent (Macs report nil ⇒ assumed 8), refreshed on change, not a heartbeat. No CPU/RAM/thermal signal yet — the margin + cooldown compensate; a battery/thermal field on the presence card is the natural next additive step.
• Rebalance is owner-driven and holder-only. A phone cannot order host A to hand a session to host B (no such wire verb), and a rebalance never clones a repo onto a new host — only dispatch does. Transfer-ineligible sessions (nvrsion, LFS/shallow/submodule repos, nested stacks, autoship chats) dispatch to Covalence fine but then stay where they land.
• Old peers degrade gracefully: an old host runs a Covalence job as an ordinary unmanaged chat (ignores the origin field); an old destination just receives the full transcript stream; an old source streams a transcript the destination already adopted (harmlessly acked as duplicates).
• The queued (parked) dispatch is durable on Macs only; a phone with zero reachable hosts reports "No Covalence host can take this chat" rather than parking (its host connections are the queue's substrate — a follow-up could park in CastCache-style local storage).

7. Remote environments (planned — added 2026-07-29)

The queue above moves whole sessions; Covalence's planned second service moves environments. An agent keeps running exactly where it is, but the containers/VMs its tools ask for may be served by another mesh member as remote environments: a Windows host's agent leases a real macOS VM from a Mac in the mesh (the one subsystem WINDOWS_PORT can never have locally); an Apple-silicon host leases a native x86_64 Linux VM/container from an amd64 member instead of emulating under qemu (and an amd64 member borrows native arm64 the other way); any member leases a Windows VM from a Windows member — Hyper-V on Windows metal, the same each-OS-served-natively-by-its-own-platform rule as macOS on Apple silicon — whose WSL engine also serves the Linux classes cheaply, and an android class (WSA where present, the Android Emulator durably) covers mobile-app work; a cloud runner leases tier-2 desktops and macOS VMs from the user's own Macs.

How it relates to the queue's rules — deliberately different on each point:

• The issuing device IS a legitimate target. Invariant §1.1 ("Covalence never selects the issuing device") is about where work runs; here the session stays put, and the Mac in front of you serving a VM to a cloud session is precisely the feature.

• Leasing itself changes no ownership. The unit is a fenced, idle-reaped lease on the serving member, not a session transfer — a dead borrower can never pin a VM slot, and the serving member's own caps (nucleic.macvm.maxConcurrent, container caps) plus a per-class Settings opt-in govern what it will serve. (When co-location does move a leased session — last bullet below — that is the queue's ordinary rebalance acting on its ordinary rules, not the lease.)

• Placement reuses the scorer, plus two preferences. Candidates come from an additive environmentProfile on the RunnerPresence card (the presence-field extension §6 anticipated: classes served, native/emulated arches, per-class capacity); the broker prefers native arch over emulation and LAN/tailnet paths over the relay (computer-use frames are heavy, and the live workspace mount pays a round trip per cold read), then falls back to the same least-loaded discipline as dispatch.

• Wire and approvals follow the house recipe. Capability-gated additive env* verbs (WireCapabilities.canServeEnvironments), E2EE end to end, and every environment tool call still surfaces on the owning session's approval pipeline before touching the serving member.

• The workspace rides the lease as a live mount. The environment sees the session's real worktree with local-virtiofs semantics — an owner-answered mesh file service the serving member bridges into the guest (FUSE → virtiofs / bind mount) — not a snapshot copied in and synced back.

• Bad paths move the agent, not the bytes — and that IS this queue's job. A lease's agent↔environment leg is thick (file service, exec streams, frames); the user's steering leg is thin (the same event stream a phone follows on cellular). When the owner's path to the serving member is poor — the hotspot-tethered laptop borrowing the desk Mac's VM — the §3 sweep gains a co-location term (fed by a measured peer-RTT path signal on the presence card, another of the additive fields §6 anticipated): a Covalence-managed session with an active or pending lease rebalances toward the member with the best path to the server, ideally the serving member itself, where the environment becomes local and the mesh-environment machinery drops out entirely — sessions and transcripts already replicate across the mesh, so the move is the ordinary between-turn 2PC transfer picking the session up where it left off. Every §3 gate applies (never mid-turn, typing veto, cooldown). A Here session is not auto-moved by default — "Here" is an explicit run-target choice — the UI offers the move instead, via the existing Move-to-Mac flow. An opt-in toggle (Settings ▸ Covalence ▸ "Move 'Here' sessions for a clear win", nucleic.covalence.hereAutoMove, default off) extends the sweep to Here sessions on a clear purpose only — an active or pending lease whose co-location gain crosses the path-quality threshold (reducing virtualization/container latency), never the ordinary load margin — under every §3 gate, with the move surfaced in the transcript.

Full design — broker seam, lease shape, the mesh file service, computer-use in v1, honesty notes — lives in COVALENCE_RUNNER.md §4.1 (work-queue item §0.2/9).