Files
nucleic/docs/GROK_ADAPTER.md
T

375 lines
24 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Nucleic — Grok Build Adapter (Implementation Plan, v0)
Plan for adding **Grok Build** (xAI's terminal coding agent) as a third `AgentBackend`
alongside Claude Code and the (stubbed) Codex adapters. This is a planning doc, not a shipped
adapter; it follows the same structure and rigor as [ADAPTERS](ADAPTERS.md) and is meant to be
the spec the implementation works against.
**Confidence key:** ✅ documented/stable · 🟡 observed/SDK-derived, pin before shipping ·
🔴 inferred, **a live-capture spike must record a real sample** (the M0 methodology —
see [M0_RESULTS](M0_RESULTS.md)). Treat 🔴/🟡 as the validation backlog, not as fact.
---
> ## ⚑ Update (2026-06-21): shipped on ACP, not streaming-json + hooks
>
> This doc was written for `grok -p --output-format streaming-json` with a generated **PreToolUse
> hook** bridging approvals over HTTP (Tier A, below). **That design has been replaced.** Grok now
> ships [`agent` mode — `grok agent stdio` — which is **ACP** (the Agent Client Protocol): standards
> JSON-RPC 2.0 over stdio, baked into the binary]. So the Grok backend is now a near-twin of the
> **Codex app-server** adapter, not of Claude:
>
> - Transport: the shared [`JSONRPCConnection`](../Sources/NucleicCore/JSONRPCConnection.swift)
> (the same one Codex uses; ACP sets `includeVersionHeader: true`). No more streaming-json decoder.
> - Approvals: native `session/request_permission` server→client *requests* — **the entire
> `/grok-hook` HTTP bridge, the generated `.grok/` PreToolUse script, and the inferred
> streaming-json event vocab are gone.** No more 🔴-inferred wire format here; ACP is a published
> schema.
> - Implemented in [`Sources/NucleicCore/Grok/`](../Sources/NucleicCore/Grok): `GrokACPBackend`,
> `GrokACPDecoder` (`session/update` → `AgentEvent`), `GrokACPDecisionMapping` (`Decision` → ACP
> permission outcome). Tests: `GrokACPDecoderTests`, `FakeGrokContractTests` (a full ACP
> round-trip against the `fake-grok` stub); fixtures under `fixtures/grok/acp/`.
>
> The §3 mapping table, §4 approval tier, and §6.1 status below are updated to ACP; the rest of the
> doc is retained as historical context for the original streaming-json plan.
---
## 0. Verdict — clean fit
The codebase was designed for this. There is a documented
[`AgentBackend`](../Sources/NucleicCore/Backend.swift) protocol, a `BackendID` enum that already
anticipates multiple agents
([`claudeCode`/`codex`/`codexExec`](../Sources/NucleicProtocol/CoreIdentifiers.swift)), a
per-session `BackendFactory` seam in `AppStore`, per-project `defaultBackend`
([Project.swift](../Sources/NucleicCore/Project.swift)), and an [ADAPTERS](ADAPTERS.md) doc whose
thesis is "wildly different wire formats converge by design." Codex is already stubbed in the enum
and Settings as "coming soon"; Grok follows the same paved path.
Grok Build is unusually convenient because it deliberately mirrors Claude Code's CLI surface
(`grok -p --output-format streaming-json`, `--permission-mode default|plan|acceptEdits|…`, MCP
servers, PreToolUse hooks, reads `CLAUDE.md` natively, `--session` resume). The *shape* of the
integration is therefore nearly identical to [`ClaudeCodeBackend`](../Sources/NucleicCore/Claude/ClaudeCodeBackend.swift).
The differences are in the wire details — and those details are the validation backlog (§2).
The net-new code is small and localized: **one stream decoder + one backend actor + one approval
endpoint + UI/catalog/factory glue.** The hard infrastructure (process host, approval suspension,
conflict locks, containers, sync, persistence, transcripts) is backend-agnostic and reused
unchanged.
---
## 1. What Grok Build provides (confirmed from xAI + community docs)
- ✅ Headless: `grok -p "<prompt>" --output-format streaming-json` — runs once, prints, exits
nonzero on refusal/tool failure.
- ✅ Permission modes: `--permission-mode {default, dontAsk, acceptEdits, bypassPermissions,
plan}` — the same vocabulary as Claude Code; `[ui] permission_mode = ask|always-approve` in
`~/.grok/config.toml` for persistent use.
- ✅ **PreToolUse hooks** (`.grok/hooks/pre_tool.sh`, `.grok/hooks.json`): execute at agent
lifecycle events, **receive event JSON on stdin and return JSON on stdout, and can block/deny**
a tool call — even when auto-approve is on. This is the interactive-approval seam.
- ✅ Tool-gating order: PreToolUse hooks → policy rules (deny > ask > allow) → built-in fast
paths → prompt policy.
- ✅ MCP server support (`/mcps`, config) so the agent can call internal APIs, not just shell.
- ✅ Auth: `XAI_API_KEY=xai-…` env, or `grok auth login` (browser OAuth for SuperGrok); config
under `~/.grok` (`config.toml`).
- ✅ Resume: conversations persist; `--session latest` / `--session <id>`.
- ✅ Reads existing `CLAUDE.md` (and `AGENTS.md`) project context natively — migration-friendly.
- 🟡 streaming-json event vocab is **xAI's own**, observed as `session.start`, `model.thinking`,
`tool.call`, `tool.result`, `model.message`, `session.end` — **not** Anthropic's
`{"type":"assistant","message":{content:[…]}}` envelope. Exact per-event fields must be pinned.
Sources (captured 2026-06-20): xAI "Introducing Grok Build", Mervin Praison "Grok Build CLI",
aimadetools "Grok Build Complete Guide", ContentBuffer "Headless Agentic Coding", xAI Enterprise
Docs.
---
## 2. Validation backlog (🔴 — a live capture must settle these before adapter code)
The same five unknowns that the M0 spike settled for Claude, restated for Grok:
1. **streaming-json line schemas** — the exact field names inside each event
(`session.start`/`model.message`/`model.thinking`/`tool.call`/`tool.result`/`session.end`),
tool-call id + input shape, file-change signal, token-usage and cost reporting, and whether
partial/delta events exist (the equivalent of Claude's `--include-partial-messages`).
2. **PreToolUse hook wire contract** — the exact JSON Grok writes to the hook's **stdin**, and
the exact JSON it expects back on **stdout** to allow / deny / (possibly) modify a tool call.
*This is the single most important unknown.* It decides whether we get full interactive
approvals (Tier A, §4) or fall back to a fixed policy (Tier B).
3. **stdin follow-ups** — whether headless `grok -p` accepts streamed follow-up user turns on
stdin (Claude's `--input-format stream-json`) or is strictly single-shot-per-process. (Claude
is run single-shot-per-turn today and resumed via `--resume`; Grok will likely use the same
model regardless.)
4. **resume flag** — exact form (`--session <id>` vs `--resume`/`--continue`), and how the
resumable session id surfaces in the stream (so we can capture it into
`SessionStarted.backendSessionID`).
5. **sandbox auth seeding** — where `grok auth login` stores credentials under `~/.grok`, so a
containerized run can export them (the Grok analog of Claude's Keychain → `.credentials.json`
dance in [`SessionController.containerSpec`](../Sources/NucleicCore/SessionController.swift)).
**Deliverable of the spike:** golden native streams under `fixtures/grok/<version>/` (see §8) and
a §6 addendum here that retags each item ✅ with verbatim samples, plus the Tier A/B decision.
---
## 3. Architecture mapping
Everything converges on the same `AgentBackend` → `AgentEvent` stream →
`ApprovalCoordinator.await` seam. **As shipped (ACP), the Grok adapter is a sibling of
`CodexAppServerBackend`** (both are JSON-RPC stdio backends whose approvals arrive as
server→client requests):
| Concern | Codex app-server (today) | Grok ACP (shipped) |
| --- | --- | --- |
| Invocation | `codex app-server` | `grok agent stdio` |
| Transport | `JSONRPCConnection` (header omitted) | `JSONRPCConnection` (`includeVersionHeader: true`) |
| Stream decode | `CodexAppServerDecoder` (thread/turn/item) | **new** `GrokACPDecoder` (`session/update` variants) → same `AgentEvent` kinds |
| Approval bridge | `…/requestApproval` request → `ApprovalCoordinator` | `session/request_permission` request → same `ApprovalCoordinator` |
| Approval reply | `{decision}` | `{outcome:{selected,optionId}}` (`GrokACPDecisionMapping`) |
| Resume | `thread/resume {threadId}` | `session/load {sessionId}` (iff `agentCapabilities.loadSession`) |
| Interrupt | `turn/interrupt {threadId,turnId}` | `session/cancel {sessionId}` |
| Auth | inherited | `authenticate {methodId}` if `authMethods` non-empty; else `grok` login (`~/.grok`) / `XAI_API_KEY` |
> _Historical (original plan): Grok was to be a sibling of `ClaudeCodeBackend` —
> `grok -p --output-format streaming-json --permission-mode default` decoded by a
> `GrokStreamDecoder`, with a generated PreToolUse hook POSTing to the in-process MCP server.
> ACP superseded this; see the update note at the top._
**Reused unchanged:** [`ProcessHost`](../Sources/NucleicCore/ProcessHost.swift),
[`ApprovalCoordinator`](../Sources/NucleicCore/ApprovalCoordinator.swift), the HTTP host inside
[`MCPApprovalServer`](../Sources/NucleicCore/Claude/MCPApprovalServer.swift),
[`RiskClassifier`](../Sources/NucleicCore/RiskClassifier.swift),
[`ContainerManager`](../Sources/NucleicCore/Container), `WorktreeManager`, the conflict/lock
system, transcripts, sync, and persistence. The new code is the decoder, the backend actor, the
approval-callback endpoint, and the catalog/UI/factory glue.
---
## 4. The pivotal decision — approval tier
> **Shipped resolution (ACP): neither tier — native permission requests.** The hook-vs-fixed-policy
> tradeoff below was moot once Grok shipped ACP: approvals arrive as native
> `session/request_permission` server→client *requests*, answered with a `{outcome:{selected,
> optionId}}` result (`GrokACPDecisionMapping`) — full interactive approvals
> (`interactiveApprovals: true`) with no hook and no HTTP bridge. `canModifyToolInput` is `false`
> (ACP's permission outcome can't rewrite tool input). The tier analysis below is historical.
This determines scope. **Target the interactive hook bridge (Tier A), with a non-interactive
fallback (Tier B) for v1 if the spike forces it.**
- **Tier A — interactive (target).** A generated PreToolUse hook posts each gated tool call to
Nucleic's existing per-session, bearer-token-gated local HTTP server. The handler suspends on
`ApprovalCoordinator.await` (emitting `AgentEvent.approvalRequested`), and the hook script
emits the allow/deny JSON Grok expects on stdout. This gives Grok sessions the *same* Mac/iPhone
approval UX as Claude → `interactiveApprovals: true`. Feasible because Grok hooks are explicitly
blocking and JSON-in/JSON-out (§1). Paired with `--permission-mode default`.
- **Tier B — non-interactive fallback.** If the hook contract cannot carry a synchronous human
allow/deny, run Grok like [`CodexExecAdapter`](ADAPTERS.md): `ApprovalPolicy.fixed` rules +
`--permission-mode acceptEdits`/`plan`, `interactiveApprovals: false`, fail-closed for anything
unmatched. Lower fidelity but ships, and matches a path the protocol already models. Add a
`BackendID.grokExec` if both tiers end up coexisting.
Everything downstream of `ApprovalCoordinator.await` is identical either way — the tier only
changes the plumbing that feeds it.
---
## 5. Phased implementation plan
### Phase 0 — Validation spike (de-risk before the adapter)
Mirror the proven M0 methodology. Install `grok`, authenticate, run real headless turns against a
throwaway repo. Extend [`nucleic-spike`](../Sources/nucleic-spike/Spike.swift) (or a `grok-spike`
target) to capture: a simple turn, a tool-approval turn, a multi-file-edit turn, an interrupt, and
a resume. Record golden native streams under `fixtures/grok/<version>/` and retag §2 with verbatim
samples. **Settle Tier A vs B here.**
### Phase 1 — Protocol & identifiers (tiny, backward-compatible)
- Add `case grok` (and `grokExec` if Tier B is needed) to
[`BackendID`](../Sources/NucleicProtocol/CoreIdentifiers.swift). Persistence already round-trips
`BackendID` by rawValue and falls back to `.claudeCode` on unknown
([GRDBMetadataStore](../Sources/NucleicCore/Persistence/GRDBMetadataStore.swift)), so old DBs and
the iOS client tolerate the new case for free.
- `AgentEvent.backend` and the sync wire
([WireMessages](../Sources/NucleicProtocol/Sync/WireMessages.swift)) carry it automatically — the
phone gets Grok-labeled sessions with no protocol change.
### Phase 2 — `GrokStreamDecoder`
- New `Sources/NucleicCore/Grok/GrokStreamDecoder.swift`, structurally like
[`ClaudeStreamDecoder`](../Sources/NucleicCore/Claude/ClaudeStreamDecoder.swift) but decoding
xAI's vocab → the existing `AgentEvent.Kind`s: `session.start``sessionStarted`,
`model.message``assistantText`, `model.thinking``thinking`,
`tool.call``toolCallStarted`/`toolCallCompleted` (+ synthesize `fileChange` from
`write_file`/edit tools, à la `inferredFileChange`), `tool.result``toolResult`,
`session.end``turnCompleted`/`runFinished` + `usage`.
- Lenient by contract: unknown lines → `.raw` (the CLI-drift early warning). Unit-tested against
the Phase 0 fixtures (decode → assert exact `AgentEvent[]`).
### Phase 3 — `GrokBuildBackend`
- New `Sources/NucleicCore/Grok/GrokBuildBackend.swift`, an actor mirroring `ClaudeCodeBackend`:
- `static let id = BackendID.grok`; `capabilities` set from spike findings
(`interactiveApprovals` per Tier; `nativeResume: true`; `emitsFileChangeEvents` per schema;
`sandboxModes` likely `[]` initially).
- `Configuration` with `executable: "grok"`, read-only tool pre-allow, single-shot-per-turn
stdin handling (the model Claude already uses).
- Build args: `-p --output-format streaming-json --permission-mode default` (no `--model` or
`--reasoning-effort` — Grok Build is one model with Auto reasoning), generated hook/MCP config
path, resume via `--resume <id>`. Reuse the same `ProcessHost`,
stderr-tail, optional raw capture, and exit-synthesis logic.
- **Approval endpoint (Tier A):** add a hook handler to `MCPApprovalServer` that accepts Grok's
hook POST and returns its allow/deny JSON, suspending on `ApprovalCoordinator` exactly like
`handleApprovalCall`. Generate the `.grok/hooks` script + config at run start, pointed at
`http://127.0.0.1:<port>` with the per-session bearer token (and the VM gateway host for
containerized runs, mirroring the Claude path).
- Reuse the same `ConflictCoordinator` arbitration in the approval path so Grok sessions
participate in the lock system + Nucleic Control autoship just like Claude.
### Phase 4 — Auth & env
- **Host-spawned (default):** Grok auth is inherited naturally — host `~/.grok` login or
`XAI_API_KEY` in the environment. Add a blank-key guard symmetric to the existing
`ANTHROPIC_API_KEY` purge in [`NucleicApp` init](../Sources/NucleicApp/NucleicApp.swift) only if
the spike shows a blank `XAI_API_KEY` causes the same 401-preference problem.
- **Sandboxed (defer for v1):** generalize
[`SessionController.containerSpec`](../Sources/NucleicCore/SessionController.swift) to seed a
Grok-home (`~/.grok`) and forward `XAI_API_KEY`/config-dir — the Grok analog of the claude-home
seeding. Reasonable to ship Grok **host-only first** and add container support in a follow-up.
Settings already has a per-agent auth model
([`ControlAuthMode`](../Sources/NucleicCore/Project.swift)) to extend.
### Phase 5 — Backend-aware model catalog
- [`ModelCatalog`](../Sources/NucleicApp/ModelCatalog.swift) is Claude-only and global. Refactor
to key models/efforts/context-windows by `BackendID` (or add a parallel `GrokModelCatalog`):
a single `grok-build` SKU with one "Auto" reasoning level. The home composer's model/effort pickers must
switch catalogs based on the selected backend. Grok uses `--permission-mode` modes rather than
Claude's `--effort`; the generic `RunSpec.effort`/`model` fields stay and each backend
translates appropriately.
### Phase 6 — Wiring & UI
- **Factory dispatch:** change the
[`backendFactory` closure](../Sources/NucleicApp/NucleicApp.swift) (and the AppStore call sites)
to switch on `session.backend``ClaudeCodeBackend` or `GrokBuildBackend`. Today it ignores
`session` and always returns Claude.
- **Agent picker:** `defaultBackend` exists on `Project` but has no selector yet. Add (a) a
project-settings backend picker, and (b) ideally a per-chat agent picker in the home composer.
`AppStore` already resolves `project.defaultBackend ?? .claudeCode` at session creation.
- **Settings:** replace the
[Codex "coming soon" placeholder](../Sources/NucleicApp/SettingsView.swift) pattern with a real
"Grok" section (default model, permission/auto mode, `XAI_API_KEY`/login status), mirroring the
Claude section.
- **Branding:** the lavender-accent gating (Nucleic Control only) is orthogonal; Grok sessions
render with the standard chrome plus a backend label/icon.
### Phase 7 — Testing
- **`fake-grok` stub:** add `Sources/fake-grok/` analog of
[`FakeClaude`](../Sources/fake-claude/FakeClaude.swift) that replays fixture streams and honors
hook callbacks, so backend tests run hermetically with no real `grok` binary (point
`Configuration.executable` at it, as Claude tests do).
- Decoder unit tests (fixtures → `AgentEvent[]`), backend integration tests
(run/approve/resume/interrupt), and an approval round-trip test asserting the exact hook-reply
bytes Grok accepts. Build/test with `swift test`.
### Phase 8 — Quota / polish (optional)
- [`SubscriptionUsage`](../Sources/NucleicCore/SubscriptionUsage.swift) /
[`QuotaIndicator`](../Sources/NucleicApp/QuotaIndicator.swift) are Claude-OAuth-specific. Gate
the indicator off for Grok sessions until/unless xAI exposes an equivalent usage endpoint.
---
## 6. Spike findings (to be filled in by Phase 0)
_Placeholder. After the live capture, retag each §2 item ✅ here with verbatim native samples and
record the Tier A/B decision, exactly as [M0_RESULTS](M0_RESULTS.md) did for Claude._
### 6.1 Implementation status — adapter landed ahead of the spike (🟡/🔴 pending validation)
> **Superseded by the ACP migration (2026-06-21).** Everything in this subsection describes the
> original streaming-json + PreToolUse-hook adapter, which has been **replaced** by the ACP backend
> (`GrokACPBackend`/`GrokACPDecoder`/`GrokACPDecisionMapping`; see the update note at the top of
> this doc and §3/§4). The 🔴-inferred streaming-json/hook unknowns below no longer apply — ACP is a
> published schema and the wire shapes are pinned to it. Retained as historical context.
The Tier-A adapter is implemented against the inferred schemas so the wiring, decoder, approval
bridge, and tests exist and pass hermetically; the §2 unknowns remain the validation backlog and
are **centralized so a real capture is a local re-pin**, not a rewrite:
- **Phase 1 — identifiers.** `BackendID.grok` added (`forModel("grok-*") → .grok`); persistence,
sync wire, and `StatusFeed`/`xAI` failover route it automatically.
- **Phase 2 — decoder.** [`GrokStreamDecoder`](../Sources/NucleicCore/Grok/GrokStreamDecoder.swift)
maps the inferred `session.*`/`model.*`/`tool.*` vocab → `AgentEvent`s, lenient (`.raw` on
drift). Golden fixtures under `fixtures/grok/0.1.0/`; unit tests in
`Tests/NucleicCoreTests/GrokStreamDecoderTests.swift`. **🔴 the per-event field names are
inferred** — re-record from a live capture and re-assert.
- **Phase 3 — backend.** [`GrokBuildBackend`](../Sources/NucleicCore/Grok/GrokBuildBackend.swift),
an actor mirroring `ClaudeCodeBackend`. **Tier A** approvals: a generated PreToolUse hook
([`GrokHookConfig`](../Sources/NucleicCore/Grok/GrokHookConfig.swift)) POSTs to the new
`MCPApprovalServer` `/grok-hook` route, which suspends on the same `ApprovalCoordinator`. Joins
the conflict-lock system like Claude. **Discovery is project-local `.grok/`** (✅ §1) — written
into the worktree, git-excluded, restored/removed at teardown — NOT a CLI flag (an earlier
`--hooks-config` guess made `grok` exit 2) and NOT a config-dir env (would relocate `~/.grok`
and break auth). **🔴 the hook stdin/stdout contract remains inferred** (Claude-Code-compatible);
re-pin after the spike.
- **Phase 4 — auth.** Blank-`XAI_API_KEY` purge added to `NucleicApp` init (host-only v1).
- **Phase 5 — catalog & effort.** A single `grok-build` SKU ("Grok Build") is offered; the picker
doubles as the backend selector. **Grok Build is one model with a single "Auto" reasoning mode**,
so the "Reasoning" menu offers only **Auto** (plus Orchestra below the divider, an orchestration
mode rather than a Grok level). The backend passes neither `--model` nor `--reasoning-effort`
the SKU/effort the UI carries are cosmetic for Grok. *(Superseded the earlier plan to drive
`--reasoning-effort none|low|medium|high`, which xAI has since collapsed to Auto.)*
- **Phase 6 — wiring.** Factory dispatches `.grok → GrokBuildBackend`; Settings names Grok Build.
- **Phase 7 — tests.** `fake-grok` stub replays a streaming-json fixture and performs the real
`/grok-hook` round-trip; `FakeGrokContractTests` covers allow + deny end-to-end.
**Resolved against the real CLI (`grok --help`, captured 2026-06-21):**
- **Invocation** — headless is `grok -p/--single "<prompt>"` (prints to stdout and exits);
`--output-format streaming-json` ✅ (`plain|json|streaming-json`).
- **Resume** — `-r/--resume [<id>]` ✅ (the `--session` guess was wrong and would have been
rejected; fixed).
- **Permission mode** — `--permission-mode` accepts `default|acceptEdits|auto|dontAsk|
bypassPermissions|plan` ✅; v1 uses `dontAsk` (headless auto-approve; the PreToolUse hook,
which runs first in the gating order, stays the human gate).
- **Effort** — Grok Build is a single model with one **"Auto"** reasoning mode (xAI picks the
depth itself), so the adapter passes no reasoning flag at all (and no `--model`). The "Reasoning"
menu offers only Auto; Orchestra is still offered as an orchestration mode. *(Earlier captures
showed `--effort`/`--reasoning-effort` flags; xAI has since collapsed Grok Build to Auto.)*
- **Hook discovery** — project-local `.grok/` (no flag); verify with `grok inspect`.
**Still open (a streaming-json capture settles these):** the exact `session.*`/`model.*`/`tool.*`
field names and the PreToolUse hook stdin/stdout contract (both decoded/emitted speculatively);
whether `dontAsk` actually lets the hook gate (the Tier-A assumption); the headless background
wait (`--background-wait-timeout` defaults to 600s — a turn that leaves a persistent monitor can
appear to hang; `--no-wait-for-background` is the escape hatch); real model IDs (`grok models`);
and sandbox auth seeding (deferred — host-only).
---
## 7. Effort & risk
- **Net-new code is small and localized:** one decoder, one backend actor, one approval endpoint,
plus catalog/UI/factory glue. The hard infrastructure is reused untouched.
- **Backward/forward compatible:** the only protocol change is one enum case, which old DBs and
the iOS client already tolerate.
- **Biggest risk = the PreToolUse hook contract (Tier A).** If the spike shows it can't carry a
synchronous human decision, fall back to Tier B (`ApprovalPolicy.fixed`, like CodexExec) — still
ships, lower fidelity.
- **Suggested first PR:** Phases 03 + 67, host-only, behind the per-project backend picker.
Sandbox (Phase 4 container seeding) and quota (Phase 8) as follow-ups.
---
## 8. Fixtures harness
Same shape as [ADAPTERS §4](ADAPTERS.md):
```
fixtures/grok/<version>/
01-simple-turn.jsonl + 01-simple-turn.events.json (golden AgentEvent[])
02-tool-approval.jsonl + 02…events.json + hook-call.json / hook-reply.json
03-file-change.jsonl + …
04-resume.jsonl + …
```
Decoder tests decode the recorded native stream → assert exact `AgentEvent[]` equality, and (for
approvals) assert the reply bytes the CLI accepts. Re-recording on a CLI upgrade is the
protocol-drift regression signal.