Files
nucleic/docs/MACOS_VM_PROVISIONING_REPAIR_PLAN.md
T
2026-07-25 20:54:52 -07:00

73 KiB
Raw Blame History

macOS VM base-image provisioning repair

Status: the three-pass sequence is wired, but the mandatory MDM pass does not complete on a live guest — see "Enrollment-pass repair" below, which is the active workstream. Opened: 2026-07-23

Objective

Make the macOS 27 golden-base build genuinely unattended: no unrecognized GUI permission sheets, no blind keystrokes into whichever dialog happens to have focus, and no base marked ready until a production-shaped clone can boot, mount its workspace, and execute through the in-guest agent.

The existing lightweight MDM is the policy plane for permissions macOS allows device management to grant. Host-side display capture and virtual HID remain the default computer-use path, so guest Screen Recording and Accessibility are not prerequisites for an ordinary usable base.

Findings at investigation start

The implementation checkpoints below record which of these findings have since been resolved.

1. MDM is implemented but disconnected

MacVMEngine+MDM.swift contains Mode A enrollment and Mode B profile-push helpers, while NucleicMDM contains a CA, enrollment profile, HTTPS server, command queue, and tests. The production base build never calls those helpers. It runs the shell provisioner and then answers the Network Volumes dialog with a separate OCR/click pass.

The MacVMBaseStatus.mdmEnrolled and profilesInstalled fields therefore describe planned state, not a readiness invariant. Recipe v7 can be stamped while both remain false.

2. Provisioning creates policy prompts before policy exists

The bootstrap enters through Apple's VirtioFS automount, then remounts the same directory at a custom path and executes the full provisioner there. The custom mount is treated as a network volume, so the build touches a protected resource before any Network Volumes policy is installed.

The later consent pass repairs only NucleicVMAgent access. It is exact-English OCR for Allow, cannot establish which dialog owns that button, and is fragile when more than one sheet is visible.

3. GUI fallback handling is not a state machine

When Command Line Tools are absent from the Software Update catalog, the guest launches xcode-select --install and writes a sentinel. While the sentinel exists, the host sends Return every three seconds. Return may accept the CLT sheet, an unrelated privacy sheet, or any other focused control. Provisioning must recognize the expected UI state or stop; it must never treat all dialogs as equivalent.

4. The MDM integration is still spike-grade

  • UAMDM approval opens Device Management and sends one Return; Install, authentication, and verification are placeholders.
  • The built-in FDA and notification profiles are not copied into the shipped app.
  • The CA, MDM configuration, device identity, and installed profile identifiers are not persisted per base.
  • Enrollment orchestration uses registry-based run(name:), but maintenance boots hold a direct MacVMInstance.
  • A drained command queue may contain errored commands, and enrollment returning false is not currently fatal.
  • CA trust, guest-to-host TLS reachability, initial no-APNs command drain, real mdmclient signatures, and headless UAMDM approval still need a live macOS 27 validation pass.

5. macOS 27 narrows the policy scope

Classic MDM profiles remain appropriate for Full Disk Access, Network Volumes, and targeted notification settings. Classic PPPC Accessibility grants are removed in macOS 27 in favor of the declarative com.apple.configuration.app-settings privacy model. Screen Recording cannot be silently granted.

This does not block the base: pixel computer use is host-side and needs no guest TCC. Semantic AX remains optional until the lightweight MDM supports Declarative Device Management.

Implementation plan

P0 — deterministic bootstrap

  • Package the built-in MDM policy profiles in Resources/macvm.
  • Add Network Volumes to the VM-agent PPPC policy.
  • Resolve packaged policy assets through one tested engine helper.
  • Stop blindly confirming the graphical CLT installer.
  • Keep the unattended path headless; if CLT cannot be installed headlessly, fail with an actionable diagnostic rather than wait behind an unknown sheet.
  • Do not execute the full provisioner from a custom network-volume mount before policy exists.
  • Record screenshots, recognized text, frontmost UI state, and TCC attribution when an unexpected modal blocks progress.

P1 — production Mode A enrollment

  • Split the base build into:
    1. declarative account creation and minimal exec-agent install through the automount;
    2. MDM enrollment and required policy installation;
    3. full toolchain provisioning after policy is active.
  • Refactor enrollment to operate on a direct MacVMInstance.
  • Persist the MDM CA, configuration, device identity, and profile identifiers beside the base metadata with credential-appropriate permissions.
  • Replace the UAMDM placeholder with a bounded, state-aware Device Management flow that enters the generated guest password only in the expected authentication sheet.
  • Treat enrollment false, command errors, missing profile identifiers, or failed functional probes as fatal.
  • Make re-provisioning update policy without attempting a second enrollment.
  • Require mdmEnrolled, required profiles, agent readiness, and a production-shaped workspace mount before stamping the provisioning recipe.
  • Remove MacVMEngine+ConsentGrant.swift's Network Volumes click path entirely. Base publication performs a bounded validation-only custom-mount touch after MDM policy installation.

P2 — live macOS 27 validation

  • Validate local-CA trust without a SecurityAgent prompt.
  • Validate guest-to-host gateway/TLS reachability and check whether the listener introduces host Local Network or incoming-connection prompts.
  • Validate enrollment's initial command drain without an MDM APNs push certificate.
  • Capture real mdmclient request signatures, then enable strict signature verification.
  • Verify reboot, clone, reprovision, and app-update behavior.
  • Test a non-English guest and reject any automation dependent on English-only button matching.

P3 — optional semantic AX policy

  • Implement Declarative Device Management enablement and declaration endpoints.
  • Deliver macOS 27 app-settings privacy defaults for Accessibility/PostEvent only when semantic AX is enabled.
  • Keep Screen Recording and semantic AX out of ordinary base readiness.

Release gate

A base is releasable only when a clean macOS 27 build proves:

  • no unrecognized GUI sheets from restore through shutdown;
  • required MDM enrollment and exact profile identifiers survive reboot;
  • the first production-shaped workspace mount succeeds without a prompt;
  • VM-agent FDA works without a Data Access Blocked notification;
  • Tips/BTM noise is suppressed without disabling arbitrary app notifications;
  • re-provisioning performs no duplicate enrollment;
  • a fresh clone boots, mounts, executes, and shuts down unattended.

Implementation checkpoint — bounded UAMDM approval

  • The single blind Return has been replaced with an OCR-driven state machine. It acts only on the exact Nucleic enrollment row, an exact Install control in Nucleic/Device Management context, or a SecurityAgent sheet containing both expected system-modification phrases, the expected account, an exact Password label, and an exact submit control.
  • Credentials are submitted at most once. Repeated, foreign, incomplete, or non-English screens receive no speculative input and eventually fail the bounded operation.
  • Failure writes a private Diagnostics/MDM bundle beside the base: the last JPEG, recognized text, lsappinfo front state, and the recent com.apple.TCC log tail. Password contents are never included.
  • Re-provisioning now branches on the guest's actual enrollment status: enrolled bases use the direct-instance policy push/verification path and never reopen the enrollment profile. An enrolled guest without mdm-state.json fails as orphaned before any replacement CA can be created.
  • The state labels and field geometry still require the P2 macOS 27 live-validation pass. The production path is enabled fail-closed: an unrecognized screen receives no input and blocks recipe publication with a diagnostic rather than falling back to a permissive action.

Implementation checkpoint — persisted Mode A state

  • mdm-state.json now retains a versioned CA, configuration, device certificate/private key, PKCS#12 password, and the profile identifiers verified in the guest. It is mode 0600, remains in the golden-base directory, and is deliberately excluded by the explicit clone copier.
  • Mode A accepts a direct maintenance MacVMInstance and uses the existing bounded vsock/nash exec channel instead of requiring the base to appear in the ordinary session registry.
  • A drained MDM queue is no longer sufficient: every UUID from the current policy batch must report Acknowledged, enrollment must report User Approved, and the guest's XML profile inventory must contain the exact top-level identifiers before they are persisted.
  • The production provisioning call site is enabled unconditionally in recipe v8. Its remaining uncertainty is confined to the P2 live macOS 27 checks below; failures stop publication and retain diagnostics rather than falling back to speculative input or optimistic status.

Implementation checkpoint — production Mode A activation

  • Recipe v8 runs three ordered writable-base boots: (1) declarative account creation plus only nash, durable sudo, auto-login, and NucleicVMAgent; (2) mandatory MDM enrollment/policy verification; (3) the complete toolchain, apps, packages, and optional SIP-off semantic-AX work. No protected custom mount or heavyweight provisioning occurs before policy exists.
  • The Mode A boot attaches only the nucleic-provision automount (recreateHostPath: false), so it can stage enrollment artifacts without touching the protected custom-mount path before MDM policy.
  • The listener bind address is discovered from the running guest's default route. The response must be exactly one ordinary, non-loopback IPv4 address; whitespace-separated values, shell fragments, unspecified, loopback, multicast, malformed, and IPv6 values fail closed.
  • The pass uses the direct MacVMInstance to enroll a clean base or update an already-enrolled base, persists only verified state, and then shuts the writable base down cleanly.
  • Before queuing policy, the host derives the designated requirement from the exact staged agent and rewrites both PPPC service entries. Release Developer ID and ad-hoc development builds therefore authorize the binary actually installed; packaging now fails if the mandatory agent cannot build.
  • Base publication no longer invokes the interactive Network Volumes Allow-click fallback. Its final custom-mount touch is validation-only: if the MDM PPPC payload is acknowledged but non-functional, the prompt remains unanswered, the probe fails boundedly, and the recipe stamp is withheld.
  • The recipe cannot be stamped until MDM reports User Approved, every built-in policy command is acknowledged, the guest inventory contains both exact profile identifiers, metadata records the account/agent/toolchain/MDM invariants, and the production-shaped workspace/agent probe passes.

Implementation checkpoint — first live provisioning retry

  • The first live retry exposed an unexplained Shutdown NOW immediately after the visible bootstrap. That message is the intended phase-boundary shutdown; the actual failure occurred earlier and was hidden because the guest log lived only in a temporary staging directory.
  • Minimal-agent launch readiness now uses the final launchctl print state. A concurrent automatic /Library/LaunchAgents load can make an explicit launchctl bootstrap return nonzero even though the service is healthy; that benign race no longer rejects the base.
  • The new preflight had also promoted sysadminctl -autologin set from the old provisioner's best-effort refresh into a fatal command. Declarative macOS 27 first boot already requests auto-login, and sysadminctl may return nonzero when reapplying existing state. It is best-effort again; the following boot's bounded agent probe remains the authoritative readiness gate.
  • The captured live error showed the next synchronous blocker: the packaged app could not derive the staged agent's designated requirement by spawning and scraping /usr/bin/codesign, so Mode A aborted before its VM boot. Requirement extraction now uses Security.framework directly (SecStaticCodeCreateWithPathSecCodeCopyDesignatedRequirementSecRequirementCopyString), eliminating the subprocess and stderr-format dependency.
  • Both minimal and full passes label their shutdown outcome, include a bounded guest-log tail in the surfaced error, and copy STATUS plus provision.log with private permissions into Diagnostics/Provisioning before deleting the staging directory. No password or staged binary is retained.
  • A second macOS 27 live run is still required to establish the next release-gate result. The focused macOS Swift test invocation was requested but not approved; materialized bootstrap scripts pass bash -n in the Linux build environment.

Verification notes

  • The two existing policy files and generated minimal-agent LaunchAgent parse as plists.
  • scripts/provision-macos-guest.sh, scripts/build-macos-base.sh, scripts/package-app.sh, and the materialized minimal-agent bootstrap pass bash -n.
  • Focused unit coverage now pins the three-pass boundary, PPPC requirement rewriting, gateway parser, UAMDM classifier, persisted-status/recipe gate, clone admission, and production-shaped mount probe.
  • The Swift suites still require an approved macOS toolchain run. The MDM protocol's synthetic TLS drain does not substitute for the P2 real macOS guest/mdmclient release gate.

Enrollment-pass repair

Opened: 2026-07-23 (second investigation) Symptom: OS setup completes and the base is left account-provisioned with a reachable agent, but the toolchain pass never starts. Phase 2 (mdmPolicyPass) throws, and its catch in provisionAndFinalize is fail-closed, so phase 3 is never reached.

Findings

Grouped by whether they are unambiguous from the source alone or depend on live macOS behavior that has not yet been reproduced here.

Blockers in the enrollment pass

  1. mdm-enroll.sh runs open as root. MacVMEngine+MDM.swift invokes the whole script with sudo /bin/bash, and MDMServerController.enrollScript() ends with open "$PROFILE" under set -euo pipefail. Either open returns non-zero and set -e aborts before MDM_ENROLL_STAGED is echoed, or it succeeds and registers the downloaded profile for root, so System Settings running as the agent account has nothing to approve. Only the /etc/hosts write and security add-trusted-cert need root.
  2. The Device Management pane is opened with a pre-Ventura identifier. x-apple.systempreferences:com.apple.preferences.configurationprofiles is the legacy .prefPane id; System Settings on macOS 13+ uses com.apple.Profiles-Settings.extension. An unrecognized anchor leaves Settings on its default pane, so the classifier never sees the enrollment row.
  3. The approval loop's budget is ~20 seconds. for _ in 0..<16 with a 1.25 s sleep has to cover System Settings cold launch, locating the row, the Install sheet, the authentication sheet, and the profile actually installing plus mdmclient's Authenticate/TokenUpdate. approvalTimeout (120 s) is not applied to this loop at all — only to waitForDrain.
  4. The MDM pass never brings the surface on-screen. MacVMSurfaceHost documents that on-screen rendering stays live "even when programmatic framebuffer captures come back blank (a known recent- guest quirk)". The minimal and full passes both honor baseObserverRequested; mdmPolicyPass only calls attach. Blank captures burn the entire (already too short) budget and surface as "stopped on an unrecognized screen" with an empty recognizedText list.
  5. The authentication classifier requires text the sheet does not render. uamdmApprovalAction requires contains(expectedUser) — the account short name — in addition to both SecurityAgent phrases. The modern sheet for an admin console user shows the icon, the phrases, and a Password field, with no account name. .authenticate is therefore never returned and the password is never typed.
  6. .installed can fire as a false positive. A stray "Installed" anywhere in the capture, next to the enrollment profile's display name, terminates the approval loop as successful. The drain then times out at 120 s with no indication that the GUI step silently no-op'd.

Protocol bugs that would break enrollment even with a working GUI

  1. The profile advertises a capability the server does not implement. MDMEnrollmentProfile sets ServerCapabilities: ["com.apple.mdm.per-user-connections"], which tells macOS to open the per-user channel and send user-scoped check-ins.
  2. Any check-in message type outside the hardcoded three returns HTTP 400. MDMCheckInMessage.parse throws unsupported for anything but Authenticate/TokenUpdate/ CheckOut, and handleCheckIn maps that to 400. Real enrollment also sends SetBootstrapToken/GetBootstrapToken and — because of (7) — UserAuthenticate. A 400 on the check-in channel is a hard enrollment failure.
  3. Every certificate the CA issues carries serial 2. MDMCertificateAuthority.nextSerial is a let on a struct and never advances. The device identity and the TLS server leaf share issuer+serial, and issueServerCertificate runs on every NucleicMDMServer.start() with a fresh key but the same serial, each pushed through SecPKCS12Import. Keychain certificate uniqueness is (type, issuer, serial), so a repeat run can fail the import and throw before the pass ever touches the guest. The existing testUniqueSerials documents the collision rather than catching it.

Correctness and gating

  1. mdmPolicyPass has no login recovery and misattributes its failure. The agent's LaunchAgent is LimitLoadToSessionType: Aqua, so a guest that did not auto-login never loads it; the minimal pass deliberately treats sysadminctl -autologin set as best-effort. The resulting error is MacVMError.appInstallFailed("the base image's in-guest agent never answered…"), which reads as an app-install failure inside "mandatory MDM enrollment/policy verification failed".
  2. The profile inventory check can produce a false negative. missingInstalledProfileIdentifiers drains profiles show -output stdout-xml through drainLines, whose 64 KB cap silently drops later lines with no truncation marker.
  3. mdm-state.json is written before enrollment is proven. loadOrCreateMDMController persists a fresh CA immediately, so every failed attempt leaves a file that satisfies the stamp gate's fileExists check.
  4. Enroll-step stdout is discarded from the failure message — the enroll script writes all of its progress there, and only stderr.suffix(400) is surfaced.
  5. defer { Task { await server.stop() } } detaches listener teardown from the function's lifetime, so the port can still be bound when a retry starts.
  6. exact("Install…") expects U+2026; Vision commonly returns three periods.

Not a defect: lowest(_:) is correct. Vision's boundingBox is bottom-left origin, so the visually lowest box has the smallest midY, which is what min { $0.midY < $1.midY } returns.

Confidence

(3), (4), (6), (9), (11), (12), (13), (14) are unambiguous from the source. (1), (2), (5), (8) are read off the code plus documented macOS behavior and have not been reproduced against a live guest — Workstream 0 exists to make the next run settle them with evidence.

Sequencing rationale

Live macOS 27 runs are the scarce resource: a full rebuild is ~40 minutes and currently yields one bit of information. Observability therefore lands first, so one run tells us everything. The GUI and protocol fixes then land together rather than serially — a working GUI flow still ends in a 400 mid-handshake without Workstream 2, and the protocol is unreachable without Workstream 1.

Workstream 0 — make the next run diagnosable

  • 0.1 Persist the MDM protocol transcript beside the base (Diagnostics/MDM/): method, path, MessageType/Status, CommandUUID, response code, and the leading bytes of any body that fails to parse. Distinguishes "device never connected" from "device connected and we 400'd it".
  • 0.2 Capture every approval-loop iteration (screenshot + recognized text), not only the last frame, so a blank framebuffer is visually obvious and a wording mismatch is readable.
  • 0.3 Include the guest enroll step's stdout in its failure message.
  • 0.4 Make awaitAgentReady(onInstance:) throw .agentUnavailable with a caller-supplied label instead of .appInstallFailed.
  • 0.5 Behind a debug default, keep the guest running with the observer visible when the MDM pass fails, so it can be inspected live instead of only post-mortem.
  • 0.6 A fast re-run entry point for the MDM pass alone — no restore, no phases 1/3. Turns a 40-minute cycle into roughly three minutes; the highest-leverage item in the plan.

Implementation checkpoint — Workstream 0

  • MacVMEngine.MDMDiagnosticsRecorder (MacVMEngine+MDMDiagnostics.swift) collects one attempt under <base>/Diagnostics/MDM/run-<label>-<token>/: a transcript.log interleaving the server's protocol lines with the orchestration's own step notes, per-iteration approval frames, and approval.json. It replaces the previous single-JPEG record, retains the newest five runs, and is best-effort throughout — losing evidence must never be why a base build fails.
  • Frame images are budgeted (40 per run) and written only when they can carry new information: the first frame, a changed classified state, a frame we acted on, and any frame that recognized nothing. Recognized text is always recorded. A capture the surface cannot produce at all is now its own frame rather than a silent continue.
  • The approval failure distinguishes "every capture recognized nothing" (the guest framebuffer is blank to the host surface) from "the screen rendered and we did not recognize it". Those have opposite fixes and previously produced the same sentence.
  • NucleicMDMServer logs every request before interpreting it, and logs a bounded, single-line preview of any body that fails to parse. Whether the guest reached the listener at all was not recoverable from the old per-message logging, which only fired after a successful parse.
  • The MDM pass now honors baseObserverRequested like the other two provisioning boots; it was the only invisible one, and the one most likely to need watching.
  • MacVMSettings.holdBaseOnMDMFailure (defaults write … -bool YES) keeps a failed pass's guest running, on-screen, and still attached to its staged share. MacVMEngine.heldDiagnosticInstances retains the instance (its only owner) and pins baseIsBusy, so nothing clones a live-mutating disk.
  • Deviation from the plan as written: 0.6 landed as MacVMEngine.revalidateBaseMDMPolicy() / MacVMManager.revalidateBaseMDMPolicy() rather than as an extension to Sources/mdm-spike. The spike is a headless CLI with no AppKit MacVMSurfaceHost, so it cannot drive the UAMDM approval at all — the very step under repair. Running the loop inside the app exercises the identical Mode A path (same surface, same direct-instance exec channel, same verification). It deliberately does not stamp the provisioning recipe: the toolchain and readiness passes have not run, so success means "enrollment works", not "this base is publishable".
  • Unrelated pre-existing failure fixed while verifying: baseBundleKeepsMDMCredentialsHostSideBesideMetadata compared mdmStateURL.deletingLastPathComponent() (a directory URL) against a non-directory root URL. The product code was correct; the assertion now compares paths.
  • Verification: swift build clean across all targets; MacVM suites 139/139 and NucleicMDMTests 34/34 pass, including new coverage for the body preview, the guest-output formatters, and the approval state labels.

First live run against Workstream 0 (2026-07-23)

The instrumented run settled finding (1) and §8.0 as the first blocker, exactly as the tooling was meant to. The transcript beside the base showed the server binding, then:

mdm-enroll.sh exit -1
  stdout:
    [mdm-enroll] discovering vmnet gateway…
    [mdm-enroll] pinned nucleic-mdm.host -> 192.168.64.1
  stderr:
    … mac_vm_exec command exceeded 1m with no exit and was terminated …

No check-in ever reached the server. The script wedged on the statement after the /etc/hosts pin — security add-trusted-cert into the System keychain — for the full 60 s exec timeout. On macOS 27 that command modifies admin trust settings, which raises a SecurityAgent authorization prompt even when run as root; with nothing to answer it, security blocks forever. This is docs/MACOS_VM_MDM.md §8.0 confirmed, and it is why open (and therefore any enrollment traffic) never ran.

First fix attempt (1.1, superseded in part): ran the enroll as the console user with per-step sudo and tried to make the trust step headless by granting com.apple.trust-settings.admin via security authorizationdb write … allow.

Second live run — the authorization override does not work (2026-07-23)

The next run got materially further: auto-login took (console user "Nucleic Agent"), the diagnostic monitor showed the desktop, and the run reached the CA-trust step — but the SecurityAgent sheet "You are making changes to the System Certificate Trust Settings. Enter your password to continue with 'security'." appeared anyway. The authorizationdb write … allow override did not suppress it on macOS 27. So the prompt must be answered, not avoided (docs/MACOS_VM_MDM.md §8.0's stated fallback). This required three coupled changes, since the surface loop only runs after the enroll returns and the flow actually has two password sheets:

  • Enroll runs the trust step detached. The script now pins /etc/hosts synchronously, then backgrounds security add-trusted-cert + open (redirected to a share-local log so the exec channel's stdio closes) and returns MDM_ENROLL_STAGED immediately, leaving the trust sheet up for the surface. Completion and the trust exit status land in an mdm-enroll.done sentinel the host reads for diagnostics. The authorizationdb dance is removed — it didn't work and only added noise.
  • The classifier recognizes the certificate-trust sheet as its own AuthKind.certificateTrust (distinct wording, "Update Settings" submit), separate from the profile-install sheet. The approval loop answers each kind once rather than latching after a single submit — the previous single latch would have left the second sheet unanswered.
  • The classifier no longer requires the account short name (finding 5): the live sheets show the full name "Nucleic Agent". The safety boundary is unchanged — a recognized intent phrase plus a Password label plus a submit control, all in one capture, and at most one submit per kind.

Folded in while here, because the trust step's latency makes the next stages reachable in the same run: the approval loop is now time-bounded by approvalTimeout (raised 120 → 300 s) instead of 16 iterations (finding 3), and it cycles Device Management pane candidates — the macOS 27 com.apple.Profiles-Settings.extension first, then privacy, then the legacy id (finding 2) — when nothing actionable is on screen, recording which one surfaced the profile.

Verified: the real emitted enroll script passes bash -n (unprivileged open, detached brace group); build clean; MacVM 140/140 and NucleicMDM 35/35, with new classifier coverage for the trust sheet, the account-name-independent install sheet, and the two distinct auth kinds. The surface actually answering these two prompts still needs a bootable guest — the next run.

Third live run — OCR line-wrapping defeated the classifier (2026-07-23)

The detached-enroll change worked (mdm-enroll.sh exit 0), the pane candidates cycled, and — the diagnostic bundle's final screenshot proved it — the certificate-trust sheet was on screen the whole time. Yet all 153 frames classified unknown ("answered: no sheets"), captures were non-blank (~420 KB), and the server logged no check-ins. The recognized text showed why: Vision emits one observation per rendered line, so the sheet's sentence arrived as three separate lines — "You are making changes to the" / "System Certificate Trust" / "Settings." — and the classifier's per-line contains("Certificate Trust Settings") matched none of them.

Fixed: context phrases are now matched against all recognized lines joined in reading order, not line by line, so a wrapped sentence still matches. exact() stays per-line (it locates the click targets). The trust context simplified to "Certificate Trust" / "System Certificate" — fragments that survive on a single line regardless. A regression test pins the exact wrapped lines Vision returned on the guest, and a second test asserts a split sentence still classifies while still requiring a Password label + submit before it is actionable.

Anticipating the next stage (never reached before, so its wording is unconfirmed): the profile-install auth context was broadened to also accept "install a new configuration profile", since macOS may word that sheet differently from the trust sheet — the Password + submit pairing remains the safety gate. If the real install sheet still misses, the transcript will now carry its exact lines to pin.

Verified: build clean; MacVM (142) and NucleicMDM (35) suites green; the seven UAMDM classifier tests include the wrapped-line regressions. The flow reaching and completing the profile install is the next run.

Fourth live run — the password field was clicked off the dialog edge (2026-07-23)

The trust sheet was now recognized and answered ("answered: trust"), but 164 frames stayed on authenticate-trust with only one marked acted, and the final screenshot showed the sheet still up with the "Password" placeholder still visible — i.e. the field was empty; the password was never typed into it. The cause was geometric: uamdmPasswordFieldPoint clicked at box.maxX + 0.18, which assumed "Password" was a label with the field to its right. On the macOS 27 SecurityAgent sheet "Password" is the placeholder inside a full-width field, and maxX + 0.18 (≈ x 0.65) lands past the right edge of the narrow centered dialog (≈ x 0.440.56). So the field never focused, the password typed into empty space, and "Update Settings" submitted an empty field.

Fixed: the field is now clicked at the recognized "Password" box's center (clickPoint), which lands inside the field. A short settle pause precedes typing, and cmd+a goes first so a re-type replaces rather than appends. uamdmPasswordFieldPoint and its test are removed.

Hardened while here, because a 40-minute rebuild is too costly to lose to one flaky keystroke: auth is now answered up to maxAuthAttemptsPerKind (3) times per sheet, and only re-answered once the same sheet has persisted past reAuthDelaySeconds (6 s) — a correct answer dismisses the sheet within a second, so a persisting sheet is the signal that the last attempt failed. The failure summary now reports per-kind attempt counts (trust×3).

Verified: build clean; MacVM (141) and NucleicMDM (35) green; the field-click test asserts the click stays inside the dialog. The trust answer landing, and the profile-install stage, are the next run.

Fifth live run — OCR corrupted the exact labels, prompting an architecture change (2026-07-23)

This run the trust sheet was on screen but classified unknown for all 123 frames again — this time because Vision mis-read the very labels the classifier keyed on: the field placeholder "Password" came back as "Peseword" and the button "Update Settings" as "Updale Settings". exact("Password") and exact("Update Settings") both missed, so the field and button could not be located even though the sheet was plainly visible. This is the fragility the exact-label approach always carried.

Two responses — one architectural, one to the OCR itself:

Removed the CA-trust prompt entirely (the better way). Apple removed headless MDM-enrollment install in macOS 11, so the profile-install approval is genuinely unavoidable — but the trust prompt is not. The CA root is now delivered inside the enrollment profile as a com.apple.security.root payload (installed first, before the MDM payload's check-in TLS), so the single profile-Install approval establishes trust too. security add-trusted-cert — and its unsuppressible SecurityAgent sheet, the wall that stalled four straight runs — is gone. The enroll script is now trivial and synchronous (pin hosts, open profile); no backgrounding, sentinel, or marker race. This bets that a profile-delivered root installed at System scope with admin approval is trusted for the check-in TLS (the standard enterprise mechanism; the doc's §2.3 "not auto-trusted" caveat appears to describe iOS's manual-trust step, which macOS lacks). If it proves wrong, the server log will show the check-in TLS failing with no successful drain — a clean, unambiguous signal.

Made the one remaining prompt generic and OCR-noise-tolerant (the user's request to detect generic Apple password requests rather than brittle exact matches):

  • Detection keys on the stable phrase "Enter your password" plus a located password field — not on a sheet-specific intent phrase or account name. This handles any Apple authorization sheet.
  • The password field is found by fuzzy match (Levenshtein ≤ 2 to "password"), so "Peseword" and similar OCR slips still resolve; the lowest such box on screen is the field.
  • Submission is Return (the sheet's default button), so the button label is never OCR-matched — removing the "Updale Settings" class of failure entirely.
  • In the sealed base build the only password sheets are our own provisioning prompts and the only secret typed is the machine's own agent password, so a generic detector is safe here; a non-English locale still yields unknown rather than speculative input.

Folded in while the check-in path is now imminent: dropped the unimplemented ServerCapabilities (Workstream 2.1, finding 7) and made the check-in channel ack unmodeled MessageTypes with an empty 200 instead of 400 (Workstream 2.2, finding 8) — both would otherwise abort the handshake the profile install is about to trigger.

Verified: emitted profile lints clean with payload order root → pkcs12 → mdm and no ServerCapabilities; emitted enroll script passes bash -n with no trust/background steps; build clean; MacVM (141) and NucleicMDM (37) green, including fuzzy-field and OCR-noise regressions and the ack-don't-throw check-in test. Whether the profile-delivered root establishes trust, and the no-APNs drain, are the next run.

Sixth live run — profile-delivered root worked; clicks landed on the desktop (2026-07-23)

The profile-delivered root removed the trust prompt entirely: the run reached System Settings ▸ Device Management showing "Nucleic MDM Enrollment — Profile not installed. Double-click to review." in a proper window. The classifier recognized the row and double-clicked it seven times — but every click landed on the desktop, not the row, so the profile never opened.

Root cause was a coordinate-scaling bug in the computer-use surface, exposed by the diagnostic monitor being on-screen. MacVMComputerSurface.guestPoint built mouse-event locations in fixed 1920×1200 coordinates. Off-screen the HID window is a native 1920×1200, so that is correct and ordinary agent computer-use works. But when the monitor is shown, setWindowOnScreen resizes the window to 1280×800 and the VZVirtualMachineView autoresizes to fit — so a location computed for 1920×1200 maps to the wrong place in the smaller view, landing a row double-click on the desktop. Showing the monitor during the MDM approval (the previous checkpoint's change) is what surfaced it.

Fixed: guestPoint now scales the event location by the view's live bounds, so clicks land correctly whether the window is off-screen (1920×1200), shown (1280×800), or user-resized. It reduces to the old identity flip when the view is 1920×1200. Applied to click, move, and drag.

Also addressed from the same report:

  • First-boot timing. The bootstrap's first Terminal launch fired before the freshly-booted desktop was ready, wasting the attempt (then waiting a full retry interval). driveHIDBootstrap now holds an initial settle before the first launch — 20 s on the declarative first boot, 8 s otherwise — the Spotlight/cold-launch pauses in typeBootstrapLaunch are a touch longer, and the launch-retry interval dropped 40 → 25 s.
  • Sampling frequency. The bootstrap poll dropped 3 s → 1 s, and the approval loop now samples every ~0.4 s (0.9 s right after it acts, to let the UI react) instead of 1.25 s, so an element is acted on promptly after it appears. Pane cycling stops once the enrollment UI first appears, so a review/ install sheet that momentarily reads as unknown is not navigated away from. Each click's target coordinate is now logged to the transcript to confirm the mapping on the next run.

Verified: build clean; MacVM (141) and NucleicMDM (37) green. The profile-review → Install → password → drain sequence, with clicks now landing correctly, is the next run.

Seventh live run — the review sheet's button is "Enroll", and ServerCapabilities is required (2026-07-23)

The coordinate fix landed: the double-click now opened the profile-review sheet ("Are you sure you want to install this device profile?"). Two issues from there, both confirmed on the live guest — and the password-field autofill + Return-submit worked correctly once reached, so those fixes are proven.

  • The confirm button is "Enroll", not "Install". The macOS 27 review sheet uses "Enroll"; the classifier only knew "Install"/"Install…", so it never clicked. Added "Enroll" (chosen over Ignore/Cancel) as a confirm-button candidate, and "install this device profile" as an additional enrollment context. After a manual Enroll click, the flow proceeded exactly as designed.

  • ServerCapabilities is mandatory, not harmful — finding 7 was backwards. After Enroll, the install failed with "MDM payload is missing 'ServerCapabilities' key that indicates it supports a user channel." macOS 27 requires the key. Restored ServerCapabilities: ["com.apple.mdm.per-user-connections"] in the enrollment profile. Workstream 2.2 (ack unmodeled check-ins with an empty 200) is what makes advertising the per-user channel safe: macOS will send user-scoped check-ins we don't fully model, and they are acked so the device channel still drains the InstallProfile queue.

Verified: build clean; MacVM (142) and NucleicMDM (37) green, with the Enroll-button classifier test and the ServerCapabilities-present profile test; the emitted profile again carries ServerCapabilities. The install completing and the queue draining are the next run.

Eighth live run — the CA cannot bootstrap itself from inside the profile (2026-07-23)

ServerCapabilities cleared the previous failure and the install proceeded further, then failed with "The certificate could not be verified (authentication error)." This lands exactly where the in-profile com.apple.security.root payload was made the only trust mechanism, so that bet was wrong and docs/MACOS_VM_MDM.md §2.3 was right: the identity/check-in TLS is validated before the profile's own root payload takes effect, so the CA cannot bootstrap its own validation.

Fixed: security add-trusted-cert into the System keychain is restored, and it must run before the profile opens. Because it raises the unsuppressible SecurityAgent trust sheet (§8.0), it is again launched detached — the script pins /etc/hosts, backgrounds add-trusted-certopen, and returns MDM_ENROLL_STAGED immediately so the surface's approval loop can answer that sheet. This is now viable where it wasn't before: the generic password-sheet handler is proven on hardware (the seventh run's install prompt autofilled and submitted correctly). The in-profile root payload is kept as well — harmless, and it makes the trust durable in the sealed base — but the System-keychain trust is what lets the install succeed. The trust exit status and the detached block's log are read back into the transcript.

Two consequences handled: the pane-cycling latch no longer trips on a password sheet (the trust sheet appears before the profile is queued, so latching there would stop cycling before the profile row was ever surfaced — it now latches only on the profile row / review sheet / installed), and the auth budget rose 3 → 5 to cover two sheets with retry headroom.

Verified: emitted enroll script passes bash -n with the trust step at line 43 preceding open at 46, both inside the detached block; profile retains root + ServerCapabilities; build clean; MacVM (142) and NucleicMDM (37) green, with a test asserting trust precedes open. Two sheets answered in sequence, the install completing, and the no-APNs drain are the next run.

Ninth live run — duplicate certificate serials, and an unsigned profile (2026-07-23)

Both password sheets were answered (trust, then install) and the Enroll click landed, but installation still failed with "The certificate could not be verified (authentication error)". Two defects found.

Every issued certificate carried serial number 2 (finding 9, finally fixed). signLeaf stamped serialBytes(nextSerial), and nextSerial is a let on a value type that never advanced — so the device identity baked into the profile and the MDM server's TLS leaf shared (issuer, serialNumber). X.509 identifies a certificate by exactly that pair, and Apple's keychain/trust machinery keys certificate lookups on it, so the guest could not resolve which certificate was which. Leaves now get a fresh 16-byte random serial (RFC 5280 §4.1.2.2; positive, minimally encoded, unique across CA reloads in a way a persisted counter would not be). The root keeps serial 1. The old testUniqueSerials, which documented the collision, is replaced by one that issues nine leaves and asserts nine distinct serials.

The profile was unsigned ("Nucleic (Not Signed)"). Strictly, the error cannot be about a missing signature — an unsigned profile has no signing certificate to verify — so this is not self-evidently the cause. But it is the right thing to fix regardless: on modern macOS, certificate payloads inside an unsigned profile are installed without being granted trust, which is precisely the failure class here, and this profile exists to deliver a CA root plus a PKCS#12 identity that the check-in TLS then depends on. The enrollment profile is now CMS-signed (attached SignedData) by a CA-issued digitalSignature leaf — a CA the guest already trusts by the time it opens the profile. Signing is best-effort: it falls back to the unsigned plist rather than breaking enrollment, and unsignedEnrollmentProfile() remains for inspection.

Verified beyond the unit tests, since this changes the staged artifact's format: the emitted profile is DER id-signedData, openssl pkcs7 reports the signer as CN=Nucleic Profile Signing issued by CN=Nucleic MDM Root CA, and — the decisive check — macOS's own security cms -D recovers the plist, which lints clean and still contains root → pkcs12 → mdm with ServerCapabilities.

Also this round: the trust sentinel and the detached block's log are now recorded on the failure path too (previously lost, so whether add-trusted-cert actually succeeded was unknowable), the diagnostics capture the guest's ManagedClient/mdmclient/profiles log — where the real reason an install failed is recorded, since the on-screen alert says nothing actionable — and control clicks (enrollment row, Enroll) retry on a bounded time cadence instead of once per distinct OCR fingerprint, which previously granted extra clicks on incidental text noise while starving a genuinely stuck screen.

Build clean; MacVM (142) and NucleicMDM (39) green.

Tenth live run — sheet focus, and a shared retry budget (2026-07-24)

The certificate failure is gone: the serial fix plus profile signing carried the flow through the review sheet and the Enroll confirmation. It then stalled at the last step — "Device Management is trying to enroll you in a remote management (MDM) service" — with the password never submitted. The operator's step-by-step named both causes precisely.

A sheet that isn't key swallows the first click. The trust sheet "is not consistently foregrounded… must be manually clicked on to foreground, at which point it is automatically completed" — i.e. the typing was always correct, but the sheet was not key, and the first click on a non-key window is consumed activating it, so the caret never landed and the keystrokes went nowhere. The answer sequence now sends two clicks at the password field, 600 ms apart: the first foregrounds the sheet, the second places the caret. Both target the same field, so neither can stray onto another control.

The retry budget was global, not per sheet. maxAuthAttempts was one counter for the whole loop, so while the unfocused trust sheet was retried every 6 s it consumed all five attempts — leaving nothing for the final enrollment sheet, which is exactly where the run stalled. Attempts are now tracked per sheet, keyed by a slug derived from the sheet's own wording (certificate-trust, mdm-enroll, modify-settings, profile-install), so each gets its own budget and the failure summary reports them individually (certificate-trust×5, mdm-enroll×0 would have made this obvious at a glance).

Verified: build clean; MacVM (143) and NucleicMDM (39) green, including a test that pins the exact macOS 27 wording of each sheet to a distinct budget key and asserts two different sheets cannot collide onto one.

Eleventh live run — the GUI now completes; the failure is guest→server TLS (2026-07-24)

The focus and per-sheet-budget fixes worked. The transcript shows the whole GUI sequence running unattended for the first time: certificate-trust answered → pane opened → enrollment row double-clicked → Enroll clickedmdm-enroll sheet answered. Installation then failed with the same "The certificate could not be verified (authentication error)".

The decisive observation is in the transcript's absence: the host listener logs MDM server listening on 192.168.64.1:8443 and then never records a single request — in this run or any previous one. Guest→host reachability has therefore never actually been demonstrated (it is an open P2 item). Combined with the error appearing at the moment mdmclient first contacts the ServerURL, the likely failure is the check-in TLS handshake, not a payload import — the earlier assumption that this was about a certificate inside the profile does not fit the evidence.

Rather than guess a fourth time, the guest is now asked directly. Two probes run after the trust step has had its chance, on both the success and failure paths, and land in the transcript:

  • CA trust: does Nucleic MDM Root CA actually appear in the guest's System keychain? (The trust sentinel only reports add-trusted-cert's exit status, which can be 0 while the resulting trust setting is not what TLS evaluation consults.)
  • Reachability: curl the CheckInURL from the guest, once normally and once with --insecure. curl's exit code separates the possibilities precisely — 6 unresolved, 7 refused, 28 timeout, 35/60 TLS/certificate — and the verified-vs-insecure pair is the clean discriminator: both failing with 7/28 means the guest cannot reach the host at all (firewall / listener binding), whereas insecure succeeding while verified fails with 60 means it reaches the server fine and distrusts our CA. The count of /etc/hosts pins is reported alongside, to rule out the DNS pin.

Build clean; MacVM (143) and NucleicMDM (39) green.

Server-side debug trace

The request-level LogSink could not answer the question the enrollment failure poses, because the suspected failure produces no request: a guest whose TLS handshake fails connects, is rejected, and disconnects, and the old logging emitted nothing — the server looked identical whether it was healthy, unreachable, or refusing the client.

MDMDebugLog (append-only, size-rotating, 0600) now records the server's own view to Diagnostics/MDM/run-*/mdm-server-debug.log, and mdm-spike writes the same trace beside its staged artifacts so failures can be reproduced without a guest. Instrumented: listener lifecycle and bind failures; the issued TLS leaf and CA root (subject/issuer/serial/validity, to compare against the guest's trust store); sec_identity_t construction; every accepted connection with its endpoint and each state transition — which is where a handshake failure actually surfaces; raw byte counts with bounded previews; parsed request line, headers (the Mdm-Signature blob is reported by size only) and body; signature verification; check-in message fields including the topic actually presented versus the one configured; and queue movement on delivery and drain.

Validated end to end against the real server rather than assumed, driving it with an untrusted and then a CA-trusted client:

conn id=1 ACCEPTED from=127.0.0.1:51656 — awaiting TLS handshake
conn id=1 FAILED: -9831: unknown Cert Authority — likely TLS/certificate
...
conn id=2 READY (TLS handshake complete)
http id=2 REQUEST #1 PUT /checkin headers=5 bodyBytes=160
checkin id=2 Authenticate udid=TEST-UDID-1234 topic=- (expected topic=com.apple.mgmt.External.…)
http id=2 RESPONSE 200 OK

-9831 is errSSLUnknownRootCert — exactly what a guest that distrusts our CA produces, and exactly what was previously invisible. This also establishes that the server itself is healthy: given a client that trusts the CA, the full check-in round-trip returns 200.

The next run therefore reads as a decision tree: no ACCEPTED line at all ⇒ the guest never reached the host (routing/firewall/binding); ACCEPTED then FAILED: -9831 ⇒ the guest reached us but does not trust the CA, so add-trusted-cert did not take effect; a full request flow ⇒ TLS is fine and the fault is elsewhere.

Twelfth live run — the trace answered it: a pointer/click race (2026-07-24)

The server trace read: listener READY, LISTENING on 192.168.65.1:8443, and then no ACCEPTED line at all. The guest never opened a TCP connection, so this was never a TLS or certificate problem in the server — the third branch of the decision tree, resolved in one run instead of by inference. (Note the vmnet subnet had moved from 192.168.64.x to 192.168.65.x; the gateway discovery handled that correctly and bound the right address.)

The transcript supplied the cause. The keystroke-landing check reported "certificate-trust" attempt N: password landed in field=false five times: the CA-trust sheet never received the password, so the CA was never trusted, so the profile's certificate payloads could not be validated ("The certificate could not be verified"), so mdmclient never reached the check-in stage and never connected. One root cause, presenting as a certificate error three layers downstream.

Why the keystrokes missed was settled by measurement rather than argument. The diagnostic frame (1920×1200) cropped around the computed click point shows (880, 466) lands squarely inside the Password field — the coordinate was right. But the sheet's "Update Settings" button renders grey, not blue, and System Settings is frontmost: the click was activating the window behind the sheet.

Fixed: the pointer move is now posted as its own event, followed by a pointerSettleMillis (180 ms) settle, before the button-down. Previously the move and press went out in one batch, so the guest could process the button-down while its pointer was still at the previous location and deliver the click to whatever was under the old position. That also explains the intermittency the operator observed — "sometimes activated and completed if lucky" is exactly what a race looks like. The fix is in the surface, so it applies to every consumer of the computer-use path, not just enrollment.

Build clean; MacVM (143) green.

Removing the fixed-resolution assumption

Reviewing the click path surfaced a latent bug independent of the race above: clickPoint(forNormalizedBox:width:height:) multiplied Vision's normalized box through a hardcoded 1920×1200. Vision already returns resolution-independent coordinates, and MacVMSurfaceHost guarantees that captures and .click share one space — so the constant was both redundant and a silent failure waiting to happen: any change to the guest panel, the view size, or the capture normalization would misplace every click with no error anywhere.

Geometry is now measured from the frame being acted on. imageSize(ofJPEG:) reads the capture's true dimensions from its header (no decode), and clickPoint(forNormalizedBox:in:) scales against that, so the only place a resolution enters the system is derived from the same image the text was recognized in. The old constants remain solely as a fallback for an unreadable capture and as unit-test defaults.

The placeholder matcher was loosened in the same spirit: it accepts password/passcode/pin with a tolerance that scales to the word (about a quarter of its characters), rather than one fixed edit distance that was lenient on short words and strict on long ones. It still refuses unrelated short UI text — Wallpaper, Cancel, and Passkeys Access for Web Browsers are all rejected by test.

Tests pin the property that matters: the same normalized box maps proportionally onto 1920×1200, 3840×2400, and a non-16:10 2560×1440 panel, each axis scaling independently.

Build clean; MacVM (145) and NucleicMDM (39) green.

CA trust moved to the System Settings install flow

The CLI keychain-trust step was abandoned. security add-trusted-cert raises a SecurityAgent sheet owned by a root process with no GUI application: it never becomes key, and five consecutive automated attempts typed into nothing while the sheet sat inactive — the operator had to click it by hand every time. Rather than keep fighting for focus on a window macOS will not activate, the CA is now delivered the ordinary interactive way: as its own configuration profile, installed through System Settings before the enrollment profile. That routes the approval through the profile-install sheet, which belongs to System Settings and which the surface already drives successfully.

This is distinct from the earlier failed attempt to carry the root inside the enrollment profile. There the root was still being installed while the very payloads depending on it were validated — the chicken-and-egg §2.3 warned about. As a separate, earlier install it is fully in place and trusted before the enrollment profile is opened at all.

Shape of the change:

  • MDMEnrollmentProfile.makeRootProfile builds a standalone com.apple.security.root profile named "Nucleic MDM Root CA", staged as nucleic-mdm-ca.mobileconfig. Deliberately unsigned — signing it with the CA it delivers would be circular, since the guest cannot verify that signature until this profile is installed.
  • The enroll script now only pins /etc/hosts and opens the root profile. It is fully synchronous again: nothing in it prompts, so no backgrounding, no sentinel, no marker race.
  • Enrollment runs in two phases, each a full install driven through Device Management: root profile → confirm the CA is in the guest's System keychain → open the enrollment profile → install it → confirm enrollment. Reachability and CA-trust probes are recorded after each phase.
  • The approval loop is generalized from "approve enrollment" to installProfileViaComputerUse, keyed to a ProfileTarget (display name + an OCR-tolerant row keyword) and a completion predicate asked of the guest, not the screen. So the same proven loop installs either profile, and pixels never decide success.

Verified on the real emitted artifacts: all four staged; the enroll script passes bash -n, opens only $ROOT_PROFILE, and contains no CLI trust invocation; the root profile lints clean and carries exactly one com.apple.security.root payload holding the CA DER; the enrollment profile is still CMS-signed (id-signedData). Build clean; MacVM (145) and NucleicMDM (40) green, including new coverage that the root profile is unsigned and single-payload and that the enroll script cannot open the enrollment profile early.

Twelfth live run — the decision tree resolves: everything reduces to the trust prompt

The trace answered it on first use. The guest's own probes and the server's connection log agree:

conn id=1 ACCEPTED from=192.168.64.64:49152 — awaiting TLS handshake
conn id=1 FAILED: -9831: unknown Cert Authority
conn id=2 ACCEPTED → READY (TLS handshake complete)      # the --insecure probe
http id=2 REQUEST GET /checkin → RESPONSE 400
guest → MDM server reachability: verified: rc=60 "unable to get local issuer certificate"
enroll trust sentinel absent

So networking is entirely healthy — the guest reaches 192.168.64.1:8443, and the server answers correctly the moment certificate validation is bypassed. The single fault is that the guest does not trust the CA, because add-trusted-cert is still blocked on its SecurityAgent prompt (sentinel absent). Every downstream symptom chased over the last several runs — "the certificate could not be verified", the failed profile installation — is a consequence of that one prompt. This retires the competing hypotheses (payload certificates, profile signing, serial collisions, reachability) as causes of this failure, though the fixes made along the way were independently correct.

Measured against the raw 1920×1200 capture, the click target is right: the Password field spans x≈851910, y≈455478 (centre ≈ 880, 466) and the loop clicked (882, 467). So this is not a targeting error. It is a focus/input race — the sheet is answered "if lucky", the operator reports it neither foregrounds nor accepts typing once foregrounded, and five attempts reported success while the field stayed empty for 397 frames.

Fixed: the answer sequence no longer fires blind. The activation click gets a much longer settle (1.2 s) before the caret click, cmd+a is dropped (the field is empty, and it is a no-op at best in a non-key window), and — the substantive change — the loop now verifies the password actually landed before submitting: a secure text field replaces its placeholder with bullets once it holds text, so the placeholder still being recognised is direct evidence the keystrokes never reached the guest. Return is sent only on confirmation; otherwise the attempt is recorded landed=false and retried rather than submitting an empty field. That single line in the transcript will distinguish "keystrokes are not reaching the guest at all" from "they land but the sheet rejects them", which no previous run could.

Workstream 1 — enrollment-pass blockers

  • 1.1 Run the enroll script as the console user with per-step sudo (hosts pin, CA trust), so open registers the profile for the console user; make the CA-trust step headless by granting com.apple.trust-settings.admin non-interactively (the confirmed §8.0 hang). Log the console user as the auto-login canary and report a non-zero open rather than swallowing it.
  • 1.2 Try an ordered list of System Settings pane identifiers (com.apple.Profiles-Settings.extension, com.apple.settings.PrivacySecurity.extension, then the legacy id), advancing when nothing is recognized, and record which worked. Mirrors the existing nudgeMDMCheckIn candidate pattern.
  • 1.3 Bound the approval loop by approvalTimeout (raised to 300 s) rather than 16 iterations, republish per-state progress, and treat an unchanged fingerprint for ~60 s as its own failure.
  • 1.4 Force the surface on-screen for the duration of the approval loop and restore the prior state afterward; honor baseObserverRequested like the other passes; fail fast on N consecutive captures with zero recognized lines.
  • 1.5 Drop contains(expectedUser) from the authentication classifier and accept the known phrasings. The safety property stays what it already is: credentials submitted at most once, with both a Password label and a submit control required in the same capture. Accept Install... alongside Install….
  • 1.6 Make guest state the success signal: OCR drives actions only, and the loop terminates on verifyEnrolled. Tighten verifyEnrolled so Mode A requires User Approved rather than also accepting a bare MDM enrollment: Yes.

Workstream 2 — MDM protocol conformance

  • [~] 2.1 Remove ServerCapabilities from the enrollment profile. Reversed by the seventh live run: macOS 27 requires the key (install fails without it). It is kept; Workstream 2.2 makes advertising the per-user channel safe by acking the unmodeled check-ins it triggers.
  • 2.2 Never 400 the check-in channel: add an other(type:udid:) case, ack unknown types with an empty 200, and reserve 400 for a body that is not a property list (logging its prefix).
  • 2.3 Regression-test both; they are Linux-runnable in NucleicMDMTests.

Workstream 3 — CA and TLS identity correctness

  • 3.1 Issue leaves with a fresh 16-byte random serial (RFC 5280 §4.1.2.2 permits 20 octets). Guarantees uniqueness across issuance and reloads without making the struct mutable or changing the persisted shape; Persisted.nextSerial stays for compatibility and becomes vestigial. Replace testUniqueSerials with one that asserts distinctness.
  • 3.2 Import the TLS identity into a temporary per-start keychain and delete it in stop(), instead of accumulating identities in the default keychain.
  • 3.3 Await server.stop() on both the success and throw paths.

Workstream 4 — verification and gating

  • 4.1 Check the profile inventory in the guest (one PRESENT:<identifier> line per hit) instead of draining a large XML document through a silently truncating cap.
  • 4.2 Persist MDM state only after verification, and strengthen the stamp gate from fileExists to "state parses and its profileIdentifiers match the required set".
  • 4.3 Recover from a missing auto-login: have the minimal pass read back the effective auto-login user and fail there, and let the MDM pass answer a recognized login window once. Same fail-closed discipline — an unrecognized screen receives no input.

Workstream 5 — escape hatch

MacVMSurfaceHost.presentOperatorAssist(name:visible:) already exists and is unused here. If the UAMDM flow still cannot be driven unattended after Workstreams 14, the approval loop's terminal state becomes "present the assist window and wait, bounded, for a human to click Install" rather than failing the build. Base builds are rare and already operator-initiated, so a one-click hand-off is a better outcome than a failed build, and it decouples shipping a working base from winning the OCR fight. Wire it opt-in first (nucleic.macvm.mdmOperatorAssist), then decide from live evidence whether it becomes the default. The SIP-off lane (provisioner Phase 7c/d) remains the deeper fallback, but it flips the base's security posture and is a last resort.

Test plan

Linux-runnable (NucleicMDMTests), no toolchain gate:

  • unknown check-in MessageType → 200 empty (2.2)
  • the enrollment profile carries no ServerCapabilities (2.1)
  • two leaves from one CA have distinct serials; the root is serial 1 (3.1)
  • enrollScript() invokes open unprivileged and sudos only the hosts pin and CA trust (1.1)
  • the classifier accepts an auth sheet lacking the account name, and a bare "Installed" does not terminate the loop (1.5, 1.6)
  • verifyEnrolled rejects MDM enrollment: Yes without User Approved (1.6)
  • the inventory command emits one PRESENT: line per installed profile (4.1)

Host-side (NucleicCoreTests, needs an approved macOS run):

  • back-to-back MDMTLSIdentity.serverIdentity calls both succeed (3.2)
  • pane-candidate ordering and the guest inventory command shape (1.2, 4.1)

Live macOS 27: the P2 checklist above, run through the mdm-spike loop (0.6) until it passes twice consecutively, then once end to end through buildBaseImage.


Queue-drain repair (third investigation)

Opened: 2026-07-26 Symptom: enrollment completes (Authenticate signed + 200 OK, User Approved), then the two queued InstallProfile commands never drain: no TokenUpdate, no /server poll, timeout at queue-drain.

Diagnosis (evidence, not inference)

  1. Why there is no TokenUpdate: apsd never obtains a push token because it never dials a courier. The guest log (run-enroll-968ae158, ~line 1217) names the gate: APSBAAClientIdentityProvider failed to obtain a BAA cert ← MobileActivation "Failed to create reference key"errSecInteractionNotAllowed, <sepk:* kid=0000000000000000>: unable to generate key, then "Told not to connect after fetching server bag". macOS 27's apsd requires a SEP-backed BAA client identity before opening a socket; SEP keygen fails in the guest, so apsd deliberately stays offline — zero packets to 17/8, a rising failure counter, and no error surfaced anywhere louder than debug logging. Networking was never the problem.
  2. Why the nudges were inert: launchctl kickstart on an already-running on-demand daemon is a no-op; profiles renew -type enrollment is ADE-only (repurposed on macOS 27); mdmclient query verbs answer locally. Twelve "successful" nudges in run-enroll-22d72c27 produced zero connections. There is no supported command that makes an enrolled Mac poll on demand.
  3. The design premise was false. "mdmclient polls immediately after TokenUpdate, so pre-queued commands drain with no APNs" (old §2.4) is not the protocol: the command channel drains on a push — or on the two triggers below. (§2.4 is corrected in docs/MACOS_VM_MDM.md.)
  4. Two drain triggers need no APNs and no push certificate:
    • Login-time sync (Apple-documented): at login, mdmclient blocks on an Idle/Acknowledged/Error exchange with the server. The guest auto-logs-in ⇒ a reboot with the server still listening is a supported, deterministic drain.
    • Cold daemon start (proven live, run-push-e3d6a047 2026-07-26 00:1500:17): one launchctl kickstart of the stopped com.apple.mdmclient.daemonIdle poll 2 m 13 s later → both InstallProfiles delivered, Acknowledged, and verified installed. The held base guest already carries both policy profiles.
  5. Why BAA keygen fails — the mechanism, after the deep dig (2026-07-26). Three facts compose:
    • VZ guests categorically cannot mint SEP-resident keys, on every macOS version. The Sequoia+ "identity derived from the host's Secure Enclave" feature is an attestation identity blob delivered via an exclave (the AppleVirtualPlatformIdentity plugin) for Apple-Account flows — it is not a virtual SEP, and kSecAttrTokenIDSecureEnclave keygen stays unavailable (khronokernel: the VP kexts stub SEP attestation with errors; Eclectic Light on Sequoia/Tahoe VMs). So MobileActivation's "reference key" creation fails in any VZ guest — nothing Nucleic did causes sepk kid=0 / errSecInteractionNotAllowed. (The same error chain even appears sporadically on physical Macs — Apple forums thread 758028, macOS 14.5 ADE — so it is a keystore-refusal signature, not a VM oddity.)
    • macOS ≤26 apsd did not require that key. Tahoe (26.0/26.2) VZ guests have full working iCloud — which rides on APNs — despite the identical SEP limitation. The courier identity there comes from the classic Albert-provisioned client certificate.
    • macOS 27 beta 4 apsd consults APSBAAClientIdentityProvider and refuses to connect without it ("identity to connect? NO → Told not to connect after fetching server bag"). The symbol has zero public mentions — this identity path is new in 27 (or newly mandatory, possibly server-bag-gated), and our failure appears to be unreported. Everything else was individually exonerated: solo macOS VM confirmed (the second VZ process was a Linux guest — no Mac machine identifier, so no identity rotation), valid serial, healthy networking, identity plugin attached, and the account comes from Apple's own VZMacGuestProvisioningOptions (which feeds Setup Assistant's first-user flow; no public link exists between SecureToken/ownership gaps and -25308). The concurrent-duplicate-MID rotation rule remains a latent fleet hazard for session clones — D.3's concern — but did not cause this. Actions: file Apple Feedback/DTS with the BAA chain and a minimal repro (D.7) — Apple fixed both adjacent 27-era VM radars (163294564 serials, 179068335 installer); retest each beta; fresh per-clone MachineIdentifiers remain a non-option (install is personalized to the base ECID, VirtualBuddy #238).

Workstreams

  • D.1 Unblock now: release the held diagnostic guest, then re-run the MDM pass (revalidateBaseMDMPolicy). The enrolled base takes .updatepushProfiles — the exact path that already succeeded. Expect apsd to keep failing (see 5) — the drain does not need it; capture the apsd/BAA lines anyway as the evidence bundle for the Apple feedback report.
  • D.2 Make the Mode A drain deterministic — done (MacVMEngine+MDM.swift). enrollBaseInMDM now calls drainQueueForModeA, which escalates: (1) 90 s quiet wait for a natural check-in; (2) coldStartMDMClient (launchctl kickstart -k … || kickstart …) + 150 s wait — the proven cheap trigger; (3) reboot the guest with the server still listening (shutdownBaseGuestinstance.start()awaitAgentReady), whose auto-login runs the login-time Idle sync, then a 420 s drainRebootDrainSeconds wait. nudgeMDMCheckIn and the waitForDrain nudge cadence are deleted; waitForDrain is now a pure poll. Mode B (pushProfiles) uses a single coldStartMDMClient (a live clone must not be rebooted under a user). Builds clean.
  • D.3 Solo-run invariant: the MDM pass (and any base-mutating boot) refuses to start — with a named error — while another macOS VM is live, so the base can never boot onto a rotated identity mid-build.
  • D.4 Diagnose-by-default: add mobileactivationd to the guest-log predicate; on drain failure, surface the apsd BAA/courier lines in the failure message instead of a bare "N still queued".
  • D.5 Deferred / optional APNs restoration: if D.1 shows 5(b), file Apple feedback and retest each beta; a macOS 26 IPSW pin (restoreImagePath) is the fallback lever if 27 stays broken. A real push certificate (mdmcert.download; Topic must then come from the cert) is needed only if Apple-initiated push ever becomes a requirement — the reboot/cold-start triggers make it unnecessary for both Mode A and Mode B today.
  • D.6 Docs: §2.4 corrected (done); close debug-log open question with these facts (done); strike the debug log's parting suggestion to GUI-install the policy profiles — PPPC and notification-settings payloads are inert unless MDM-delivered, so that fallback never worked.
  • D.7 Minimal repro + in-guest confirmation (closes the residual doubt). Two cheap experiments, both independent of the app's held state: 1. Near-pristine guest: fresh IPSW install into a temp bundle, started with only Apple's VZMacGuestProvisioningOptions and enablesRemoteLogin = true; ssh in and run sudo apsctl status (production block), log stream on com.apple.apsd + mobileactivationd, sysadminctl -secureTokenStatus <user>, and diskutil apfs listUsers /. The BAA failure reproducing on a guest Nucleic never scripted is the definitive "it's macOS, not us" — and the artifact IS the Apple Feedback repro. 2. Current base (after release): same ladder, plus confirm the account's SecureToken/ownership state — closing the last "our provisioning" suspect empirically rather than by argument.

Rollout

No on-disk format changes, so the provisioning recipe stays at v8 and existing bases need no forced rebuild beyond what v8 already triggers. On landing, close the relevant docs/MACOS_VM_MDM.md §8 spikes with facts from the transcript rather than assumptions.