blakeslee-research/gitea-macos-vm-orchestrator
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge nucleic/vivid-glass-urchin-xoym into main

Browse Source
This commit is contained in:
abkslm
2026-08-07 15:41:01 -07:00
parent 6b0c01b74b
commit 902bea5091
11 changed files with 450 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/troubleshooting.md
+48 -21
View File
@@ -24,7 +24,7 @@ gitea-macos-runner service status
| VM won't start; entitlement / `com.apple.security.virtualization` error | Running an unsigned binary, or one outside the signed `.app` bundle | `make sign` (or re-run `make install`); invoke the installed bundle, never `.build/release/…` |
| `virtualMachineLimitExceeded` at boot | macOS allows at most **2** concurrent macOS VMs | Set `scheduler.maxConcurrentVMs` ≤ 2; kill stray VMs from earlier runs |
| VM boots but never gets an IP | DHCP lease not yet written, or Local Network privacy denial (macOS 15+) | Check `/var/db/dhcpd_leases`; grant Local Network permission or pre-authorize the subnet |
| Runner not listed under Privacy & Security → Local Network | Expected — the list is populated only after the app first attempts a local connection; it cannot be pre-approved | Boot one VM by hand from a GUI Terminal to create the entry, or (better on CI) allowlist the subnet with `defaults write com.apple.network.local-network` |
| Runner not listed under Privacy & Security → Local Network | Expected — the list is populated only after the app first attempts a local connection, and a run started from a shell is attributed to Terminal, not to the app | Allowlist the subnet with `defaults write com.apple.network.local-network` and reboot; the interactive grant does not reach the LaunchAgent |
| SSH times out on a freshly built image | Guest macOS < 27, so provisioning options were ignored and Setup Assistant is waiting | Rebuild the image from a macOS **27+** IPSW |
| VMs boot in a loop; every teardown says `reason=cancelled` and nothing is logged between the lease and the teardown | `scheduler.bootTimeoutSeconds` is below the guest's *worst-case* boot on a contended host, so each clone is killed while still starting — and each replacement makes the next one slower | Raise `scheduler.bootTimeoutSeconds` (default 900) and reduce the number of concurrent guests; see [The daemon boots VMs forever](#the-daemon-boots-vms-forever-and-every-teardown-says-reasoncancelled) |
| `ssh failed: cannot connect … No route to host) (errno: 65)` part-way through provisioning | macOS 15+ Local Network privacy blocking the app — the grant is keyed on the executable's UUID, so `make install` withdraws it | Allowlist the subnet (`192.168.64.0/18`) and **reboot**; see [SSH fails with "No route to host" mid-run](#ssh-fails-with-no-route-to-host-errno-65-mid-run) |
@@ -63,8 +63,9 @@ codesign -d --entitlements - ~/Applications/GiteaMacosRunner.app # verify
The output must list `com.apple.security.virtualization`. Then confirm the command you're running
resolves to the installed bundle's binary — `which -a gitea-macos-runner` should point at
`~/Applications/GiteaMacosRunner.app/Contents/MacOS/gitea-macos-runner`, not at
`.build/release/gitea-macos-runner`. Ad-hoc signing is sufficient; you do not need a paid developer
account.
`.build/release/gitea-macos-runner`. Ad-hoc signing is sufficient for *this* error; you do not need
a developer account to start VMs. (A Developer ID certificate solves a different problem — Local
Network grants evaporating on rebuild. See [setup.md](setup.md#code-signing).)
---
@@ -120,9 +121,10 @@ prompt**: a LaunchAgent that was never granted permission (or was denied) cannot
see [Local Network: the app is not listed in System Settings](#local-network-the-app-is-not-listed-in-system-settings)
for why the interactive grant is not.
3. Or grant it interactively: run `gitea-macos-runner vm boot --image default` from a Terminal in
the GUI session and click **Allow**. The app is not listed under **System Settings → Privacy &
Security → Local Network** until it has made that first attempt.
Do not reach for the interactive grant instead. Booting a VM by hand from a Terminal makes the
prompt (and the **System Settings → Privacy & Security → Local Network** row) belong to
*Terminal* rather than to this app, and approving it there does not carry over to the
LaunchAgent.
---
@@ -142,17 +144,24 @@ privacy controls, Local Network privacy is not stored in TCC — per
the checks live "deep in the networking stack" as a Network Extension packet filter, so the
permission is absent from `TCC.db` and `tccutil reset` does not apply to it.
**Fix — interactive.** Make the app connect once, from a GUI session where a human can answer:
**Why you will probably never see the app's own row.** macOS assigns a privacy decision to the
*responsible process*, not necessarily to the binary that opened the socket. Launching the runner
the obvious way —
```sh
gitea-macos-runner vm boot --image default
```
Click **Allow**. The entry now exists and can be toggled later. Do not wait for the LaunchAgent to
trigger it; a background agent cannot answer the prompt, so it simply fails to reach the guest.
— execs the bundle's binary from a shell, so the system holds **Terminal** responsible. Both the
prompt and the Settings row belong to Terminal, and allowing it there does **not** carry over to the
LaunchAgent, which is the process that actually needs it. Meanwhile the LaunchAgent itself has no UI
to show a prompt in, so from it the connection is denied outright and surfaces as `No route to host`
(errno 65) rather than as a permission error. Between the two, there is no reliable way to grant
this interactively on an unattended host.
**Fix — deterministic, and what to use on a CI box.** Allowlist the VM subnet instead. It is keyed
on the network rather than on the app, so no prompt is involved and nothing needs redoing:
**Fix — deterministic, and what to use on any host running the LaunchAgent.** Allowlist the VM
subnet instead. It is keyed on the network rather than on the app, so no prompt is involved, and
nothing needs redoing:
```sh
sudo defaults write com.apple.network.local-network \
@@ -165,12 +174,18 @@ Reboot afterwards. `doctor` then reports `local network access` as a **pass**. B
Apple's, documented in TN3179; the same pair is what [Tart's FAQ](https://tart.run/faq/) recommends
for this exact problem on CI hosts.
> **Why the interactive grant does not stick here.** This project ships an **ad-hoc signed** bundle
> (`codesign --sign -`), and TN3179 notes that "local network privacy uses your main executable UUID
> as part of its implementation". The linker writes a new `LC_UUID` on essentially every rebuild, so
> a rebuilt-and-reinstalled runner can read as a *different* program and prompt again — while the
> old row lingers, since macOS provides no way to reset a Local Network decision to undetermined.
> Expect duplicate entries after a few upgrades. The subnet allowlist avoids all of this.
> **And an ad-hoc signed bundle cannot hold the grant anyway.** TN3179 notes that "local network
> privacy uses your main executable UUID as part of its implementation". An ad-hoc signature has no
> team anchor, so that UUID *is* the app's identity — and the linker writes a new `LC_UUID` on
> essentially every rebuild, so a rebuilt-and-reinstalled runner reads as a *different* program and
> prompts again, while the old row lingers (macOS provides no way to reset a Local Network decision
> to undetermined). Expect duplicate entries after a few upgrades.
>
> Signing with a **Developer ID Application** certificate fixes the churn: its designated
> requirement is anchored to your team, so every build is recognised as the same program. `make sign`
> uses one automatically when it is in the keychain — check with `codesign -dvv` or `doctor`'s
> `code identity` line, and see [setup.md](setup.md#code-signing). The subnet allowlist avoids the
> mechanism entirely and is still the right answer for an unattended host.
---
@@ -591,23 +606,35 @@ error: provisioning failed: could not upload provision.sh from …/provision.sh:
ssh failed: cannot connect to 192.168.65.232:22: … No route to host) (errno: 65)
```
**First, check whether it is actually failing.** A single errno 65 at `attempt=1 elapsed=0s`,
seconds after `guest leased address`, is normal and not worth chasing. `waitForSSH` logs its first
probe unconditionally and then heartbeats every 30 s, and that first probe usually lands before the
host has an ARP entry for the guest — which is also `EHOSTUNREACH`. What matters is whether the
message *repeats* at `elapsed=30s`, `60s`, `90s`, … If it does not, boot is proceeding normally. If
it does, read on.
**Cause.** On macOS 15 and newer, an app that has not been granted Local Network access does not get
a "permission denied": the packet filter answers **`EHOSTUNREACH` — errno 65, "No route to host"**,
which is indistinguishable from a guest that is genuinely off the network. Guests here live on a
host-private NAT link that is reachable whenever the VM is up, so on this path errno 65 is far more
often the privacy filter than a routing problem.
host-private NAT link that is reachable whenever the VM is up, so on this path a *persistent* errno
65 is far more often the privacy filter than a routing problem.
Two details make it look intermittent rather than like a permission problem:
- **The grant is keyed on the executable's UUID.** Per
- **Under an ad-hoc signature the grant is keyed on the executable's UUID.** Per
[TN3179](https://developer.apple.com/documentation/technotes/tn3179-understanding-local-network-privacy),
Local Network privacy "uses your main executable UUID as part of its implementation", and the
linker mints a fresh `LC_UUID` on essentially every build. A `make install` after a code change
therefore presents a program macOS has never seen, whose permission is undetermined again — even
though the binary you ran ten minutes ago worked.
though the binary you ran ten minutes ago worked. A Developer ID signature anchors identity to the
team instead and does not drift; `doctor`'s `code identity` check tells you which you have, and
[setup.md](setup.md#code-signing) covers switching.
- **Processes started over SSH are exempt.** Running the same command through `ssh you@host …`
succeeds while running it from a GUI Terminal fails. A remote-shell success proves nothing about
the interactive path.
- **A shell-launched run is attributed to Terminal.** macOS charges the privacy decision to the
responsible process, so the app's own identity is not what is being evaluated when you launch it
by hand — and a grant given to Terminal does nothing for the LaunchAgent.
**Fix.** Allowlist the subnet — it is keyed on the network, not on the app, so no rebuild can
withdraw it and no prompt has to be answered: