Nucleic: Gitea Runner macOS VM Support
This commit is contained in:
+581
@@ -0,0 +1,581 @@
|
||||
# gitea-macos-runner — Design
|
||||
|
||||
## 1. Overview
|
||||
|
||||
`gitea-macos-runner` is a single-host daemon for an Apple Silicon Mac. It watches
|
||||
a Gitea instance for queued Actions jobs that require macOS, and for each one it
|
||||
boots a **fresh, ephemeral macOS VM** on Apple's Virtualization.framework,
|
||||
registers a single-use runner inside it, lets the job run, and then destroys the
|
||||
VM.
|
||||
|
||||
The design goal is that **no state survives a job**. Not a checkout, not a
|
||||
keychain entry, not a `~/Library` mutation, not a leftover process. The guest
|
||||
that runs job *N+1* is a byte-identical copy-on-write clone of the same base
|
||||
image that job *N* started from. This is the property that a persistent
|
||||
self-hosted Mac runner cannot offer, and it is the whole reason this tool exists.
|
||||
|
||||
Three constraints shape everything below:
|
||||
|
||||
1. **Apple's kernel allows at most two concurrent macOS guests per host.** Not a
|
||||
policy, not a licence term we chose — a hard limit that surfaces as
|
||||
`VZError.virtualMachineLimitExceeded` from `start()`. Concurrency is therefore
|
||||
2, permanently, and the config value is clamped rather than trusted.
|
||||
2. **Virtualization needs a GUI session and a signed bundle.** The daemon runs as
|
||||
a LaunchAgent in a logged-in user session, from inside an ad-hoc-signed `.app`
|
||||
carrying `com.apple.security.virtualization`.
|
||||
3. **Gitea decides which job a runner claims, not us.** We supply capacity; the
|
||||
server matches. Trying to pin a specific job to a specific VM would mean
|
||||
reimplementing Gitea's matching rules, and would be wrong the moment they
|
||||
change.
|
||||
|
||||
### Non-goals
|
||||
|
||||
* Multi-host scheduling. One daemon, one Mac, two slots.
|
||||
* Container-based execution. Gitea's `host` schema runs jobs directly on the
|
||||
guest; that is the point of having a real macOS VM.
|
||||
* Bridged networking. NAT only — see §6.
|
||||
* Guest reuse or warm pools. See §9 for why save/restore is deferred rather than
|
||||
rejected.
|
||||
|
||||
---
|
||||
|
||||
## 2. Component diagram
|
||||
|
||||
```
|
||||
┌──────────────────────────────── Host (Apple Silicon Mac, macOS 26+) ─────────────────────────────┐
|
||||
│ │
|
||||
│ LaunchAgent (user session, auto-login, login.keychain unlocked) │
|
||||
│ └── GiteaMacosRunner.app (ad-hoc signed, com.apple.security.virtualization, LSUIElement) │
|
||||
│ │ │
|
||||
│ │ NSApplication(.prohibited).run() ── main thread, required by Virtualization │
|
||||
│ │ │
|
||||
│ ┌─────▼──────────────────────────── Orchestrator (actor) ────────────────────────────────┐ │
|
||||
│ │ │ │
|
||||
│ │ poll loop ──► GiteaClient.listQueuedJobs() ──► [WorkflowJob] │ │
|
||||
│ │ │ │ │
|
||||
│ │ ├──────► SchedulerCore.plan(...) ── PURE, no I/O ──► [SchedulerAction] │ │
|
||||
│ │ │ │ │
|
||||
│ │ ├──► bootVM(slot,jobHint) ──► VMStore.cloneImage ──► VMInstance.start │ │
|
||||
│ │ │ │ │ │ │
|
||||
│ │ │ │ ▼ │ │
|
||||
│ │ │ │ DHCPLeaseParser(/var/db/…) │ │
|
||||
│ │ │ │ │ │ │
|
||||
│ │ │ │ ▼ │ │
|
||||
│ │ │ │ SSHExecutor ──► gitea-runner │ │
|
||||
│ │ │ │ register+daemon │ │
|
||||
│ │ └──► teardownVM(slot,reason) ──► VMInstance.requestStopThenForce ──► deleteClone │ │
|
||||
│ │ │ │
|
||||
│ │ reconcile loop ──► GiteaClient.listRunners / deleteRunner (sweep orphaned rows) │ │
|
||||
│ └────────────────────────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ <storeDir>/ │
|
||||
│ images/default/{disk.asif, nvram.bin, config.json} ← built once, provisioned, read-only │
|
||||
│ vms/<uuid>/{disk.asif, nvram.bin, config.json} ← APFS CoW clones, destroyed per job │
|
||||
│ ipsw/ ← downloaded restore images │
|
||||
│ state.json ← the two persistent per-slot MACs │
|
||||
│ │
|
||||
│ ┌──── VM slot 0 (MAC A) ────┐ ┌──── VM slot 1 (MAC B) ────┐ ← at most 2, kernel-enforced │
|
||||
│ │ macOS guest │ │ macOS guest │ │
|
||||
│ │ gitea-runner --ephemeral │ │ gitea-runner --ephemeral │ │
|
||||
│ │ node, git, bash │ │ node, git, bash │ │
|
||||
│ └───────────┬───────────────┘ └───────────┬───────────────┘ │
|
||||
└──────────────┼───────────────────────────────┼───────────────────────────────────────────────────┘
|
||||
│ NAT (vmenet, bootpd) │
|
||||
└───────────────┬───────────────┘
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ Gitea 1.25+ │
|
||||
│ /api/v1/admin/… │
|
||||
└─────────────────────┘
|
||||
```
|
||||
|
||||
### Module boundaries
|
||||
|
||||
| Target | Contains | Constraint |
|
||||
|---|---|---|
|
||||
| `RunnerCore` | Config, Gitea models + client, `LabelSet`, `DHCPLeaseParser`, `SSHExec`, `SchedulerCore` | **No `import Virtualization`.** Builds on Linux, so scheduling and parsing logic can be unit-tested anywhere. |
|
||||
| `RunnerHost` | `VMBundle`, `VMStore`, `VZConfigFactory`, `VMInstance`, `IPSW`, `ImageBuilder`, `GuestProvisioner`, `Orchestrator`, `LaunchdService`, `Doctor` | macOS-only. Everything that touches the framework. |
|
||||
| `gitea-macos-runner` | CLI + daemon entry point | Depends on both. |
|
||||
|
||||
The split is not cosmetic: `SchedulerCore` being pure and portable is what makes
|
||||
the scheduling policy — the part most likely to have subtle bugs — testable
|
||||
without a Mac, a VM, or a Gitea instance.
|
||||
|
||||
---
|
||||
|
||||
## 3. Job lifecycle
|
||||
|
||||
```
|
||||
Gitea Orchestrator VMStore / VMInstance Guest
|
||||
│ │ │ │
|
||||
│◄── listQueuedJobs ────────┤ (every pollIntervalSeconds) │ │
|
||||
├─── [job 4711, labels ─────► │ │
|
||||
│ ["macos-arm64"]] │ │ │
|
||||
│ ├── LabelSet.matches? ─────────┤ │
|
||||
│ ├── SchedulerCore.plan ────────┤ │
|
||||
│ │ → .bootVM(slot: 0, │ │
|
||||
│ │ jobHint: 4711) │ │
|
||||
│ │ │ │
|
||||
│ ├── ensureFreeSpace(minGB) ───►│ │
|
||||
│ ├── cloneImage("default", ───►│ APFS CoW copy │
|
||||
│ │ slotMAC: MAC-A) │ + rewrite config.json │
|
||||
│ ├── VMInstance.start() ───────►│ ──── boot ───────────►│
|
||||
│ │ │ │
|
||||
│ ├── poll /var/db/dhcpd_leases ─┤◄─── DHCP request ──────┤
|
||||
│ │ until MAC-A has an IP │ │
|
||||
│ ├── waitForSSH(ip) ────────────┼───────────────────────►│
|
||||
│ │ │ │
|
||||
│ ├── uploadData(token, 0600) ───┼───────────────────────►│
|
||||
│ ├── ssh: gitea-runner register ┼───────────────────────►│
|
||||
│◄────────────────────── register (name=macos-vm-<uuid>, --ephemeral) ──────────────┤
|
||||
│ ├── ssh: rm -f <tokenfile> │ │
|
||||
│ ├── ssh: gitea-runner daemon ──┼───────────────────────►│
|
||||
│ │ │ │
|
||||
│◄────────────────────── poll for task ─────────────────────────────────────────────┤
|
||||
├─── assign job 4711 ───────────────────────────────────────────────────────────────►
|
||||
│ │ │ ...running... │
|
||||
│◄────────────────────── job result, logs ──────────────────────────────────────────┤
|
||||
├─── auto-deregister runner (server-enforced --ephemeral) ──────────────────────────►
|
||||
│ │ │ daemon exits │
|
||||
│ │◄─ SSH command returns ───────┼────────────────────────┤
|
||||
│ ├── teardownVM(slot: 0) ──────►│ │
|
||||
│ │ requestStopThenForce ──────┼───────────────────────►│ (halt)
|
||||
│ │ deleteClone ──────────────►│ rm -rf vms/<uuid> │
|
||||
│ ├── markIdle(slot: 0) │ │
|
||||
```
|
||||
|
||||
Two details in that sequence carry more weight than their size suggests.
|
||||
|
||||
**The token goes through a file, not an argument.** `gitea-runner register` is
|
||||
invoked with `--token-file <f>`, where `<f>` was written by `uploadData` with
|
||||
mode `0600` and is `rm -f`'d in the same shell command. Passing `--token` would
|
||||
put a fleet-wide credential into the guest's process table, visible to any
|
||||
process the job spawns — and the job is arbitrary code from a repository.
|
||||
|
||||
**`--ephemeral`, not `--once`.** `--ephemeral` (Gitea 1.24+) is enforced *by the
|
||||
server*: it hands this runner exactly one task and then deletes the registration.
|
||||
`--once` is a runner-side convention only — the server still considers the runner
|
||||
live, and a misbehaving or patched runner could claim more work. Since the whole
|
||||
security story here rests on "one VM, one job", the enforcement has to live on
|
||||
the side we don't hand to the job.
|
||||
|
||||
---
|
||||
|
||||
## 4. Image build pipeline
|
||||
|
||||
Base images are built once with `image build`, and every job clones one. The
|
||||
build is slow (most of an hour, mostly a ~15 GB download); the clone is
|
||||
milliseconds.
|
||||
|
||||
```
|
||||
image build --name default [--ipsw PATH]
|
||||
│
|
||||
├─ 1. IPSWProvider.latestSupported() → CDN url + buildVersion
|
||||
│ (VZMacOSRestoreImage.latestSupported returns a NETWORK url —
|
||||
│ it cannot be handed to the installer)
|
||||
├─ 2. IPSWProvider.download() → <storeDir>/ipsw/*.ipsw
|
||||
├─ 3. IPSWProvider.load(localPath:) → VZMacOSRestoreImage
|
||||
│ (resolveSymlinksInPath first; the framework rejects symlinks)
|
||||
│
|
||||
├─ 4. restoreImage.mostFeaturefulSupportedConfiguration
|
||||
│ nil ⇒ this host cannot run this image. Fail loudly; do not guess.
|
||||
│
|
||||
├─ 5. createBundle()
|
||||
│ hardwareModel.dataRepresentation → config.json
|
||||
│ VZMacMachineIdentifier() (fresh) → config.json
|
||||
│ VZMacAuxiliaryStorage(creatingStorageAt:hardwareModel:) → nvram.bin
|
||||
│ disk: diskutil image create blank --fs none --format ASIF --size <N>G
|
||||
│ └─ fallback: sparse RAW file (format recorded in config.json)
|
||||
│
|
||||
├─ 6. VZMacOSInstaller(virtualMachine:restoringFromImageAt:) on a STOPPED vm
|
||||
│ KVO on installer.progress → percentage
|
||||
│
|
||||
├─ 7. first boot with Setup Assistant automation
|
||||
│ #available(macOS 27.0, *):
|
||||
│ VZMacGuestProvisioningOptions(username/password/fullName,
|
||||
│ logsInAutomatically: true,
|
||||
│ enablesRemoteLogin: true)
|
||||
│ → VZMacOSVirtualMachineStartOptions.setGuestProvisioning(_:)
|
||||
│ ⚠ an OLDER GUEST SILENTLY IGNORES THIS — no error, no account, no SSH
|
||||
│
|
||||
├─ 8. wait for DHCP lease (by MAC) → wait for SSH → GuestProvisioner
|
||||
│ provision.sh (sudoers, no-sleep, no-Spotlight, maxfiles, known_hosts)
|
||||
│ Node.js (official arm64 .pkg → installer -pkg) ← REQUIRED
|
||||
│ verify git / bash / node
|
||||
│ gitea-runner (host downloads asset → upload → chmod +x)
|
||||
│ [optional] Xcode from a .xip
|
||||
│
|
||||
└─ 9. clean shutdown → config.provisioned = true ← only now is it clonable
|
||||
```
|
||||
|
||||
### On step 7 and its failure mode
|
||||
|
||||
`VZMacGuestProvisioningOptions` needs **macOS 27 or newer on both the host and
|
||||
the guest**. The host side is a compile/availability check we control. The guest
|
||||
side is not: an older guest accepts the boot and simply ignores the options.
|
||||
There is no error to catch. The observable symptom is that the VM boots, sits at
|
||||
Setup Assistant forever, never requests a DHCP lease with a usable hostname, and
|
||||
never answers SSH — so the build fails at step 8 with a timeout that says nothing
|
||||
useful.
|
||||
|
||||
`firstBootAndProvision` therefore detects the timeout and reports it as an
|
||||
explicit "guest is too old for unattended setup; supply a macOS 27+ IPSW"
|
||||
failure. A `--manual-setup` flow that opens a window and lets a human click
|
||||
through Setup Assistant once is **out of scope for v1** — deliberately, because a
|
||||
GUI step in a tool whose whole purpose is unattended operation is a trap. It is
|
||||
noted here so the omission is a decision rather than an oversight.
|
||||
|
||||
### On step 5's disk format
|
||||
|
||||
ASIF is preferred because it is sparse: a 64 GB nominal disk costs what the guest
|
||||
actually writes, and it CoW-clones cleanly on APFS. `diskutil image create` is
|
||||
shelled out to because there is no framework API for it. If that call fails for
|
||||
any reason — older `diskutil`, unusual volume — a sparse RAW file is created
|
||||
instead and the format is recorded in `config.json`, so `VZConfigFactory` attaches
|
||||
the right file without re-probing.
|
||||
|
||||
---
|
||||
|
||||
## 5. Scheduling semantics
|
||||
|
||||
`SchedulerCore.plan` is a pure function: `(state, queuedJobs, labels, maxVMs,
|
||||
now, jobTimeout, bootTimeout) → (state', [action])`. It performs no I/O, reads no
|
||||
clock, and is fully deterministic — which is what allows the entire scheduling
|
||||
policy to be tested with a fixed `now` and a synthetic job list.
|
||||
|
||||
### Capacity, not assignment
|
||||
|
||||
This is the central idea and the easiest thing to get wrong.
|
||||
|
||||
A booted VM is **capacity**. It is not a promise to run a particular job. We see
|
||||
job 4711 queued, we boot a VM, we register an ephemeral runner — and the *server*
|
||||
then decides which queued job that runner claims. It may well claim job 4712
|
||||
instead. That is fine and in fact preferable: Gitea's matching rules (labels,
|
||||
repo permissions, ordering, priority) are its business, and any attempt to
|
||||
predict them here would be a reimplementation that drifts out of sync.
|
||||
|
||||
The `jobHint` threaded through `SchedulerAction.bootVM` and
|
||||
`SlotState.running` exists for exactly two purposes: log messages, and the dedup
|
||||
ledger below. Nothing else may depend on it.
|
||||
|
||||
### Dedup by job id
|
||||
|
||||
`SchedulerState.dispatchedJobIDs` is a `Set<Int64>` of jobs that have already
|
||||
caused a boot.
|
||||
|
||||
Without it, the loop is pathological. A VM takes tens of seconds to boot,
|
||||
provision, and register. The poll interval is 5 seconds. So a single queued job
|
||||
would still be queued on the next poll, and the next, and the next — triggering a
|
||||
second boot, then exhausting the slot budget, all for one job.
|
||||
|
||||
The ledger is expired against reality rather than against a timer: any id no
|
||||
longer appearing in the queued set is dropped. That way a slot freed by a
|
||||
completed job can be re-earned by a genuinely new job, but a job that is *still*
|
||||
waiting does not double-book.
|
||||
|
||||
### The cap
|
||||
|
||||
`maxVMs` is clamped to 2 in `plan`, and again in `RunnerConfig.validated()`. Both
|
||||
places, because the kernel limit is not something a config file gets to
|
||||
negotiate: a third `start()` raises `VZError.virtualMachineLimitExceeded`, which
|
||||
`VMInstance.mapVZError` translates into `CoreError.vmLimitExceeded` and the
|
||||
scheduler treats as transient back-pressure rather than a failure.
|
||||
|
||||
### Timeouts
|
||||
|
||||
* A slot in `.provisioning(since:)` longer than `bootTimeoutSeconds` (default
|
||||
300) is torn down. Covers a guest that never gets a lease, never starts `sshd`,
|
||||
or hangs in Setup Assistant.
|
||||
* A slot in `.running(jobHint:since:)` longer than `jobTimeoutMinutes` (default
|
||||
120) is torn down. Covers a job that hangs. This is comfortably below Gitea's
|
||||
own `ABANDONED_JOB_TIMEOUT` (24 h), so our teardown always happens first and
|
||||
the server sees a clean deregistration rather than an abandonment.
|
||||
|
||||
Teardown actions are emitted **before** boot actions in the returned list, so a
|
||||
slot freed in one pass can be reused in that same pass.
|
||||
|
||||
### Reconcile
|
||||
|
||||
Every `reconcileIntervalSeconds` (default 300), the orchestrator lists runners and
|
||||
deletes any that are:
|
||||
|
||||
* `ephemeral == true`, **and**
|
||||
* `busy == false`, **and**
|
||||
* `name` starts with our configured `namePrefix`, **and**
|
||||
* not backed by a live VM in this process.
|
||||
|
||||
All four conditions, because deleting a live runner fails a running job. The loop
|
||||
is deliberately conservative: a row we are unsure about is left alone, and will be
|
||||
revisited in five minutes.
|
||||
|
||||
This loop is not optional housekeeping — it is load-bearing. See Verified Fact 7.
|
||||
|
||||
---
|
||||
|
||||
## 6. Security model
|
||||
|
||||
### The threat
|
||||
|
||||
A CI job is arbitrary code from a repository, running with the privileges of the
|
||||
account it executes under. On a persistent self-hosted Mac runner, that code can
|
||||
read every previous job's checkout, poison caches, install launch agents, and
|
||||
harvest whatever credentials the machine has accumulated. Every subsequent job on
|
||||
that host inherits the compromise.
|
||||
|
||||
### The mitigation: genuinely ephemeral guests
|
||||
|
||||
* **One job per VM, enforced server-side.** `--ephemeral` means Gitea hands the
|
||||
runner exactly one task and then deletes the registration. A patched or
|
||||
hijacked runner binary cannot ask for more work, because the server will not
|
||||
give it any.
|
||||
* **The VM is destroyed after that job.** Not reset, not cleaned — the clone
|
||||
directory is `rm -rf`'d and the next job clones the base image afresh. There is
|
||||
no path by which job *N* influences job *N+1* short of compromising the host.
|
||||
* **The guest holds nothing worth stealing.** Its account credentials
|
||||
(`admin`/`admin` by default) are meaningful only on a host-private NAT link to a
|
||||
machine that is about to be deleted.
|
||||
|
||||
### The shared registration token
|
||||
|
||||
Registration tokens in Gitea are **reusable and scope-wide**, and minting a new
|
||||
one for a scope **invalidates all prior tokens of that scope**. That makes
|
||||
per-VM tokens actively harmful: generating one for each VM would break every
|
||||
other runner registered against that scope, including ones on other hosts.
|
||||
|
||||
So the fleet shares one token. The mitigations are:
|
||||
|
||||
* It is written into the guest as a **file with mode `0600`**, never as a command
|
||||
argument (arguments are world-readable via `ps`).
|
||||
* It is **deleted immediately** after `gitea-runner register` consumes it, in the
|
||||
same `&&` chain, before `gitea-runner daemon` starts and long before any job
|
||||
code runs.
|
||||
* It is a *registration* token, not an API token: it grants the ability to
|
||||
register a runner, not to read repositories or act as a user.
|
||||
|
||||
The residual risk is real but bounded — a job that wins a race against `rm -f`
|
||||
could register additional runners for that scope. The recommended deployment
|
||||
seeds a fixed token server-side via `GITEA_RUNNER_REGISTRATION_TOKEN` so that
|
||||
rotating it is a deliberate, coordinated act rather than an API call side effect.
|
||||
|
||||
### `:host` schema risk
|
||||
|
||||
Jobs run in `host` schema: directly on the guest OS, not in a container. That is
|
||||
the point — a macOS job needs real macOS. But it means the job has full user-level
|
||||
access to the guest, including `sudo` (which `provision.sh` makes passwordless,
|
||||
because Xcode and `installer` need it). Everything above rests on the guest being
|
||||
disposable and isolated, not on the job being constrained inside it.
|
||||
|
||||
### Host-side posture
|
||||
|
||||
* The daemon runs as a **LaunchAgent in a user session**, not as root. The
|
||||
entitlement it carries (`com.apple.security.virtualization`) grants VM creation
|
||||
and nothing else.
|
||||
* Networking is **NAT**, not bridged. Guests can reach the LAN and Gitea, but are
|
||||
not first-class hosts on it. Bridged networking would require the restricted
|
||||
`com.apple.vm.networking` entitlement, which ad-hoc signing cannot grant — a
|
||||
constraint that happens to align with what we want anyway.
|
||||
* **SSH host keys are not verified.** The peer is a VM this process booted
|
||||
moments ago on a link no other machine shares; pinning would break on every
|
||||
clone and add nothing.
|
||||
|
||||
---
|
||||
|
||||
## 7. Failure modes and recovery
|
||||
|
||||
| Failure | Detection | Recovery |
|
||||
|---|---|---|
|
||||
| Guest never gets a DHCP lease | `bootTimeout` in `waitForLease` | Teardown, slot recycled, retried next poll |
|
||||
| Guest never answers SSH | `bootTimeout` in `waitForSSH` | Same |
|
||||
| Job hangs | `jobTimeoutMinutes` | Teardown; Gitea reaps the task via its zombie sweep (~10–15 min) |
|
||||
| VM dies uncleanly | Runner row left behind, task stuck Running | Reconcile loop deletes the row (§5); Gitea's zombie sweep handles the task |
|
||||
| Daemon crashes with VMs live | Clones orphaned on disk | `purgeClones()` at startup, then one immediate reconcile pass |
|
||||
| Third VM requested | `VZError.virtualMachineLimitExceeded` | Mapped to `CoreError.vmLimitExceeded`, treated as back-pressure |
|
||||
| Disk fills | `ensureFreeSpace(minGB:)` before each clone | Boot refused, logged; jobs stay queued (safe — Gitea holds them ~24 h) |
|
||||
| Gitea unreachable | Request error in the poll loop | Logged, retried next tick; no state change |
|
||||
|
||||
---
|
||||
|
||||
## 8. Configuration and operations summary
|
||||
|
||||
Config lives at `~/.config/gitea-macos-runner/config.json`; see
|
||||
`Resources/config.example.json`. Order of operations for a new host:
|
||||
|
||||
```
|
||||
gitea-macos-runner doctor # verify arch, macOS, entitlement, keychain, Gitea
|
||||
gitea-macos-runner config init # write an annotated config
|
||||
gitea-macos-runner image build # ~1 hour, mostly IPSW download
|
||||
gitea-macos-runner vm boot # optional smoke test: boot a clone, print its IP
|
||||
gitea-macos-runner service install # LaunchAgent, RunAtLoad + KeepAlive
|
||||
gitea-macos-runner doctor # again, now that it runs from the signed .app
|
||||
```
|
||||
|
||||
`doctor` exists because every one of its checks corresponds to a failure that
|
||||
otherwise appears as an opaque error deep inside a VM boot. The most common by
|
||||
far: running from `.build/` instead of the signed `.app`, so the entitlement is
|
||||
absent.
|
||||
|
||||
---
|
||||
|
||||
## 9. Future work
|
||||
|
||||
* **Save/restore for warm boots.** `VZVirtualMachine.saveMachineStateTo` (macOS
|
||||
14+) could cut per-job boot from ~60 s to near-instant by restoring a snapshot
|
||||
taken just after `gitea-runner` is ready. The blocker is that restore forbids
|
||||
changing the MAC address or ECID, which collides with our per-slot MAC scheme
|
||||
(§ Verified Fact 12) — a restored state would have to be captured per slot, and
|
||||
the interaction with DHCP lease reuse needs care. Deferred, not rejected.
|
||||
* **vsock control channel.** `VZVirtioSocketDeviceConfiguration` is already in the
|
||||
VM configuration. Replacing SSH with a vsock agent would remove password auth,
|
||||
the `waitForSSH` poll, and the Local Network privacy prompt entirely.
|
||||
* **`--manual-setup`** for pre-macOS-27 guests (§4).
|
||||
* **Image versioning / garbage collection** for multiple base images.
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Verified Facts
|
||||
|
||||
Researched facts this design depends on, each with its consequence for the
|
||||
implementation. Anyone changing the corresponding code should re-verify the fact
|
||||
first.
|
||||
|
||||
**1. Job discovery is `GET /api/v1/admin/actions/jobs?status=queued` (Gitea
|
||||
1.25+).** The `labels` field on a returned job is the workflow's `runs-on:`
|
||||
value. The external status string `queued` maps to Gitea's internal
|
||||
`StatusWaiting`, meaning "ready, waiting for a matching runner".
|
||||
→ *Consequence:* the external string `waiting` means something entirely
|
||||
different — the job is **blocked** on a dependency — and must **never** be
|
||||
treated as schedulable. `WorkflowJob.isQueued` checks `status == "queued"` and
|
||||
nothing else.
|
||||
|
||||
**2. A queued job waits for a matching runner up to `ABANDONED_JOB_TIMEOUT`
|
||||
(default 24 h, swept every 6 h).**
|
||||
→ *Consequence:* there is no urgency in the poll loop. A 5-second interval is for
|
||||
responsiveness, not correctness; a daemon that is down for an hour loses nothing.
|
||||
It also sets the ceiling that `jobTimeoutMinutes` (default 120) must stay well
|
||||
under, so our teardown always precedes the server's abandonment.
|
||||
|
||||
**3. The runner binary is `gitea-runner` v3.x**, renamed from `act_runner` and
|
||||
published from `gitea.com/gitea/runner`. The register-time flag `--ephemeral`
|
||||
(Gitea 1.24+) is **server-enforced**: single job, then automatic deregistration.
|
||||
`--once` is weaker — runner-side only.
|
||||
→ *Consequence:* `--ephemeral` is mandatory and `--once` is never used. The
|
||||
"one VM, one job" guarantee in §6 rests on the server enforcing it, not on the
|
||||
runner cooperating. Download URLs and the binary name must reference
|
||||
`gitea-runner`, not `act_runner`.
|
||||
|
||||
**4. Registration tokens are REUSABLE, and minting a new token for a scope
|
||||
INVALIDATES prior tokens of that scope.** `POST
|
||||
/api/v1/admin/actions/runners/registration-token` in practice returns the
|
||||
existing active token. Seeding server-side via `GITEA_RUNNER_REGISTRATION_TOKEN`
|
||||
is the recommended deployment.
|
||||
→ *Consequence:* **never pre-generate a token per VM** — doing so would break
|
||||
every other runner registered against that scope. One token is resolved once,
|
||||
cached for the process lifetime, and shared by the fleet;
|
||||
`fetchRegistrationTokenViaAPI` defaults to `false`.
|
||||
|
||||
**5. Label syntax is `name:schema`, schema defaults to `host`, and only BARE
|
||||
names are stored server-side.** If the guest's runner `config.yaml` sets
|
||||
`runner.labels`, it **silently overrides** `--labels` passed at registration.
|
||||
→ *Consequence:* `LabelSet` matches bare names (`macos-arm64`) and only appends
|
||||
`:host` when building the `register --labels` argument. The guest must **never**
|
||||
ship a `config.yaml` containing labels — noted in both `GuestProvisioner` and
|
||||
`Resources/provision.sh`, because the failure is silent: the runner registers
|
||||
successfully and is simply never matched.
|
||||
|
||||
**6. Guest requirements: the `gitea-runner` binary, `node`, `git`, `bash`, and a
|
||||
writable `$HOME`.** JavaScript actions such as `actions/checkout` spawn `node`
|
||||
**directly**.
|
||||
→ *Consequence:* Node.js installation is **not optional** and not a convenience —
|
||||
without it essentially every real workflow fails at its first step.
|
||||
`GuestProvisioner.verifyToolchain` fails the build rather than shipping an image
|
||||
that will break at job time.
|
||||
|
||||
**7. An unclean VM death leaves both a runner row and a Running task behind.**
|
||||
The task is reaped by Gitea's zombie sweep in roughly 10–15 minutes. The runner
|
||||
row is swept only at midnight — and **never at all** if that runner claimed no
|
||||
task. Rows are removed with `DELETE /api/v1/admin/actions/runners/{id}`.
|
||||
→ *Consequence:* the reconcile loop (§5) is load-bearing, not housekeeping.
|
||||
Without it, every crashed boot leaves a permanent phantom runner. It is also why
|
||||
every VM registers under a **globally unique** name (`namePrefix` + UUID): that
|
||||
uniqueness is what lets us look at a row and decide with certainty that it is
|
||||
ours and unbacked.
|
||||
|
||||
**8. macOS guests are capped at 2 concurrent per host, enforced by the kernel.**
|
||||
A third `start()` raises `VZError.virtualMachineLimitExceeded`.
|
||||
→ *Consequence:* `maxConcurrentVMs` is hard-clamped to 2 in both
|
||||
`RunnerConfig.validated()` and `SchedulerCore.plan`; the slot table is fixed-size;
|
||||
and the error is mapped to `CoreError.vmLimitExceeded` and treated as transient
|
||||
back-pressure rather than a failure.
|
||||
|
||||
**9. `VZMacGuestProvisioningOptions` (macOS 27+ host AND guest) automates Setup
|
||||
Assistant**, including `enablesRemoteLogin` (SSH). Older guests **silently
|
||||
ignore** it.
|
||||
→ *Consequence:* the API is gated at `#available(macOS 27.0, *)`, and because the
|
||||
guest-side failure produces no error, `firstBootAndProvision` must translate its
|
||||
lease/SSH timeout into an explicit "guest too old" message rather than a bare
|
||||
timeout. The `--manual-setup` fallback is out of v1 scope (§4).
|
||||
|
||||
**10. Headless Virtualization requires an `NSApplication` run loop with
|
||||
`.prohibited` activation policy, inside a signed `.app` bundle** carrying
|
||||
`com.apple.security.virtualization`. Ad-hoc signing (`codesign -s -`) suffices.
|
||||
Bridged networking would additionally need a restricted entitlement; NAT does
|
||||
not.
|
||||
→ *Consequence:* `CommandDaemon` starts `NSApplication` and runs the orchestrator
|
||||
in a detached `Task`. This applies to **every** command that starts a VM, not
|
||||
just the daemon: `vm boot`, `image build`, and `image provision` all go through
|
||||
the same `VZAppRuntime.run` host, since `VZMacOSInstaller` and the first-boot
|
||||
provisioning pass need the run loop exactly as much as a job VM does. The
|
||||
`Makefile` has `bundle` and `sign` targets and
|
||||
`install` deliberately installs the bundle rather than the bare binary;
|
||||
`Info.plist` sets `LSUIElement`; `Doctor` checks the entitlement on the running
|
||||
binary because running from `.build/` is the most common setup failure.
|
||||
Two packaging constraints follow from the same fact. The entitlements plist must
|
||||
contain **no XML comments**: `plutil -lint` accepts them, but `codesign` hands
|
||||
the file to AMFI's stricter parser, which fails with `AMFIUnserializeXML: syntax
|
||||
error` and then signs the bundle with *zero* entitlements — a silent
|
||||
downgrade that only surfaces as a failed VM start. And `bundle` must copy
|
||||
`provision.sh`, `launchd.plist.template`, and `config.example.json` into
|
||||
`Contents/Resources`, since `GuestProvisioner`, `LaunchdService`, and
|
||||
`config init` look there before falling back to repo-relative paths; an installed
|
||||
`.app` without them is a working binary with a broken `image build`,
|
||||
`service install`, and `config init`.
|
||||
|
||||
**11. macOS 15+ requires an unlocked `login.keychain` to start a VM.**
|
||||
→ *Consequence:* the service **must** be a LaunchAgent in the auto-logged-in
|
||||
user's session, never a LaunchDaemon (which has no session and no unlocked
|
||||
keychain). `LaunchdService` only ever writes to `~/Library/LaunchAgents`, and
|
||||
`Doctor` probes with `security show-keychain-info login.keychain`.
|
||||
|
||||
**12. Guest IPs come from parsing `/var/db/dhcpd_leases`, keyed by MAC.**
|
||||
`hw_address` lines carry a `1,` hardware-type prefix and octets that may lack
|
||||
zero-padding (`aa:bb:c:dd:ee:ff`). Duplicate MACs occur; the newest lease wins.
|
||||
macOS's DHCP lease time is 24 hours.
|
||||
→ *Consequence:* `DHCPLeaseParser.normalizeMAC` must strip the prefix and
|
||||
zero-pad, or lookups silently fail against `VZMACAddress.string`. And ephemeral
|
||||
fleets must **not** randomize MACs per clone — a day of dead leases would
|
||||
accumulate and exhaust the NAT subnet. Hence exactly two **persistent per-slot
|
||||
MACs**, generated once with `VZMACAddress.randomLocallyAdministered()` and stored
|
||||
in `state.json`.
|
||||
|
||||
**13. APFS copy-on-write cloning via `FileManager.copyItem` requires source and
|
||||
destination on the same volume.** Clones grow as the guest writes.
|
||||
→ *Consequence:* base images and ephemeral clones both live under `storeDir`, and
|
||||
cloning is done **per file** rather than by copying a directory wholesale.
|
||||
`ensureFreeSpace(minGB:)` runs before every clone, with a floor (default 20 GB)
|
||||
well above one clone's nominal cost, because the apparent size and the real cost
|
||||
diverge over a job's lifetime.
|
||||
|
||||
**14. The ASIF sparse disk format is created via `diskutil image create` (macOS
|
||||
26+); RAW is the fallback.**
|
||||
→ *Consequence:* `ImageBuilder.createDisk` shells out to
|
||||
`/usr/sbin/diskutil image create blank --fs none --format ASIF --size <N>G <path>`
|
||||
and falls back to a sparse RAW file, recording which was used in
|
||||
`VMBundleConfig.diskFormat` so `VZConfigFactory` attaches the right file without
|
||||
re-probing. This is also the reason the package's minimum platform is macOS 26.
|
||||
|
||||
**15. Save/restore (macOS 14+) could give near-instant warm boots, but forbids
|
||||
changing the MAC address or ECID.**
|
||||
→ *Consequence:* documented as future work (§9) rather than implemented. The
|
||||
prohibition collides directly with the per-slot MAC scheme from Fact 12, so
|
||||
adopting it would require per-slot saved states and a careful look at DHCP lease
|
||||
reuse — not a drop-in optimization.
|
||||
@@ -0,0 +1,150 @@
|
||||
# Security model
|
||||
|
||||
This document states what `gitea-macos-runner` protects, what it does not, and the operational
|
||||
choices that follow. Read it before pointing the runner at an instance where untrusted code can
|
||||
open a pull request.
|
||||
|
||||
## Summary
|
||||
|
||||
Workflow code runs **as the guest admin user, with passwordless sudo, and with no container
|
||||
isolation**. The only isolation boundary is the virtual machine, and that VM is destroyed after a
|
||||
single job. This is the same trust posture as GitHub's hosted macOS runners: the job owns the
|
||||
machine, and the machine is thrown away.
|
||||
|
||||
## Trust boundary
|
||||
|
||||
```
|
||||
┌─ host Mac ─────────────────────────────────────────────┐
|
||||
│ daemon, admin PAT, registration token, base images │ ← trusted
|
||||
│ ┌─ ephemeral VM ─────────────────────────────────┐ │
|
||||
│ │ gitea-runner + workflow code (root-capable) │ │ ← untrusted
|
||||
│ └────────────────────────────────────────────────┘ │
|
||||
└────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**The VM is the boundary.** Everything inside it is treated as untrusted and disposable;
|
||||
everything outside it is trusted. A guest that escapes Virtualization.framework and compromises the
|
||||
host is **out of scope** — the design assumes Apple's hypervisor holds. If your threat model
|
||||
includes hypervisor escape, this tool is not sufficient on its own; isolate the host at the network
|
||||
level and treat it as a machine that may be compromised.
|
||||
|
||||
## What runs where
|
||||
|
||||
- **In the guest:** the `gitea-runner` binary and every step of the workflow. The runner registers
|
||||
with `--labels "macos-arm64:host"` — the `:host` execution mode means steps run directly on the
|
||||
guest OS rather than inside a container. There is no second layer of isolation, by design;
|
||||
container isolation is not meaningfully available for macOS builds, and it would defeat the point
|
||||
of a real macOS environment.
|
||||
- **On the host:** the daemon, the Gitea admin PAT, the registration token, base images, and the
|
||||
clone/boot/destroy lifecycle. **The admin PAT is never copied into a guest.**
|
||||
|
||||
Because the guest admin has passwordless sudo, a job can install software, load kernel extensions
|
||||
the guest permits, read every file in the image, and reconfigure the guest arbitrarily. None of
|
||||
that persists: the clone is deleted when the job ends and the next job starts from the untouched
|
||||
base image. Nothing a job writes is visible to any later job.
|
||||
|
||||
## Ephemeral registration is server-enforced
|
||||
|
||||
Each VM registers a fresh runner with `--ephemeral`. Ephemerality is enforced **by the Gitea
|
||||
server**, not by the runner or by this daemon:
|
||||
|
||||
- After the runner accepts one job, Gitea **refuses to dispatch a second job** to that
|
||||
registration.
|
||||
- Gitea **deletes the registration** once that job completes.
|
||||
|
||||
The security consequence matters: a runner token exfiltrated from inside a running job cannot be
|
||||
used to fetch additional jobs, because the server has already spent that registration. The worst an
|
||||
attacker can do with it is nothing. This is why a compromised job does not become a persistent
|
||||
foothold in your CI queue — the compromised VM is destroyed and its credential is already dead.
|
||||
|
||||
## The shared registration token and its blast radius
|
||||
|
||||
The registration token is a different matter, and it is the sharpest edge in this design.
|
||||
|
||||
Gitea's registration tokens are **reusable by construction**, and creating a new token for a scope
|
||||
**invalidates the previous one**. There is no API for minting a single-use, per-VM registration
|
||||
token. A per-VM token is therefore impossible — not merely unimplemented.
|
||||
|
||||
So every VM presents the same registration token, and that token is necessarily present inside an
|
||||
untrusted guest for the duration of registration.
|
||||
|
||||
**Blast radius if the token leaks:** an attacker can register arbitrary runners against the scope
|
||||
the token covers (instance-wide, if you seeded `GITEA_RUNNER_REGISTRATION_TOKEN` on the server).
|
||||
Such a runner can advertise your labels and thereby **receive and execute jobs**, which means it
|
||||
can read whatever secrets those jobs are given and return forged results. It does not by itself
|
||||
grant API access to Gitea, read repositories the runner is not assigned jobs from, or confer admin
|
||||
rights.
|
||||
|
||||
Mitigations, in order of effectiveness:
|
||||
|
||||
1. **Constrain who can dispatch jobs to the label** (below) so a rogue runner has a small pool of
|
||||
jobs to intercept.
|
||||
2. **Register at org or repo scope** rather than instance-wide when only a few repos need macOS.
|
||||
The token's reach is then limited to that scope.
|
||||
3. **Rotate the token** by changing `GITEA_RUNNER_REGISTRATION_TOKEN` and restarting Gitea. Update
|
||||
`gitea.registrationTokenFile` on the host at the same time; in-flight VMs that have already
|
||||
registered are unaffected.
|
||||
4. **Keep secrets out of macOS jobs where possible.** Prefer short-lived, narrowly scoped
|
||||
credentials injected per job over long-lived org-level secrets.
|
||||
|
||||
## Limiting who can use the runner
|
||||
|
||||
By default an instance-wide runner will execute any job from any repo that writes
|
||||
`runs-on: macos-arm64`. On an instance where untrusted users can push branches or open PRs that
|
||||
trigger workflows, that is effectively arbitrary code execution on your CI Mac's VM.
|
||||
|
||||
Restrict it:
|
||||
|
||||
- **Register at repo or org scope** instead of instance-wide — the runner is then only offered jobs
|
||||
from that repo or org.
|
||||
- Use Gitea's per-repo and per-org **Actions runner settings** to control which repositories may
|
||||
use a shared runner.
|
||||
- Configure Gitea so that **workflows from forked-repository pull requests require approval** before
|
||||
running. This is the single most important setting if your instance accepts outside
|
||||
contributions.
|
||||
|
||||
Decide this deliberately. Nothing in the daemon restricts job origin; that control lives entirely
|
||||
in Gitea.
|
||||
|
||||
## Secrets hygiene
|
||||
|
||||
- **Prefer the file forms of every credential:** `gitea.adminTokenFile` and
|
||||
`gitea.registrationTokenFile` rather than `adminToken`/`registrationToken` inline in
|
||||
`config.json`. For the admin token this is not merely a preference in one direction: config
|
||||
validation requires **exactly one** of `adminToken` and `adminTokenFile`, so a stale inline token
|
||||
cannot sit unnoticed beside a live token file.
|
||||
- **`chmod 600` every token file** and keep it owned by the runner user:
|
||||
|
||||
```sh
|
||||
chmod 600 ~/.config/gitea-macos-runner/admin-token \
|
||||
~/.config/gitea-macos-runner/registration-token
|
||||
```
|
||||
|
||||
`gitea-macos-runner doctor` checks this: its `token file permissions` check warns, and prints the
|
||||
exact `chmod` to run, if any configured token file is group- or world-readable.
|
||||
|
||||
- **Token *files* keep secrets off `ps`.** Any local user can read another process's argument
|
||||
vector; a token passed as `--token …` is visible there for the life of the process, and often in
|
||||
shell history and logs too. The `--token-file` form passes a path instead, so the secret never
|
||||
enters the command line.
|
||||
- **Do not commit `config.json`** or any token file. If your config lives in a dotfiles repo, keep
|
||||
the token files outside it.
|
||||
- **The `guest.password`** is a credential for a throwaway machine, but it appears in the host's
|
||||
config file and grants SSH into running guests. Treat the config file itself as sensitive
|
||||
(`chmod 600`) and do not reuse a password from anywhere else.
|
||||
- **Rotate the admin PAT** on the normal schedule you use for admin credentials. Its scope
|
||||
(`read:admin` + `write:admin`) is genuinely powerful — it can enumerate and delete runner
|
||||
registrations instance-wide — so it is the highest-value secret on the host. It lives only on the
|
||||
host and never enters a VM; keep it that way.
|
||||
|
||||
## Host hardening notes
|
||||
|
||||
- Use a **dedicated Mac** for CI. The auto-login recommendation in
|
||||
[setup.md](setup.md#25-install-the-service) means the disk is unlocked at boot, which is
|
||||
acceptable for a purpose-built CI machine and not for a workstation holding other data.
|
||||
- Run the daemon as a **non-administrator user** where practical. It needs a GUI session and
|
||||
virtualization entitlements, not host root.
|
||||
- Place the Mac on a **network segment that cannot reach production**. Guests get NAT'd egress
|
||||
through the host; anything the host can reach, a job can reach.
|
||||
- Keep the base image current. It is rebuilt from an IPSW, so refreshing macOS and the toolchain is
|
||||
an `image build` away — and every job automatically picks up the new image.
|
||||
+479
@@ -0,0 +1,479 @@
|
||||
# Setup
|
||||
|
||||
This walkthrough covers everything needed to go from a bare Apple Silicon Mac and a self-hosted
|
||||
Gitea instance to a working macOS CI runner: Gitea-side configuration, host build and signing,
|
||||
base image creation, service installation, and verification.
|
||||
|
||||
Work through it in order. The Gitea side can be done from any machine; the host side must be done
|
||||
on the Mac that will run the VMs.
|
||||
|
||||
---
|
||||
|
||||
## 1. Gitea-side configuration
|
||||
|
||||
### 1.1 Version requirements
|
||||
|
||||
| Component | Minimum | Notes |
|
||||
| --- | --- | --- |
|
||||
| Gitea server | **1.25** | 1.25 added `GET /api/v1/admin/actions/jobs`, including the `labels` field that carries the job's `runs-on`. The daemon cannot work without it. |
|
||||
| Gitea server | 1.26+ recommended | Later fixes to Actions job dispatch and runner cleanup. |
|
||||
| Runner binary in the guest | **`gitea-runner` v3.x** | This is Gitea's own runner. It is *not* the older `act_runner`; do not substitute it. |
|
||||
|
||||
Actions must be enabled on the instance (`[actions] ENABLED = true` in `app.ini`) and on any repo
|
||||
that will use the runner.
|
||||
|
||||
### 1.2 Create the admin token (PAT)
|
||||
|
||||
The daemon needs a personal access token belonging to a **site administrator** so it can read the
|
||||
queued-jobs list and delete stale runner registrations.
|
||||
|
||||
1. Sign in as a site-admin user.
|
||||
2. **Settings → Applications → Manage Access Tokens → Generate New Token.**
|
||||
3. Grant the token **`read:admin` and `write:admin`** scopes. Nothing else is required.
|
||||
4. Copy the token immediately — Gitea shows it once.
|
||||
|
||||
Store it on the host in a file readable only by the runner user rather than inline in the config:
|
||||
|
||||
```sh
|
||||
install -m 600 /dev/null ~/.config/gitea-macos-runner/admin-token
|
||||
printf '%s' 'PASTE_TOKEN_HERE' > ~/.config/gitea-macos-runner/admin-token
|
||||
```
|
||||
|
||||
Then set `gitea.adminTokenFile` to that path. See [security.md](security.md) for why the file form
|
||||
is preferred.
|
||||
|
||||
### 1.3 Choose a registration token strategy
|
||||
|
||||
Every VM registers itself as a runner before it can accept a job, and registration requires a
|
||||
registration token. Gitea's registration tokens are **reusable**, and creating a new token for a
|
||||
given scope **invalidates the previous one** — so a distinct token per VM is impossible by design.
|
||||
You have two options.
|
||||
|
||||
**Option A (recommended): seed a stable instance-wide token on the Gitea server.**
|
||||
|
||||
Set an environment variable on the Gitea server process before it starts:
|
||||
|
||||
```sh
|
||||
# Must be at least 32 characters.
|
||||
GITEA_RUNNER_REGISTRATION_TOKEN=<32+ character random string>
|
||||
```
|
||||
|
||||
For example, in a systemd unit:
|
||||
|
||||
```ini
|
||||
[Service]
|
||||
Environment=GITEA_RUNNER_REGISTRATION_TOKEN=<32+ character random string>
|
||||
```
|
||||
|
||||
or in `docker-compose.yml`:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
gitea:
|
||||
environment:
|
||||
- GITEA_RUNNER_REGISTRATION_TOKEN=<32+ character random string>
|
||||
```
|
||||
|
||||
Restart Gitea, then put the same value on the host in `gitea.registrationTokenFile`. This token is
|
||||
stable across restarts and is not invalidated by someone clicking "create new token" in the UI for
|
||||
a different scope.
|
||||
|
||||
**Option B: let the daemon fetch a token via the admin API.**
|
||||
|
||||
Set `gitea.fetchRegistrationTokenViaAPI: true` and omit `gitea.registrationToken`/`…File`. The
|
||||
daemon requests a token with the admin PAT when it needs one. This is simpler to set up, but any
|
||||
out-of-band token creation for the same scope invalidates the token the daemon is holding, and the
|
||||
daemon must re-fetch. Option A is more predictable for an unattended host.
|
||||
|
||||
Generate a token value with:
|
||||
|
||||
```sh
|
||||
openssl rand -hex 24
|
||||
```
|
||||
|
||||
### 1.4 Target the runner from a workflow
|
||||
|
||||
```yaml
|
||||
# .gitea/workflows/macos.yml
|
||||
name: macOS build
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: macos-arm64 # bare label name — must match runner.labels
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Show host
|
||||
run: sw_vers && uname -m
|
||||
- name: Build
|
||||
run: swift build
|
||||
```
|
||||
|
||||
Two things to know about labels:
|
||||
|
||||
- Use **bare label names** in `runs-on`. The runner registers with `--labels "macos-arm64:host"`;
|
||||
the `:host` suffix is runner-side only and declares that the label executes directly on the host
|
||||
OS rather than in a container. It never appears in workflow YAML.
|
||||
- Every label in `runs-on` must be present in the runner's `runner.labels`. A typo means the job
|
||||
sits queued forever with no error.
|
||||
|
||||
Because no runner exists until a job appears, jobs necessarily wait for a VM to boot. Gitea holds a
|
||||
queued job for `ABANDONED_JOB_TIMEOUT` (default **24 hours**) before marking it abandoned, so
|
||||
scale-from-zero is safe. If your queue can legitimately back up for more than a day — a long
|
||||
maintenance window, for example — raise that setting:
|
||||
|
||||
```ini
|
||||
[actions]
|
||||
ABANDONED_JOB_TIMEOUT = 72h
|
||||
```
|
||||
|
||||
### 1.5 Optionally restrict which repos may use the runner
|
||||
|
||||
The runner is registered instance-wide by default, meaning any repo whose workflow says
|
||||
`runs-on: macos-arm64` can execute code on it. If that is broader than you want, register the
|
||||
runner at the org or repo level instead, or restrict via Gitea's runner settings. See
|
||||
[security.md](security.md#limiting-who-can-use-the-runner).
|
||||
|
||||
---
|
||||
|
||||
## 2. Host-side setup
|
||||
|
||||
### 2.1 Host requirements
|
||||
|
||||
| Requirement | Value |
|
||||
| --- | --- |
|
||||
| Architecture | Apple Silicon (arm64). Intel Macs cannot run macOS guests. |
|
||||
| Host macOS | 26 minimum; **27+ strongly recommended** (see below). |
|
||||
| Guest macOS | 27+ if you want unattended image builds. |
|
||||
| Free disk | ~60 GB for a vanilla image; **140 GB+** with Xcode installed. |
|
||||
| RAM | 16 GB minimum; each guest defaults to 8 GB. |
|
||||
| Concurrent VMs | **2 maximum**, enforced by the macOS kernel. `scheduler.maxConcurrentVMs` must be ≤ 2. |
|
||||
|
||||
The macOS 27 recommendation is not cosmetic. The automated image builder uses
|
||||
`VZMacGuestProvisioningOptions` — introduced in macOS 27 — to create the admin account and skip
|
||||
Setup Assistant during first boot. Both host and guest must be 27+. An older guest **silently
|
||||
ignores** the options: the install succeeds, the VM boots, and then sits at Setup Assistant with no
|
||||
SSH server, so the build appears to hang. See
|
||||
[troubleshooting.md](troubleshooting.md).
|
||||
|
||||
### 2.2 Build, sign, and install
|
||||
|
||||
Virtualization.framework refuses to run unless the calling binary carries the
|
||||
`com.apple.security.virtualization` entitlement, and entitlements are only honoured on a signed
|
||||
binary inside a proper `.app` bundle. Ad-hoc signing (`codesign -s -`) satisfies this — **no paid
|
||||
Apple developer account is needed.**
|
||||
|
||||
```sh
|
||||
git clone <this repo> && cd gitea-macos-runner
|
||||
make install
|
||||
```
|
||||
|
||||
`make install` runs the full chain and places the result:
|
||||
|
||||
| Target | What it does |
|
||||
| --- | --- |
|
||||
| `make build` | `swift build -c release --arch arm64` |
|
||||
| `make bundle` | Assemble `GiteaMacosRunner.app` around the binary: `Contents/MacOS/gitea-macos-runner`, `Contents/Info.plist`, and `Contents/Resources/` (`provision.sh`, `launchd.plist.template`, `config.example.json`) |
|
||||
| `make sign` | `codesign --sign - --entitlements …` (ad-hoc) and print the resulting entitlements |
|
||||
| `make all` | `build` + `bundle` + `sign`. The default target. |
|
||||
| `make install` | Runs `all`, copies the app to `~/Applications`, and symlinks the CLI to `/usr/local/bin/gitea-macos-runner` |
|
||||
| `make dev` | Debug build + bundle + sign, for fast iteration. Does **not** install. |
|
||||
| `make test` | `swift test` |
|
||||
| `make uninstall` | Remove the installed app and symlink |
|
||||
| `make clean` | Remove `.build/` |
|
||||
| `make help` | List the targets |
|
||||
|
||||
The three files under `Contents/Resources/` are not decoration. `image build`
|
||||
uploads `provision.sh` into the guest, `service install` renders
|
||||
`launchd.plist.template`, and `config init` writes `config.example.json`. The
|
||||
code looks in `Contents/Resources` first and only then falls back to
|
||||
repo-relative paths, so an installed `.app` missing them is a working binary
|
||||
with three broken commands.
|
||||
|
||||
If `/usr/local/bin` isn't writable, `make install` says so and prints the `sudo ln -sf …` command to
|
||||
run yourself.
|
||||
|
||||
Never run the raw binary from `.build/release/` — it is outside the signed bundle, so every VM
|
||||
operation fails with an entitlement error. Always invoke the symlink (or
|
||||
`~/Applications/GiteaMacosRunner.app/Contents/MacOS/gitea-macos-runner`).
|
||||
|
||||
### 2.3 Create the configuration file
|
||||
|
||||
```sh
|
||||
gitea-macos-runner config init # add --force to overwrite an existing file
|
||||
$EDITOR "$(gitea-macos-runner config path)"
|
||||
gitea-macos-runner config show # parse + validate, printing secrets redacted
|
||||
```
|
||||
|
||||
`config init` writes the annotated example shipped in the app bundle, so the
|
||||
file you land in has a `_comment` key explaining each section; those keys are
|
||||
ignored when the config is read back. Pass `--instance-url https://…` to seed
|
||||
`gitea.instanceURL` instead of the placeholder. Every subcommand takes
|
||||
`--config PATH` (`-c`) if you keep the file somewhere else, and `--verbose`.
|
||||
|
||||
A minimal working config:
|
||||
|
||||
```json
|
||||
{
|
||||
"gitea": {
|
||||
"instanceURL": "https://gitea.example.com",
|
||||
"adminTokenFile": "~/.config/gitea-macos-runner/admin-token",
|
||||
"registrationTokenFile": "~/.config/gitea-macos-runner/registration-token"
|
||||
},
|
||||
"runner": {
|
||||
"labels": ["macos-arm64"],
|
||||
"namePrefix": "macos-vm-"
|
||||
},
|
||||
"scheduler": {
|
||||
"maxConcurrentVMs": 2
|
||||
},
|
||||
"guest": {
|
||||
"username": "admin",
|
||||
"password": "CHANGE_ME",
|
||||
"cpuCount": 4,
|
||||
"memoryGB": 8,
|
||||
"diskGB": 64
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Configuration reference
|
||||
|
||||
**`gitea`**
|
||||
|
||||
| Key | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `instanceURL` | — (required) | Base URL of the Gitea instance, e.g. `https://gitea.example.com`. Must be `http://` or `https://` with a host. A trailing slash is harmless; a subpath is preserved. |
|
||||
| `adminToken` | — | Site-admin PAT with `read:admin` + `write:admin`, inline. |
|
||||
| `adminTokenFile` | — | Path to a file containing the PAT (tilde-expanded). Should be `chmod 600`. |
|
||||
| `registrationToken` | — | Runner registration token, inline. Prefer `registrationTokenFile`. |
|
||||
| `registrationTokenFile` | — | Path to a file containing the registration token (tilde-expanded). Takes precedence over `registrationToken` if both are set. |
|
||||
| `fetchRegistrationTokenViaAPI` | `false` | Fetch a registration token with the admin PAT when no static one is configured. A *fallback*, not an override: a configured token still wins. |
|
||||
|
||||
Set **exactly one** of `adminToken` and `adminTokenFile`. Setting both is a
|
||||
validation error — a stale inline token next to a live token file is precisely
|
||||
the ambiguity that turns into a baffling 401 later — and setting neither is too.
|
||||
|
||||
For the registration token the rule is looser: configure `registrationToken`,
|
||||
`registrationTokenFile`, or `fetchRegistrationTokenViaAPI: true`. At least one
|
||||
is required; the file form wins over the inline form when both are present.
|
||||
|
||||
**`runner`**
|
||||
|
||||
| Key | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `labels` | `["macos-arm64"]` | Labels this runner offers. A queued job runs here only if its `runs-on` labels are all in this set. **Bare names only** — a `:schema` suffix here is a validation error; the `:host` suffix is appended automatically at registration. |
|
||||
| `namePrefix` | `"macos-vm-"` | Prefix for generated runner names in the Gitea UI; a unique suffix is appended per VM. |
|
||||
| `runnerDownloadURL` | `https://gitea.com/gitea/runner/releases/download/v{version}/gitea-runner-{version}-darwin-arm64` | Release asset installed into the guest. `{version}` is substituted with `version`. |
|
||||
| `version` | `"3.0.2"` | The `gitea-runner` version to install. |
|
||||
|
||||
**`scheduler`**
|
||||
|
||||
| Key | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `maxConcurrentVMs` | `2` | Simultaneous VMs. **Hard-capped at 2 by macOS.** A larger value is silently clamped to 2 at load rather than rejected; the kernel is not something a config file gets to negotiate. |
|
||||
| `pollIntervalSeconds` | `5` | How often to poll the queued-jobs API. |
|
||||
| `reconcileIntervalSeconds` | `300` | How often to sweep Gitea for orphaned runner registrations from uncleanly-killed VMs. |
|
||||
| `jobTimeoutMinutes` | `120` | Wall-clock limit for one job; the VM is destroyed when exceeded. |
|
||||
| `bootTimeoutSeconds` | `300` | Time allowed from VM start to a usable SSH connection. |
|
||||
|
||||
**`guest`**
|
||||
|
||||
| Key | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `username` | `"admin"` | Guest admin account created during image build. Has passwordless sudo. |
|
||||
| `password` | — (required) | Password for that account. Also used for SSH if key auth is unavailable. |
|
||||
| `cpuCount` | `4` | vCPUs per guest. |
|
||||
| `memoryGB` | `8` | RAM per guest. Two concurrent guests at 8 GB need a 16 GB+ host with headroom. |
|
||||
| `diskGB` | `64` | Guest disk size. Sparse, so this is a ceiling, not immediate consumption. Raise for Xcode. |
|
||||
|
||||
**`storage`**
|
||||
|
||||
| Key | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `storeDir` | `~/Library/Application Support/gitea-macos-runner` | Base images (`images/`) and running clones (`vms/`) live here. |
|
||||
| `minFreeDiskGB` | `20` | The scheduler refuses to start a VM below this free-space threshold rather than filling the disk mid-job. |
|
||||
|
||||
### 2.4 Build the base image
|
||||
|
||||
Download a macOS **27 or newer** IPSW for Apple Silicon (Apple's restore images; the URL for the
|
||||
current release is also discoverable from Virtualization.framework's latest-supported-restore-image
|
||||
endpoint), then:
|
||||
|
||||
```sh
|
||||
gitea-macos-runner image build \
|
||||
--ipsw ~/Downloads/UniversalMac_27.0_XXXXX_Restore.ipsw \
|
||||
--disk-gb 64
|
||||
```
|
||||
|
||||
`--name` defaults to `default`, which is also what `daemon` and `vm boot` look
|
||||
for. If you name the image something else, pass the same name to
|
||||
`daemon --image NAME` (and to `vm boot --image NAME`) or the daemon will not
|
||||
find it. `--ipsw` is optional: omit it and the latest supported restore image is
|
||||
downloaded into `storeDir/ipsw/` first, which is most of the build's wall-clock
|
||||
time. `--disk-gb` overrides `guest.diskGB` for this image only.
|
||||
|
||||
This takes a long time — macOS installs from the IPSW, boots, and is then provisioned over SSH.
|
||||
**Do not interrupt it during the install phase.** Stopping a VM mid-install leaves the disk image
|
||||
in an undefined state; delete the image and start over rather than trying to resume.
|
||||
|
||||
The finished image contains:
|
||||
|
||||
- macOS installed from the IPSW.
|
||||
- The `guest.username` admin account, auto-created via provisioning options, with SSH (Remote
|
||||
Login) enabled and **passwordless sudo**.
|
||||
- Sleep, screensaver, and Spotlight indexing disabled — a sleeping guest stalls a job, and Spotlight
|
||||
wastes I/O on a throwaway machine.
|
||||
- Raised file-descriptor limits, which Node- and Xcode-based builds routinely exhaust at the
|
||||
default.
|
||||
- **Node.js — required, not optional.** Gitea Actions' JavaScript actions (`actions/checkout` and
|
||||
most of the ecosystem) are executed by spawning `node` in the guest. Without it, every workflow
|
||||
fails on its first step.
|
||||
- `git`, verified present.
|
||||
- The `gitea-runner` v3.x binary.
|
||||
|
||||
Verify:
|
||||
|
||||
```sh
|
||||
gitea-macos-runner image list
|
||||
```
|
||||
|
||||
#### Adding Xcode (optional)
|
||||
|
||||
Xcode is not installed automatically. Apple requires an authenticated Apple ID for the download and
|
||||
its download endpoints are impractical to drive unattended, so you fetch the `.xip` yourself from
|
||||
[developer.apple.com/download](https://developer.apple.com/download/) and hand it to the
|
||||
provisioner:
|
||||
|
||||
```sh
|
||||
gitea-macos-runner image provision default --xcode-xip ~/Downloads/Xcode_XX.xip
|
||||
```
|
||||
|
||||
Budget disk accordingly: **~60 GB free is enough for a vanilla image, but plan on 140 GB+ with
|
||||
Xcode**, and remember each running VM is a copy-on-write clone whose divergence from the base
|
||||
consumes additional space while it runs. Raise `guest.diskGB` (e.g. to 120) before building if the
|
||||
image will carry Xcode.
|
||||
|
||||
### 2.5 Install the service
|
||||
|
||||
```sh
|
||||
gitea-macos-runner service install
|
||||
gitea-macos-runner service status
|
||||
```
|
||||
|
||||
This installs a **LaunchAgent in the logged-in user's GUI session — not a LaunchDaemon.** Two
|
||||
reasons this is not negotiable:
|
||||
|
||||
1. Virtualization.framework requires a GUI session; a system-context daemon cannot start VMs.
|
||||
2. macOS 15 and later require an **unlocked `login.keychain`** for key operations the VM lifecycle
|
||||
performs. In a locked or headless session these fail with `SecKeyCreateRandomKey` /
|
||||
"Interaction is not allowed" errors.
|
||||
|
||||
**Recommended: enable auto-login for the runner user on a dedicated CI Mac.**
|
||||
**System Settings → Users & Groups → Automatically log in as → \<runner user\>.** Combine with
|
||||
disabling sleep (`sudo pmset -a sleep 0 disablesleep 1`) so the Mac comes back into a live session
|
||||
after a power event without a human present. This does mean the disk is effectively unlocked at
|
||||
boot — appropriate for a dedicated CI machine, not for a shared workstation.
|
||||
|
||||
`service uninstall` removes the LaunchAgent; it does not delete images or config.
|
||||
|
||||
### 2.6 macOS 15+ Local Network privacy prompt
|
||||
|
||||
Starting with macOS 15, a process that contacts other hosts on the local network triggers a
|
||||
one-time Local Network permission prompt. A LaunchAgent that is denied (or that never gets a human
|
||||
to click Allow) cannot reach the guest's NAT address, so VMs boot but SSH never connects.
|
||||
|
||||
Grant it interactively the first time — run `gitea-macos-runner vm boot` from a
|
||||
Terminal in the GUI session and click **Allow** — or pre-authorize the VM subnet:
|
||||
|
||||
```sh
|
||||
sudo defaults write com.apple.network.local-network \
|
||||
AllowedEthernetLocalNetworkAddresses -array "192.168.0.0/16"
|
||||
```
|
||||
|
||||
Adjust the range to match the subnet Virtualization.framework's NAT hands out on your host (check
|
||||
`/var/db/dhcpd_leases` after a VM boots). Reboot, or restart the service, for the change to take
|
||||
effect. Also confirm the runner is enabled under **System Settings → Privacy & Security → Local
|
||||
Network**.
|
||||
|
||||
---
|
||||
|
||||
## 3. Verification
|
||||
|
||||
### 3.1 Preflight
|
||||
|
||||
```sh
|
||||
gitea-macos-runner doctor
|
||||
```
|
||||
|
||||
It reports one line per check, each `✓` pass, `✗` fail, `!` warn, or `·` note,
|
||||
with a remediation hint under anything that is not a pass, and exits non-zero if
|
||||
anything failed. `--json` emits the same results machine-readably; `--no-fail`
|
||||
exits zero regardless, which is what you want when running it from a script that
|
||||
handles the results itself.
|
||||
|
||||
The checks, in order:
|
||||
|
||||
| Check | What it looks at |
|
||||
| --- | --- |
|
||||
| `host capability` | Apple Silicon, and host macOS ≥ 26 |
|
||||
| `Virtualization.framework` | `VZVirtualMachine.isSupported` |
|
||||
| `virtualization entitlement` | `com.apple.security.virtualization` on the *running* executable — this is the check that catches running from `.build/` instead of the signed `.app` |
|
||||
| `configuration` | The config file loads, parses, and passes validation |
|
||||
| `free disk space` | Free space on the `storeDir` volume against `storage.minFreeDiskGB` |
|
||||
| `login.keychain unlocked` | `security show-keychain-info login.keychain` |
|
||||
| `gitea admin token` / `gitea admin api` | The admin token resolves, and an admin-only endpoint answers with it (so a non-admin PAT fails here rather than at 3am) |
|
||||
| `registration token` | A static token resolves, or one can be fetched when `fetchRegistrationTokenViaAPI` is on |
|
||||
| `runner download url` | The `gitea-runner` release asset is reachable |
|
||||
| `token file permissions` | Warns — not fails — when a token file is group- or world-readable |
|
||||
| `local network access` | An informational note about the macOS 15+ Local Network prompt |
|
||||
|
||||
If the config file is missing or invalid, the host checks still run and the rest
|
||||
are skipped — which is exactly the state a first-time operator is in. Resolve
|
||||
everything it reports before going further.
|
||||
|
||||
There is deliberately **no** "a base image exists" check; use `image list`.
|
||||
|
||||
### 3.2 Boot a VM by hand
|
||||
|
||||
```sh
|
||||
gitea-macos-runner vm boot
|
||||
```
|
||||
|
||||
This clones the base image and boots it without registering a runner — the fastest way to confirm
|
||||
that virtualization, networking, and SSH all work. Once it is up, check that the guest got a lease:
|
||||
|
||||
```sh
|
||||
cat /var/db/dhcpd_leases
|
||||
```
|
||||
|
||||
and that you can reach it:
|
||||
|
||||
```sh
|
||||
ssh admin@<guest-ip> 'sw_vers; node --version; git --version; which gitea-runner'
|
||||
```
|
||||
|
||||
All four should answer. A guest at Setup Assistant instead of a login window means the image was
|
||||
built from a pre-27 IPSW; rebuild.
|
||||
|
||||
### 3.3 Watch a real job
|
||||
|
||||
Start the service, push the example workflow from §1.4, and watch both sides:
|
||||
|
||||
```sh
|
||||
# Host: daemon logs
|
||||
log stream --predicate 'process == "gitea-macos-runner"' --info
|
||||
|
||||
# Host: VM lifecycle
|
||||
gitea-macos-runner service status
|
||||
```
|
||||
|
||||
In the Gitea UI, the job should move from queued to running within roughly one poll interval plus
|
||||
boot time; a runner named `macos-vm-<suffix>` appears under **Site Administration → Actions →
|
||||
Runners** while the job runs and disappears when it finishes. That disappearance is Gitea deleting
|
||||
the ephemeral registration itself, and it is the signal that the whole loop worked.
|
||||
|
||||
If the job stays queued, or the runner appears but the job fails immediately, go to
|
||||
[troubleshooting.md](troubleshooting.md).
|
||||
@@ -0,0 +1,314 @@
|
||||
# Troubleshooting
|
||||
|
||||
Start with `gitea-macos-runner doctor` — it catches most misconfiguration before you go
|
||||
symptom-hunting. Then find your symptom below.
|
||||
|
||||
Useful log commands throughout:
|
||||
|
||||
```sh
|
||||
# Daemon logs, live
|
||||
log stream --predicate 'process == "gitea-macos-runner"' --info
|
||||
|
||||
# Daemon logs, last hour
|
||||
log show --predicate 'process == "gitea-macos-runner"' --info --last 1h
|
||||
|
||||
gitea-macos-runner service status
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick reference
|
||||
|
||||
| Symptom | Cause | Fix |
|
||||
| --- | --- | --- |
|
||||
| VM won't start; entitlement / `com.apple.security.virtualization` error | Running an unsigned binary, or one outside the signed `.app` bundle | `make sign` (or re-run `make install`); invoke the installed bundle, never `.build/release/…` |
|
||||
| `virtualMachineLimitExceeded` at boot | macOS allows at most **2** concurrent macOS VMs | Set `scheduler.maxConcurrentVMs` ≤ 2; kill stray VMs from earlier runs |
|
||||
| VM boots but never gets an IP | DHCP lease not yet written, or Local Network privacy denial (macOS 15+) | Check `/var/db/dhcpd_leases`; grant Local Network permission or pre-authorize the subnet |
|
||||
| SSH times out on a freshly built image | Guest macOS < 27, so provisioning options were ignored and Setup Assistant is waiting | Rebuild the image from a macOS **27+** IPSW |
|
||||
| `SecKeyCreateRandomKey` / "Interaction is not allowed" | `login.keychain` is locked — no GUI session | Run as a LaunchAgent in an unlocked GUI session; enable auto-login |
|
||||
| Job stays queued forever | Label mismatch, or the daemon isn't running/reaching Gitea | Use bare label names in `runs-on`; match `runner.labels`; check daemon logs |
|
||||
| `actions/checkout` fails instantly | Node.js missing from the guest image | `gitea-macos-runner image provision <name>` |
|
||||
| Runner rows piling up in the Gitea UI | VMs killed uncleanly; registrations orphaned | Reconcile loop cleans them; force it by restarting the daemon; delete manually if needed |
|
||||
| Disk filling up | Copy-on-write clones grow as jobs write | Raise `storage.minFreeDiskGB`; delete stale clones in `storeDir/vms` |
|
||||
| `image build` appears to hang during install | Normal — macOS install is slow | Wait. **Do not stop the VM mid-install**; if you did, delete the image and rebuild |
|
||||
|
||||
---
|
||||
|
||||
## VM won't start — entitlement error
|
||||
|
||||
**Symptom.** Any VM operation fails immediately with an error naming
|
||||
`com.apple.security.virtualization`, or a generic "operation not permitted" from
|
||||
Virtualization.framework.
|
||||
|
||||
**Cause.** Virtualization.framework checks the entitlement on the calling binary, and entitlements
|
||||
are only honoured on a signed binary inside a proper `.app` bundle. The raw product of
|
||||
`swift build` has neither.
|
||||
|
||||
**Fix.**
|
||||
|
||||
```sh
|
||||
make install # build → bundle → sign → install (make sign alone re-signs in place)
|
||||
codesign -d --entitlements - ~/Applications/GiteaMacosRunner.app # verify
|
||||
```
|
||||
|
||||
The output must list `com.apple.security.virtualization`. Then confirm the command you're running
|
||||
resolves to the installed bundle's binary — `which -a gitea-macos-runner` should point at
|
||||
`~/Applications/GiteaMacosRunner.app/Contents/MacOS/gitea-macos-runner`, not at
|
||||
`.build/release/gitea-macos-runner`. Ad-hoc signing is sufficient; you do not need a paid developer
|
||||
account.
|
||||
|
||||
---
|
||||
|
||||
## `virtualMachineLimitExceeded`
|
||||
|
||||
**Symptom.** The first VM boots fine; a second or third fails with `virtualMachineLimitExceeded`.
|
||||
|
||||
**Cause.** macOS permits **two** concurrent macOS guests per host. This is an Apple kernel and
|
||||
licensing limit, not a resource constraint — more RAM will not raise it.
|
||||
|
||||
**Fix.** Set `scheduler.maxConcurrentVMs` to 2 or less. If you're already at 2 and still hitting
|
||||
the limit, a VM from a previous run is still alive — check for stray processes and for leftover
|
||||
directories under `storeDir/vms`, then restart the daemon so it starts from a clean state.
|
||||
|
||||
To handle more macOS jobs in parallel, add another Mac.
|
||||
|
||||
---
|
||||
|
||||
## VM starts but never gets an IP
|
||||
|
||||
**Symptom.** The VM boots (you can see it progress if you use `vm boot`), but the daemon reports
|
||||
that it could not resolve the guest address, or gives up at `scheduler.bootTimeoutSeconds`.
|
||||
|
||||
**Cause.** The daemon resolves the guest's NAT address from the host's DHCP lease file, which is
|
||||
only written once the guest requests a lease — several seconds after the boot screen appears. If the
|
||||
address never appears at all, the usual culprit on macOS 15+ is the **Local Network privacy
|
||||
prompt**: a LaunchAgent that was never granted permission (or was denied) cannot talk to the guest.
|
||||
|
||||
**Fix.**
|
||||
|
||||
1. Check the lease file after the guest has been up for ~30 seconds:
|
||||
|
||||
```sh
|
||||
cat /var/db/dhcpd_leases
|
||||
```
|
||||
|
||||
An entry with a recent `lease` timestamp and the guest's MAC means networking is fine and the
|
||||
problem is timing — raise `scheduler.bootTimeoutSeconds`.
|
||||
|
||||
2. No entry at all: check **System Settings → Privacy & Security → Local Network** and enable the
|
||||
runner. If it isn't listed, trigger the prompt interactively from a Terminal in the GUI session:
|
||||
|
||||
```sh
|
||||
gitea-macos-runner vm boot
|
||||
```
|
||||
|
||||
and click **Allow**.
|
||||
|
||||
3. Or pre-authorize the VM subnet, then reboot:
|
||||
|
||||
```sh
|
||||
sudo defaults write com.apple.network.local-network \
|
||||
AllowedEthernetLocalNetworkAddresses -array "192.168.0.0/16"
|
||||
```
|
||||
|
||||
Match the range to what your host's NAT actually hands out.
|
||||
|
||||
---
|
||||
|
||||
## SSH times out on a freshly built image
|
||||
|
||||
**Symptom.** The VM boots and gets an IP, but SSH never connects. Attaching a display to the guest
|
||||
shows **Setup Assistant** — the region/Apple ID welcome flow — rather than a login window.
|
||||
|
||||
**Cause.** The image builder uses `VZMacGuestProvisioningOptions` to create the admin account, enable
|
||||
Remote Login, and skip Setup Assistant. That API requires **macOS 27 or newer in the guest as well
|
||||
as the host**. An older guest **silently ignores** the options: no error, no account, no SSH server
|
||||
— it just sits at first-run setup forever.
|
||||
|
||||
**Fix.** Rebuild the base image from a macOS 27+ IPSW:
|
||||
|
||||
```sh
|
||||
gitea-macos-runner image delete default
|
||||
gitea-macos-runner image build --ipsw ~/Downloads/UniversalMac_27.0_XXXXX_Restore.ipsw
|
||||
```
|
||||
|
||||
Verify the host is also 27+ (`sw_vers`). There is no way to make a pre-27 guest work unattended
|
||||
with this builder.
|
||||
|
||||
---
|
||||
|
||||
## `SecKeyCreateRandomKey` / "Interaction is not allowed"
|
||||
|
||||
**Symptom.** The daemon starts but fails during VM setup with a Security-framework error mentioning
|
||||
`SecKeyCreateRandomKey`, `errSecInteractionNotAllowed`, or "Interaction is not allowed". Often it
|
||||
works when you run the daemon by hand in Terminal and fails under launchd.
|
||||
|
||||
**Cause.** The **`login.keychain` is locked.** macOS 15+ requires it unlocked for key operations the
|
||||
VM lifecycle performs, and it is only unlocked inside a live, logged-in GUI session. A LaunchDaemon,
|
||||
an SSH-only session, or a Mac sitting at the login window all fail this.
|
||||
|
||||
**Fix.**
|
||||
|
||||
1. Confirm the service is installed as a **LaunchAgent**, not a LaunchDaemon:
|
||||
`gitea-macos-runner service install` does the right thing; a hand-written plist in
|
||||
`/Library/LaunchDaemons` does not.
|
||||
2. Ensure the runner user is actually logged in with the desktop loaded. Enable auto-login:
|
||||
**System Settings → Users & Groups → Automatically log in as**.
|
||||
3. Prevent the machine from returning to a locked state:
|
||||
|
||||
```sh
|
||||
sudo pmset -a sleep 0 disablesleep 1
|
||||
```
|
||||
|
||||
and disable "Require password after screen saver begins" for the runner user.
|
||||
|
||||
Connecting over Screen Sharing to a Mac at the login window does not unlock `login.keychain` for
|
||||
launchd's session — auto-login is the reliable answer on a dedicated CI Mac.
|
||||
|
||||
---
|
||||
|
||||
## Job stays queued and no VM boots
|
||||
|
||||
**Symptom.** The workflow shows as queued in Gitea indefinitely. Nothing appears in the daemon logs
|
||||
about it.
|
||||
|
||||
**Causes and fixes, in the order worth checking:**
|
||||
|
||||
1. **Label mismatch.** `runs-on` must use **bare label names** (`macos-arm64`), and every label
|
||||
listed must appear in the host config's `runner.labels`. The `:host` suffix used at registration
|
||||
is runner-side only and must never appear in workflow YAML. A single typo produces exactly this
|
||||
symptom with no error anywhere.
|
||||
|
||||
2. **Daemon not running or not polling.**
|
||||
|
||||
```sh
|
||||
gitea-macos-runner service status
|
||||
log show --predicate 'process == "gitea-macos-runner"' --info --last 15m
|
||||
```
|
||||
|
||||
You should see a poll every `scheduler.pollIntervalSeconds`.
|
||||
|
||||
3. **Gitea too old.** The queued-jobs API with the `labels` field requires **Gitea ≥ 1.25**. On an
|
||||
older instance the daemon can never see jobs. `doctor` reports the server version.
|
||||
|
||||
4. **Admin PAT wrong or under-scoped.** The token must belong to a **site admin** and carry
|
||||
`read:admin` + `write:admin`. Test it:
|
||||
|
||||
```sh
|
||||
curl -H "Authorization: token $TOKEN" \
|
||||
"https://gitea.example.com/api/v1/admin/actions/jobs?status=queued"
|
||||
```
|
||||
|
||||
A 403 means scope or admin status; a 404 usually means the Gitea version predates the endpoint.
|
||||
|
||||
5. **A VM booted but its runner never came online.** Then the job is queued *and* you see VM
|
||||
activity in the logs. Look at registration failures — most often an invalid or invalidated
|
||||
registration token (see below).
|
||||
|
||||
6. **Job expired.** Gitea abandons a job after `ABANDONED_JOB_TIMEOUT` (default 24h). If the daemon
|
||||
was down longer than that, the job is gone; re-run it.
|
||||
|
||||
### Registration fails with an invalid token
|
||||
|
||||
Registration tokens are reusable, but **creating a new token for a scope invalidates the previous
|
||||
one**. If someone clicked "create new registration token" in the Gitea UI, the token in your
|
||||
`registrationTokenFile` is now dead. Re-seed
|
||||
`GITEA_RUNNER_REGISTRATION_TOKEN` on the server (and restart Gitea), or switch to
|
||||
`fetchRegistrationTokenViaAPI: true`. See
|
||||
[setup.md §1.3](setup.md#13-choose-a-registration-token-strategy).
|
||||
|
||||
---
|
||||
|
||||
## `actions/checkout` fails instantly
|
||||
|
||||
**Symptom.** The job starts, the runner connects, and the very first step fails immediately —
|
||||
typically a spawn error naming `node`, or an unhelpful non-zero exit before any output.
|
||||
|
||||
**Cause.** Gitea Actions' JavaScript actions (`actions/checkout` and most of the ecosystem) run by
|
||||
spawning `node` inside the guest. **Node.js is required in the image**, and if provisioning was
|
||||
interrupted it may be absent.
|
||||
|
||||
**Fix.** Confirm, then reprovision:
|
||||
|
||||
```sh
|
||||
gitea-macos-runner vm boot
|
||||
ssh admin@<guest-ip> 'node --version && git --version'
|
||||
|
||||
gitea-macos-runner image provision default
|
||||
```
|
||||
|
||||
If `git` is also missing, the provisioning step failed early — check the build log and rerun
|
||||
provisioning.
|
||||
|
||||
---
|
||||
|
||||
## Runner rows piling up in the Gitea UI
|
||||
|
||||
**Symptom.** **Site Administration → Actions → Runners** accumulates offline `macos-vm-…` entries.
|
||||
|
||||
**Cause.** Gitea deletes an ephemeral registration when its job completes normally. A VM that is
|
||||
killed uncleanly — daemon crash, host power loss, `jobTimeoutMinutes` kill — never reaches that
|
||||
point, so the row is orphaned.
|
||||
|
||||
**Fix.** Usually nothing: the daemon's reconcile loop sweeps orphaned registrations every
|
||||
`scheduler.reconcileIntervalSeconds` (default 300) via
|
||||
`DELETE /api/v1/admin/actions/runners/{id}`, plus a daily midnight sweep. Orphans should clear
|
||||
within a few minutes.
|
||||
|
||||
If they persist, the daemon's admin PAT probably lacks `write:admin` — check the logs for delete
|
||||
failures. To clear them by hand, delete the rows in the Gitea UI; they are inert (offline
|
||||
registrations that have already been spent cannot receive jobs).
|
||||
|
||||
---
|
||||
|
||||
## Disk filling up
|
||||
|
||||
**Symptom.** Free space falls steadily; the daemon starts refusing to launch VMs, citing
|
||||
`storage.minFreeDiskGB`.
|
||||
|
||||
**Cause.** Each VM is an APFS copy-on-write clone of the base image. The clone is free at creation
|
||||
but **grows as the job writes** — dependency caches, build outputs, Xcode's derived data. Clones
|
||||
from uncleanly-killed VMs are not reclaimed automatically.
|
||||
|
||||
**Fix.**
|
||||
|
||||
```sh
|
||||
# See what's there.
|
||||
du -sh ~/Library/Application\ Support/gitea-macos-runner/*
|
||||
ls -la ~/Library/Application\ Support/gitea-macos-runner/vms
|
||||
|
||||
# With the daemon stopped, remove stale clones.
|
||||
gitea-macos-runner service uninstall # or stop the daemon
|
||||
rm -rf ~/Library/Application\ Support/gitea-macos-runner/vms/<stale-clone>
|
||||
gitea-macos-runner service install
|
||||
```
|
||||
|
||||
Only delete entries under `vms/` — `images/` holds the base images you'd otherwise have to rebuild.
|
||||
Longer term: raise `storage.minFreeDiskGB` so the guard trips earlier, delete unused base images
|
||||
with `image delete`, and remember an Xcode image needs 140 GB+ of headroom, more with two
|
||||
concurrent clones diverging.
|
||||
|
||||
---
|
||||
|
||||
## `image build` hangs at install
|
||||
|
||||
**Symptom.** `image build` sits for a long time at the macOS install phase with little visible
|
||||
progress.
|
||||
|
||||
**Cause.** Usually none — installing macOS from an IPSW genuinely takes a long time (tens of
|
||||
minutes, longer on slower storage). The install phase is largely silent.
|
||||
|
||||
**Fix.** **Wait, and do not stop the VM mid-install.** Interrupting the installer leaves the disk
|
||||
image in an undefined state; the resulting image may boot and then fail in confusing ways later.
|
||||
There is no resume.
|
||||
|
||||
If you did interrupt it, or the build genuinely failed:
|
||||
|
||||
```sh
|
||||
gitea-macos-runner image delete <name>
|
||||
gitea-macos-runner image build --ipsw <path> --name <name>
|
||||
```
|
||||
|
||||
Before rebuilding, verify the IPSW is complete and matches your host architecture (Apple Silicon)
|
||||
and version requirement (macOS 27+ for unattended provisioning), and that you have enough free disk
|
||||
for the IPSW plus the target disk size.
|
||||
Reference in New Issue
Block a user