gitea-macos-vm-orchestrator/docs/setup.md

# Setup

This walkthrough covers everything needed to go from a bare Apple Silicon Mac and a self-hosted
Gitea instance to a working macOS CI runner: Gitea-side configuration, host build and signing,
base image creation, service installation, and verification.

Work through it in order. The Gitea side can be done from any machine; the host side must be done
on the Mac that will run the VMs.

---

## 1. Gitea-side configuration

### 1.1 Version requirements

| Component | Minimum | Notes |
| --- | --- | --- |
| Gitea server | **1.25** | 1.25 added `GET /api/v1/admin/actions/jobs`, including the `labels` field that carries the job's `runs-on`. The daemon cannot work without it. |
| Gitea server | 1.26+ recommended | Later fixes to Actions job dispatch and runner cleanup. |
| Runner binary in the guest | **`gitea-runner` v3.x** | This is Gitea's own runner. It is *not* the older `act_runner`; do not substitute it. |

Actions must be enabled on the instance (`[actions] ENABLED = true` in `app.ini`) and on any repo
that will use the runner.

### 1.2 Create the admin token (PAT)

The daemon needs a personal access token belonging to a **site administrator** so it can read the
queued-jobs list and delete stale runner registrations.

1. Sign in as a site-admin user.
2. **Settings → Applications → Manage Access Tokens → Generate New Token.**
3. Grant the token **`read:admin` and `write:admin`** scopes. Nothing else is required.
4. Copy the token immediately — Gitea shows it once.

Store it on the host in a file readable only by the runner user rather than inline in the config:

```sh
install -m 600 /dev/null ~/.config/gitea-macos-runner/admin-token
printf '%s' 'PASTE_TOKEN_HERE' > ~/.config/gitea-macos-runner/admin-token
```

Then set `gitea.adminTokenFile` to that path. See [security.md](security.md) for why the file form
is preferred.

### 1.3 Choose a registration token strategy

Every VM registers itself as a runner before it can accept a job, and registration requires a
registration token. Gitea's registration tokens are **reusable**, and creating a new token for a
given scope **invalidates the previous one** — so a distinct token per VM is impossible by design.
You have two options.

**Option A (recommended): seed a stable instance-wide token on the Gitea server.**

Set an environment variable on the Gitea server process before it starts:

```sh
# Must be at least 32 characters.
GITEA_RUNNER_REGISTRATION_TOKEN=<32+ character random string>
```

For example, in a systemd unit:

```ini
[Service]
Environment=GITEA_RUNNER_REGISTRATION_TOKEN=<32+ character random string>
```

or in `docker-compose.yml`:

```yaml
services:
  gitea:
    environment:
      - GITEA_RUNNER_REGISTRATION_TOKEN=<32+ character random string>
```

Restart Gitea, then put the same value on the host in `gitea.registrationTokenFile`. This token is
stable across restarts and is not invalidated by someone clicking "create new token" in the UI for
a different scope.

**Option B: let the daemon fetch a token via the admin API.**

Set `gitea.fetchRegistrationTokenViaAPI: true` and omit `gitea.registrationToken`/`…File`. The
daemon requests a token with the admin PAT when it needs one. This is simpler to set up, but any
out-of-band token creation for the same scope invalidates the token the daemon is holding, and the
daemon must re-fetch. Option A is more predictable for an unattended host.

Generate a token value with:

```sh
openssl rand -hex 24
```

### 1.4 Target the runner from a workflow

```yaml
# .gitea/workflows/macos.yml
name: macOS build
on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: macos-arm64          # bare label name — must match runner.labels
    steps:
      - uses: actions/checkout@v4
      - name: Show host
        run: sw_vers && uname -m
      - name: Build
        run: swift build
```

Two things to know about labels:

- Use **bare label names** in `runs-on`. The runner registers with `--labels "macos-arm64:host"`;
  the `:host` suffix is runner-side only and declares that the label executes directly on the host
  OS rather than in a container. It never appears in workflow YAML.
- Every label in `runs-on` must be present in the runner's `runner.labels`. A typo means the job
  sits queued forever with no error.

Because no runner exists until a job appears, jobs necessarily wait for a VM to boot. Gitea holds a
queued job for `ABANDONED_JOB_TIMEOUT` (default **24 hours**) before marking it abandoned, so
scale-from-zero is safe. If your queue can legitimately back up for more than a day — a long
maintenance window, for example — raise that setting:

```ini
[actions]
ABANDONED_JOB_TIMEOUT = 72h
```

### 1.5 Optionally restrict which repos may use the runner

The runner is registered instance-wide by default, meaning any repo whose workflow says
`runs-on: macos-arm64` can execute code on it. If that is broader than you want, register the
runner at the org or repo level instead, or restrict via Gitea's runner settings. See
[security.md](security.md#limiting-who-can-use-the-runner).

---

## 2. Host-side setup

### 2.1 Host requirements

| Requirement | Value |
| --- | --- |
| Architecture | Apple Silicon (arm64). Intel Macs cannot run macOS guests. |
| Host macOS | 26 minimum; **27+ strongly recommended** (see below). |
| Guest macOS | 27+ if you want unattended image builds. |
| Free disk | ~60 GB for a vanilla image; **140 GB+** with Xcode installed. |
| RAM | 16 GB minimum; each guest defaults to 8 GB. |
| Concurrent VMs | **2 maximum**, enforced by the macOS kernel. `scheduler.maxConcurrentVMs` must be ≤ 2. |

The macOS 27 recommendation is not cosmetic. The automated image builder uses
`VZMacGuestProvisioningOptions` — introduced in macOS 27 — to create the admin account and skip
Setup Assistant during first boot. Both host and guest must be 27+. An older guest **silently
ignores** the options: the install succeeds, the VM boots, and then sits at Setup Assistant with no
SSH server, so the build appears to hang. See
[troubleshooting.md](troubleshooting.md).

### 2.2 Build, sign, and install

Virtualization.framework refuses to run unless the calling binary carries the
`com.apple.security.virtualization` entitlement, and entitlements are only honoured on a signed
binary inside a proper `.app` bundle. Ad-hoc signing (`codesign -s -`) satisfies this — **no paid
Apple developer account is needed.**

```sh
git clone <this repo> && cd gitea-macos-runner
make install
```

`make install` runs the full chain and places the result:

| Target | What it does |
| --- | --- |
| `make build` | `swift build -c release --arch arm64` |
| `make bundle` | Assemble `GiteaMacosRunner.app` around the binary: `Contents/MacOS/gitea-macos-runner`, `Contents/Info.plist`, and `Contents/Resources/` (`provision.sh`, `launchd.plist.template`, `config.example.json`) |
| `make sign` | `codesign --sign - --entitlements …` (ad-hoc) and print the resulting entitlements |
| `make all` | `build` + `bundle` + `sign`. The default target. |
| `make install` | Runs `all`, copies the app to `~/Applications`, and symlinks the CLI to `/usr/local/bin/gitea-macos-runner` |
| `make dev` | Debug build + bundle + sign, for fast iteration. Does **not** install. |
| `make test` | `swift test` |
| `make uninstall` | Remove the installed app and symlink |
| `make clean` | Remove `.build/` |
| `make help` | List the targets |

The three files under `Contents/Resources/` are not decoration. `image build`
uploads `provision.sh` into the guest, `service install` renders
`launchd.plist.template`, and `config init` writes `config.example.json`. The
code looks in `Contents/Resources` first and only then falls back to
repo-relative paths, so an installed `.app` missing them is a working binary
with three broken commands.

If `/usr/local/bin` isn't writable, `make install` says so and prints the `sudo ln -sf …` command to
run yourself.

Never run the raw binary from `.build/release/` — it is outside the signed bundle, so every VM
operation fails with an entitlement error. Always invoke the symlink (or
`~/Applications/GiteaMacosRunner.app/Contents/MacOS/gitea-macos-runner`).

### 2.3 Create the configuration file

```sh
gitea-macos-runner config init          # add --force to overwrite an existing file
$EDITOR "$(gitea-macos-runner config path)"
gitea-macos-runner config show          # parse + validate, printing secrets redacted
```

`config init` writes the annotated example shipped in the app bundle, so the
file you land in has a `_comment` key explaining each section; those keys are
ignored when the config is read back. Pass `--instance-url https://…` to seed
`gitea.instanceURL` instead of the placeholder. Every subcommand takes
`--config PATH` (`-c`) if you keep the file somewhere else, and `--verbose`.

A minimal working config:

```json
{
  "gitea": {
    "instanceURL": "https://gitea.example.com",
    "adminTokenFile": "~/.config/gitea-macos-runner/admin-token",
    "registrationTokenFile": "~/.config/gitea-macos-runner/registration-token"
  },
  "runner": {
    "labels": ["macos-arm64"],
    "namePrefix": "macos-vm-"
  },
  "scheduler": {
    "maxConcurrentVMs": 2
  },
  "guest": {
    "username": "admin",
    "password": "CHANGE_ME",
    "cpuCount": 4,
    "memoryGB": 8,
    "diskGB": 64
  }
}
```

#### Configuration reference

**`gitea`**

| Key | Default | Description |
| --- | --- | --- |
| `instanceURL` | — (required) | Base URL of the Gitea instance, e.g. `https://gitea.example.com`. Must be `http://` or `https://` with a host. A trailing slash is harmless; a subpath is preserved. |
| `adminToken` | — | Site-admin PAT with `read:admin` + `write:admin`, inline. |
| `adminTokenFile` | — | Path to a file containing the PAT (tilde-expanded). Should be `chmod 600`. |
| `registrationToken` | — | Runner registration token, inline. Prefer `registrationTokenFile`. |
| `registrationTokenFile` | — | Path to a file containing the registration token (tilde-expanded). Takes precedence over `registrationToken` if both are set. |
| `fetchRegistrationTokenViaAPI` | `false` | Fetch a registration token with the admin PAT when no static one is configured. A *fallback*, not an override: a configured token still wins. |

Set **exactly one** of `adminToken` and `adminTokenFile`. Setting both is a
validation error — a stale inline token next to a live token file is precisely
the ambiguity that turns into a baffling 401 later — and setting neither is too.

For the registration token the rule is looser: configure `registrationToken`,
`registrationTokenFile`, or `fetchRegistrationTokenViaAPI: true`. At least one
is required; the file form wins over the inline form when both are present.

**`runner`**

| Key | Default | Description |
| --- | --- | --- |
| `labels` | `["macos-arm64"]` | Labels this runner offers. A queued job runs here only if its `runs-on` labels are all in this set. **Bare names only** — a `:schema` suffix here is a validation error; the `:host` suffix is appended automatically at registration. |
| `namePrefix` | `"macos-vm-"` | Prefix for generated runner names in the Gitea UI; a unique suffix is appended per VM. |
| `runnerDownloadURL` | `https://gitea.com/gitea/runner/releases/download/v{version}/gitea-runner-{version}-darwin-arm64` | Release asset installed into the guest. `{version}` is substituted with `version`. |
| `version` | `"3.0.2"` | The `gitea-runner` version to install. |

**`scheduler`**

| Key | Default | Description |
| --- | --- | --- |
| `maxConcurrentVMs` | `2` | Simultaneous VMs. **Hard-capped at 2 by macOS.** A larger value is silently clamped to 2 at load rather than rejected; the kernel is not something a config file gets to negotiate. |
| `pollIntervalSeconds` | `5` | How often to poll the queued-jobs API. |
| `reconcileIntervalSeconds` | `300` | How often to sweep Gitea for orphaned runner registrations from uncleanly-killed VMs. |
| `jobTimeoutMinutes` | `120` | Wall-clock limit for one job; the VM is destroyed when exceeded. |
| `bootTimeoutSeconds` | `300` | Time allowed from VM start to a usable SSH connection. |

**`guest`**

| Key | Default | Description |
| --- | --- | --- |
| `username` | `"admin"` | Guest admin account created during image build. Has passwordless sudo. |
| `password` | — (required) | Password for that account. Also used for SSH if key auth is unavailable. |
| `cpuCount` | `4` | vCPUs per guest. |
| `memoryGB` | `8` | RAM per guest. Two concurrent guests at 8 GB need a 16 GB+ host with headroom. |
| `diskGB` | `64` | Guest disk size. Sparse, so this is a ceiling, not immediate consumption. Raise for Xcode. |

**`storage`**

| Key | Default | Description |
| --- | --- | --- |
| `storeDir` | `~/Library/Application Support/gitea-macos-runner` | Base images (`images/`) and running clones (`vms/`) live here. |
| `minFreeDiskGB` | `20` | The scheduler refuses to start a VM below this free-space threshold rather than filling the disk mid-job. |

### 2.4 Build the base image

Download a macOS **27 or newer** IPSW for Apple Silicon (Apple's restore images; the URL for the
current release is also discoverable from Virtualization.framework's latest-supported-restore-image
endpoint), then:

```sh
gitea-macos-runner image build \
  --ipsw ~/Downloads/UniversalMac_27.0_XXXXX_Restore.ipsw \
  --disk-gb 64
```

`--name` defaults to `default`, which is also what `daemon` and `vm boot` look
for. If you name the image something else, pass the same name to
`daemon --image NAME` (and to `vm boot --image NAME`) or the daemon will not
find it. `--ipsw` is optional: omit it and the latest supported restore image is
downloaded into `storeDir/ipsw/` first, which is most of the build's wall-clock
time. `--disk-gb` overrides `guest.diskGB` for this image only.

`--ipsw` (and `image provision --xcode-xip`) accept a `~` and a glob, quoted or not — these are
equivalent, and neither depends on what your shell did with the pattern first:

```sh
gitea-macos-runner image build --ipsw ~/Downloads/UniversalMac_27.0_*.ipsw
gitea-macos-runner image build --ipsw '~/Downloads/UniversalMac_27.0_*.ipsw'
```

A pattern must identify exactly one file. If it matches several, the build stops before doing any
work and lists them so you can name the one you meant; if it matches none, it says so.

This takes a long time — macOS installs from the IPSW, boots, and is then provisioned over SSH.
**Do not interrupt it during the install phase.** Stopping a VM mid-install leaves the disk image
in an undefined state; delete the image and start over rather than trying to resume.

The finished image contains:

- macOS installed from the IPSW.
- The `guest.username` admin account, auto-created via provisioning options, with SSH (Remote
  Login) enabled and **passwordless sudo**.
- Sleep, screensaver, and Spotlight indexing disabled — a sleeping guest stalls a job, and Spotlight
  wastes I/O on a throwaway machine.
- Raised file-descriptor limits, which Node- and Xcode-based builds routinely exhaust at the
  default.
- **Node.js — required, not optional.** Gitea Actions' JavaScript actions (`actions/checkout` and
  most of the ecosystem) are executed by spawning `node` in the guest. Without it, every workflow
  fails on its first step.
- `git`, verified present.
- The `gitea-runner` v3.x binary.

Verify:

```sh
gitea-macos-runner image list
```

#### Adding Xcode (optional)

Xcode is not installed automatically. Apple requires an authenticated Apple ID for the download and
its download endpoints are impractical to drive unattended, so you fetch the `.xip` yourself from
[developer.apple.com/download](https://developer.apple.com/download/) and hand it to the
provisioner:

```sh
gitea-macos-runner image provision default --xcode-xip ~/Downloads/Xcode_XX.xip
```

Budget disk accordingly: **~60 GB free is enough for a vanilla image, but plan on 140 GB+ with
Xcode**, and remember each running VM is a copy-on-write clone whose divergence from the base
consumes additional space while it runs. Raise `guest.diskGB` (e.g. to 120) before building if the
image will carry Xcode.

### 2.5 Install the service

```sh
gitea-macos-runner service install
gitea-macos-runner service status
```

This installs a **LaunchAgent in the logged-in user's GUI session — not a LaunchDaemon.** Two
reasons this is not negotiable:

1. Virtualization.framework requires a GUI session; a system-context daemon cannot start VMs.
2. macOS 15 and later require an **unlocked `login.keychain`** for key operations the VM lifecycle
   performs. In a locked or headless session these fail with `SecKeyCreateRandomKey` /
   "Interaction is not allowed" errors.

**Recommended: enable auto-login for the runner user on a dedicated CI Mac.**
**System Settings → Users & Groups → Automatically log in as → \<runner user\>.** Combine with
disabling sleep (`sudo pmset -a sleep 0 disablesleep 1`) so the Mac comes back into a live session
after a power event without a human present. This does mean the disk is effectively unlocked at
boot — appropriate for a dedicated CI machine, not for a shared workstation.

`service uninstall` removes the LaunchAgent; it does not delete images or config.

### 2.6 macOS 15+ Local Network privacy prompt

Starting with macOS 15, a process that contacts other hosts on the local network triggers a
one-time Local Network permission prompt. A LaunchAgent that is denied (or that never gets a human
to click Allow) cannot reach the guest's NAT address, so VMs boot but SSH never connects.

**On a CI host, use the subnet allowlist.** It is the only deterministic option — no prompt, no GUI
session, and nothing to redo after a rebuild:

```sh
sudo defaults write com.apple.network.local-network \
  AllowedEthernetLocalNetworkAddresses -array "192.168.64.0/18"
sudo defaults write com.apple.network.local-network \
  AllowedWiFiLocalNetworkAddresses -array "192.168.64.0/18"
```

Then **reboot** — these are read at boot, so restarting the service alone is not enough.

**Do not pin this to a single /24.** Virtualization.framework's NAT starts at `192.168.64.0/24` but
picks the subnet at runtime and steps to the next free one when that range is already in use, so the
same host can hand out `192.168.65.x` tomorrow. An allowlist naming only `192.168.64.0/24` then
looks configured while silently blocking every guest — the failure surfaces as `No route to host`
(errno 65) on the SSH connection, not as a permission error. The `/18` above spans
`192.168.64.0`–`192.168.127.255`, which covers the drift; if you would rather not think about
ranges at all, the RFC 1918 set `"10.0.0.0/8" "172.16.0.0/12" "192.168.0.0/16"` also works. `doctor`
reports `local network access` as a **pass** once it sees an allowlist that covers that span, and as
a **warning** when an allowlist exists but does not. Both keys are documented by Apple in
[TN3179](https://developer.apple.com/documentation/technotes/tn3179-understanding-local-network-privacy)
and are the workaround [Tart's FAQ](https://tart.run/faq/) recommends for the same problem.

**Approving interactively instead.** The app cannot be pre-approved: it appears under **System
Settings → Privacy & Security → Local Network** only *after* it has actually attempted a connection
to a guest. An empty list is expected on a fresh install and does not mean anything is broken. To
create the entry and answer the prompt, run one boot by hand from a Terminal in the GUI session:

```sh
gitea-macos-runner vm boot --image default
```

and click **Allow**. Do not wait for the LaunchAgent to hit it — a background agent has no way to
answer the prompt.

> **Caveat with ad-hoc signing.** An interactive grant is not durable for this project's ad-hoc
> signed bundle. Local Network privacy does not use TCC; per TN3179 it "uses your main executable
> UUID as part of its implementation", and the linker mints a fresh `LC_UUID` on essentially every
> rebuild. So `make install` after a code change is liable to present as a new app that must be
> approved again — and macOS offers no way to reset a Local Network decision back to undetermined,
> so the stale entries accumulate. This is why the allowlist above, which is keyed on the subnet
> rather than on the app, is the recommendation for an unattended machine.

---

## 3. Verification

### 3.1 Preflight

```sh
gitea-macos-runner doctor
```

It reports one line per check, each `✓` pass, `✗` fail, `!` warn, or `·` note,
with a remediation hint under anything that is not a pass, and exits non-zero if
anything failed. `--json` emits the same results machine-readably; `--no-fail`
exits zero regardless, which is what you want when running it from a script that
handles the results itself.

The checks, in order:

| Check | What it looks at |
| --- | --- |
| `host capability` | Apple Silicon, and host macOS ≥ 26 |
| `Virtualization.framework` | `VZVirtualMachine.isSupported` |
| `virtualization entitlement` | `com.apple.security.virtualization` on the *running* executable — this is the check that catches running from `.build/` instead of the signed `.app` |
| `configuration` | The config file loads, parses, and passes validation |
| `free disk space` | Free space on the `storeDir` volume against `storage.minFreeDiskGB` |
| `login.keychain unlocked` | `security show-keychain-info login.keychain` |
| `gitea admin token` / `gitea admin api` | The admin token resolves, and an admin-only endpoint answers with it (so a non-admin PAT fails here rather than at 3am) |
| `registration token` | A static token resolves, or one can be fetched when `fetchRegistrationTokenViaAPI` is on |
| `runner download url` | The `gitea-runner` release asset is reachable |
| `token file permissions` | Warns — not fails — when a token file is group- or world-readable |
| `local network access` | Passes when a subnet allowlist covers `192.168.64.0/18`; warns when an allowlist exists but is scoped too narrowly; otherwise an informational note about the macOS 15+ Local Network prompt (§2.6) |

If the config file is missing or invalid, the host checks still run and the rest
are skipped — which is exactly the state a first-time operator is in. Resolve
everything it reports before going further.

There is deliberately **no** "a base image exists" check; use `image list`.

### 3.2 Boot a VM by hand

```sh
gitea-macos-runner vm boot
```

This clones the base image and boots it without registering a runner — the fastest way to confirm
that virtualization, networking, and SSH all work. Once it is up, check that the guest got a lease:

```sh
cat /var/db/dhcpd_leases
```

and that you can reach it:

```sh
ssh admin@<guest-ip> 'sw_vers; node --version; git --version; which gitea-runner'
```

All four should answer. A guest at Setup Assistant instead of a login window means the image was
built from a pre-27 IPSW; rebuild.

### 3.3 Watch a real job

Start the service, push the example workflow from §1.4, and watch both sides:

```sh
# Host: daemon logs
log stream --predicate 'process == "gitea-macos-runner"' --info

# Host: VM lifecycle
gitea-macos-runner service status
```

In the Gitea UI, the job should move from queued to running within roughly one poll interval plus
boot time; a runner named `macos-vm-<suffix>` appears under **Site Administration → Actions →
Runners** while the job runs and disappears when it finishes. That disappearance is Gitea deleting
the ephemeral registration itself, and it is the signal that the whole loop worked.

If the job stays queued, or the runner appears but the job fails immediately, go to
[troubleshooting.md](troubleshooting.md).