Compare commits

..
6 Commits
Author SHA1 Message Date
abkslm b682cfd0ba Merge nucleic/vivid-glass-urchin-xoym into main
build / build (push) Successful in 2m30s
2026-08-08 17:39:40 -07:00
abkslm de3fc45777 Merge nucleic/vivid-glass-urchin-xoym into main 2026-08-08 17:19:44 -07:00
abkslm 26739f9487 Merge nucleic/vivid-glass-urchin-xoym into main 2026-08-07 16:38:56 -07:00
abkslm 902bea5091 Merge nucleic/vivid-glass-urchin-xoym into main 2026-08-07 15:41:01 -07:00
abkslm 6b0c01b74b Merge nucleic/upbeat-opal-viper-cc8l into main
build / build (push) Canceled after 0s
2026-08-07 05:11:01 -07:00
abkslm ee19744496 update README.md
build / build (push) Successful in 2m32s
2026-08-07 04:37:01 -07:00
20 changed files with 2304 additions and 264 deletions
+7 -4
View File
@@ -44,10 +44,13 @@ jobs:
- name: unit tests
run: swift test
# Release build, .app assembly, ad-hoc signature. `codesign --sign -`
# needs no signing identity and no keychain, so it works unattended in a
# throwaway guest — which is the same property that makes it the
# project's shipping signature.
# Release build, .app assembly, signature. `make sign` looks for a
# Developer ID identity and falls back to ad-hoc when there is none — which
# is always the case here, since this runs in a throwaway guest with no
# keychain and no certificate. That fallback is why this step works
# unattended, and it is deliberately not treated as a failure: the point of
# the CI signature is to prove the entitlement survives, not to produce a
# distributable artifact. Release builds are signed on a real host.
- name: build and sign the app bundle
run: make all
+52 -5
View File
@@ -3,7 +3,8 @@
# Virtualization.framework refuses to start a VM unless the calling process
# carries the `com.apple.security.virtualization` entitlement, and entitlements
# only survive on a signed bundle. So the shipping artifact is not a bare
# executable but a minimal `.app` bundle that we ad-hoc sign. See docs/DESIGN.md
# executable but a minimal `.app` bundle that we sign -- with a Developer ID
# certificate when one is in the keychain, ad-hoc otherwise. See docs/DESIGN.md
# ("Verified Facts", item 10).
SHELL := /bin/bash
@@ -18,8 +19,9 @@ INFO_PLIST := Resources/Info.plist
# The entitlements plist grants exactly one entitlement,
# `com.apple.security.virtualization`. Virtualization.framework refuses to
# create a VM without it, and it is granted by ad-hoc signing
# (`codesign --sign -`) -- no Apple developer account required.
# create a VM without it. It is not a restricted entitlement: ad-hoc signing
# (`codesign --sign -`) grants it, and a Developer ID certificate grants it
# without a provisioning profile.
#
# Deliberately absent: com.apple.vm.networking, which would be needed for a
# bridged network attachment. That one IS restricted and requires an approved
@@ -43,6 +45,31 @@ APP_RESOURCES := Resources/provision.sh \
INSTALL_DIR := $(HOME)/Applications
LINK_PATH := /usr/local/bin/$(BIN_NAME)
# Code signing identity.
#
# A real Developer ID certificate is what makes the bundle's code identity
# *stable across rebuilds*. Its designated requirement is anchored to the team
# ("... and certificate leaf[subject.OU] = L7UDTQ6F5W"), so macOS recognises
# every subsequent build as the same program. An ad-hoc signature has no such
# anchor, so the system falls back to the main executable's Mach-O UUID -- which
# the linker regenerates on essentially every link. Each `make install` then
# presents a program macOS has never seen, and per TN3179 that silently
# withdraws the app's Local Network grant. See docs/troubleshooting.md.
#
# TEAM_ID picks the certificate out of the keychain. When no matching
# "Developer ID Application" identity is present the build still succeeds --
# ad-hoc, with a warning -- because CI runs `make all` inside a throwaway guest
# that has neither a keychain nor a certificate, and that path must keep
# working. Override with `make sign TEAM_ID=...`, or `TEAM_ID=` to force ad-hoc.
TEAM_ID ?= L7UDTQ6F5W
# Hardened runtime plus a trusted timestamp: the pair notarization requires.
# Neither costs anything at runtime here, and having them means the bundle can
# be notarized later without re-signing. `--timestamp` contacts Apple's
# timestamp authority, so signing needs network access. Both are rejected by an
# ad-hoc signature, hence they are only passed on the Developer ID path.
SIGN_OPTS ?= --options runtime --timestamp
# Release by default; `make dev` overrides to debug.
CONFIG ?= release
BIN_PATH = $(BUILD_DIR)/$(CONFIG)/$(BIN_NAME)
@@ -72,11 +99,31 @@ bundle:
cp $(APP_RESOURCES) "$(RES_DIR)/"
chmod +x "$(RES_DIR)/provision.sh"
## sign: ad-hoc sign the bundle with the virtualization entitlement
## sign: sign the bundle (Developer ID when available, else ad-hoc) with the virtualization entitlement
sign:
codesign --sign - --entitlements "$(ENTITLEMENTS)" --force "$(APP_DIR)"
@identity=$$(security find-identity -v -p codesigning 2>/dev/null \
| grep "Developer ID Application" | grep -F "($(TEAM_ID))" \
| head -1 | awk '{print $$2}'); \
if [ -n "$$identity" ]; then \
echo "signing with Developer ID $$identity (team $(TEAM_ID))"; \
codesign --sign "$$identity" $(SIGN_OPTS) \
--entitlements "$(ENTITLEMENTS)" --force "$(APP_DIR)"; \
else \
echo "warning: no 'Developer ID Application' identity for team '$(TEAM_ID)' in the keychain."; \
echo " Falling back to an ad-hoc signature. The bundle runs and the entitlement"; \
echo " works, but its code identity changes on every rebuild, so a macOS Local"; \
echo " Network grant will not survive the next 'make install'."; \
echo " See docs/troubleshooting.md."; \
codesign --sign - --entitlements "$(ENTITLEMENTS)" --force "$(APP_DIR)"; \
fi
@echo "--- entitlements ---"
@codesign -d --entitlements - "$(APP_DIR)" 2>/dev/null || true
@echo "--- identity ---"
@# -dvv, not -dv: the Authority chain is only printed at the second -v.
@# The CodeDirectory line is where `flags=0x10000(runtime)` shows up, which
@# is the only proof the hardened runtime actually landed.
@codesign -dvv "$(APP_DIR)" 2>&1 \
| grep -E "^(Identifier|TeamIdentifier|Authority|Timestamp|CodeDirectory)" || true
## dev: debug build + bundle + sign (fast iteration loop)
dev:
+14 -7
View File
@@ -1,4 +1,4 @@
# gitea-macos-runner
# gitea-macos-vm-orchestrator
A Swift daemon that gives a self-hosted Gitea instance on-demand macOS CI capacity from a single
Apple Silicon Mac. It polls Gitea for queued Actions jobs that request macOS, boots a fresh
@@ -51,17 +51,20 @@ were killed uncleanly, so the Gitea runner list does not accumulate dead entries
- **Gitea 1.25 or newer** (1.26+ recommended). 1.25 added the admin jobs API with the `labels`
field this daemon depends on.
- A code-signed app bundle. The binary must carry the `com.apple.security.virtualization`
entitlement; ad-hoc signing (`codesign -s -`) is sufficient, so no paid Apple developer account
is required.
entitlement, which is not a restricted entitlement — ad-hoc signing (`codesign -s -`) grants it,
so the runner *works* with no Apple developer account. A **Developer ID Application** certificate
is nonetheless recommended: it anchors the bundle's code identity to your team, which is what
keeps a macOS Local Network grant alive across rebuilds. See
[Code signing](docs/setup.md#code-signing).
## Quickstart
```sh
git clone <this repo> && cd gitea-macos-runner
# Build, bundle (binary + Resources + Info.plist), ad-hoc sign with the
# virtualization entitlement, then copy to ~/Applications and symlink the CLI
# into /usr/local/bin.
# Build, bundle (binary + Resources + Info.plist), sign with the virtualization
# entitlement (Developer ID if a matching certificate is in the keychain, ad-hoc
# otherwise), then copy to ~/Applications and symlink the CLI into /usr/local/bin.
make install # = make build bundle sign, then the install step
# Write a starter config to ~/.config/gitea-macos-runner/config.json
@@ -84,6 +87,8 @@ gitea-macos-runner image build --ipsw ~/Downloads/UniversalMac_27.0_*.ipsw
gitea-macos-runner image provision default --xcode-xip ~/Downloads/Xcode_XX.xip
# Install and start the LaunchAgent (runs in your GUI login session — not a LaunchDaemon).
# On a terminal this also offers to grant macOS Local Network access, which the agent
# needs to reach its guests; `permissions grant` does the same thing on its own.
gitea-macos-runner service install
gitea-macos-runner service status
```
@@ -145,9 +150,11 @@ Every subcommand accepts the global options `--config PATH` (`-c`, default
| `image delete NAME [--force]` | Delete a base image and its disk. `--force` (`-f`) skips the confirmation prompt. |
| `vm boot [--image NAME] [--slot N] [--keep]` | Clone an image, boot it, print its IP, and wait for Ctrl-C. `--slot` picks which persistent per-slot MAC to use (default `0`); `--keep` leaves the clone on disk. |
| `vm list` | List ephemeral VM clones on disk. |
| `service install [--executable PATH]` | Write and load `~/Library/LaunchAgents/xyz.blakeslee.gitea-macos-runner.plist`. |
| `service install [--executable PATH] [--grant-local-network allowlist\|prompt\|none]` | Write and load `~/Library/LaunchAgents/xyz.blakeslee.gitea-macos-vm-orchestrator.plist`. Also evicts any agent left behind under a previous label. On a terminal it offers to configure Local Network access when that is unconfigured, defaulting to no; `--grant-local-network` decides it up front. |
| `service uninstall` | Unload the LaunchAgent and remove its plist. |
| `service status` | Report LaunchAgent installation and run state. |
| `permissions status` | Report whether macOS Local Network access is configured, and whether the code identity is stable enough to hold an interactive grant. Run it under `sudo` to see the allowlist — it is written into root's preferences, which an ordinary login cannot read. |
| `permissions grant [--method allowlist\|prompt] [--subnet CIDR ...] [--reboot\|--no-reboot]` | Grant Local Network access. `allowlist` (default) writes the subnet allowlist with sudo — all of RFC 1918 unless `--subnet` narrows it — and needs a reboot. `prompt` launches the installed `.app` so the system alert is attributed to it rather than to Terminal, and applies immediately. |
| `doctor [--json] [--no-fail]` | Preflight checks. `--json` emits machine-readable results; `--no-fail` exits zero even when checks fail. |
| `config init [--force] [--instance-url URL]` | Write the annotated example config. `--force` (`-f`) overwrites an existing file. |
| `config show` | Print the effective configuration with secrets redacted. |
+1 -1
View File
@@ -3,7 +3,7 @@
<plist version="1.0">
<dict>
<key>CFBundleIdentifier</key>
<string>xyz.blakeslee.gitea-macos-runner</string>
<string>xyz.blakeslee.gitea-macos-vm-orchestrator</string>
<key>CFBundleName</key>
<string>GiteaMacosRunner</string>
+14 -1
View File
@@ -140,6 +140,19 @@ public struct RunnerConfig: Codable, Sendable, Equatable {
/// Ceiling on boot + DHCP lease + SSH readiness before a slot is
/// declared dead and recycled.
///
/// - Important: This must exceed the guest's *worst-case* time to become
/// SSH-ready, not its typical one. A clone that is still booting when
/// this expires is destroyed and replaced by another clone that starts
/// from zero — and because the replacement adds load to an already
/// contended host, the next boot is slower still. Set too tight, this
/// is not a timeout but a livelock: the daemon boots forever and no
/// runner ever registers.
///
/// A guest sharing an Apple Silicon host with other Virtualization
/// guests can take several minutes to reach `sshd`, so the default is
/// deliberately generous. A genuinely wedged guest still gets caught;
/// it just takes longer to notice, which is the cheaper mistake.
public var bootTimeoutSeconds: Int
public init(
@@ -147,7 +160,7 @@ public struct RunnerConfig: Codable, Sendable, Equatable {
pollIntervalSeconds: Int = 5,
reconcileIntervalSeconds: Int = 300,
jobTimeoutMinutes: Int = 120,
bootTimeoutSeconds: Int = 300
bootTimeoutSeconds: Int = 900
) {
self.maxConcurrentVMs = maxConcurrentVMs
self.pollIntervalSeconds = pollIntervalSeconds
+280
View File
@@ -0,0 +1,280 @@
import Foundation
/// The macOS 15+ Local Network subnet allowlist: where it lives, what counts as
/// covering the guest range, and the commands that write it.
///
/// ## Why an allowlist at all
///
/// Local Network privacy is not TCC. It is a Network Extension packet filter,
/// so there is no database to query, `tccutil` does not apply, and a blocked
/// flow is not reported as "denied" — it comes back `EHOSTUNREACH` (errno 65,
/// "No route to host"), indistinguishable from a guest that is genuinely off
/// the network (Apple, TN3179).
///
/// Worse, nothing here is well placed to *answer* the prompt. A LaunchAgent has
/// no UI to show it in, and a run started from a shell is attributed to the
/// **responsible process** — Terminal — so both the prompt and the System
/// Settings row belong to Terminal, and granting it there does not carry over
/// to the agent.
///
/// The allowlist sidesteps all of that: it is consulted before the per-app
/// check, so a flow to a listed subnet is never subject to a prompt, by any
/// process. Its one cost is that the values are read at boot, so setting it
/// requires a reboot to take effect. That is the trade this type exists to make
/// explicit.
///
/// Everything here is pure — reading a plist and formatting argument vectors —
/// so it lives in `RunnerCore` and is unit-tested. The effectful half (running
/// `sudo`, launching the app to trigger a prompt) is `RunnerHost`'s
/// `LocalNetworkPermission`.
public enum LocalNetworkPolicy {
// MARK: - Where the setting lives
/// The preferences domain macOS reads the allowlist from.
public static let domain = "com.apple.network.local-network"
/// The wired interfaces key.
public static let ethernetKey = "AllowedEthernetLocalNetworkAddresses"
/// The Wi-Fi interfaces key.
public static let wifiKey = "AllowedWiFiLocalNetworkAddresses"
/// Both keys. Guests are reached over a virtual interface, and which of the
/// two the filter consults is not something we get to observe — so both are
/// always written, and both are read back.
public static let keys = [ethernetKey, wifiKey]
/// Every preferences file the allowlist could plausibly be written to.
///
/// The domain is written with `sudo`, so which preferences directory it
/// lands in depends on whether that `sudo` preserved `HOME`. Rather than
/// guess at the host's sudoers configuration, check each candidate.
///
/// In practice the first candidate is where it lands, and `/var/root` is
/// mode 700 — so an unprivileged process cannot read back what it just
/// wrote. That is what ``Status/unreadablePaths`` exists to report, and
/// what `RunnerHost`'s `LocalNetworkPermission.observedStatus()` works
/// around by re-reading as root.
public static func preferenceCandidates() -> [String] {
[
"/var/root/Library/Preferences/\(domain).plist",
"/Library/Preferences/\(domain).plist",
NSHomeDirectory() + "/Library/Preferences/\(domain).plist",
]
}
// MARK: - What to write
/// The subnets granted by default: all of RFC 1918.
///
/// Deliberately wider than the `192.168.64.0/18` that vmnet actually uses.
/// The allowlist is read at boot, so getting it wrong costs a reboot to fix,
/// and the failure mode of "too narrow" is silent — guests simply stop being
/// reachable the day the host joins a network that shifts things around.
/// This is also the set Tart, orchard, and packer-plugin-tart ship, so a
/// host already configured for one of those needs no second entry.
///
/// Narrow it with `permissions grant --subnet` on a host where that matters.
public static let defaultSubnets = ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"]
/// The argument vectors that write `subnets` to both keys.
///
/// Returned as arrays, not a shell string: these are handed straight to
/// `/usr/bin/defaults` with no shell in between, so a subnet containing
/// something shell-significant cannot become an injection.
///
/// - Parameter subnets: CIDR entries, e.g. `["192.168.0.0/16"]`.
/// - Returns: One `defaults write …` argument vector per key.
public static func writeArguments(subnets: [String]) -> [[String]] {
keys.map { key in ["write", domain, key, "-array"] + subnets }
}
/// The same commands as copy-pasteable shell, for the message shown when we
/// cannot run them ourselves.
public static func writeCommandLines(subnets: [String]) -> [String] {
writeArguments(subnets: subnets).map { arguments in
"sudo defaults " + arguments.map(quoteForShell).joined(separator: " ")
}
}
/// Single-quotes an argument unless it is plainly inert.
private static func quoteForShell(_ argument: String) -> String {
let safe = argument.allSatisfy { $0.isLetter || $0.isNumber || "./_-".contains($0) }
guard !safe || argument.isEmpty else { return argument }
return "'" + argument.replacingOccurrences(of: "'", with: #"'\''"#) + "'"
}
// MARK: - Reading it back
/// What the host's allowlist currently says.
public struct Status: Sendable, Equatable {
/// Every distinct entry found, across both keys and all candidate files.
public let allowlist: [String]
/// The files the entries came from, in the order they were checked.
public let sourcePaths: [String]
/// Whether at least one entry covers the whole vmnet range.
public let coversGuestRange: Bool
/// Candidate files this process was refused permission to read.
///
/// Not the same as "absent". `sudo defaults write` normally lands in
/// `/var/root/Library/Preferences`, which is mode 700, so an ordinary
/// user is refused before it can find out whether the file is even
/// there. An empty ``allowlist`` with a non-empty `unreadablePaths`
/// means *unknown*, not *unconfigured*, and must not be reported as
/// the latter.
public let unreadablePaths: [String]
/// Whether anything is configured at all.
public var isConfigured: Bool { !allowlist.isEmpty }
/// Whether nothing was found and something could not be read, so the
/// answer is genuinely unknown without administrator rights.
public var isIndeterminate: Bool { allowlist.isEmpty && !unreadablePaths.isEmpty }
public init(
allowlist: [String],
sourcePaths: [String],
coversGuestRange: Bool,
unreadablePaths: [String] = []
) {
self.allowlist = allowlist
self.sourcePaths = sourcePaths
self.coversGuestRange = coversGuestRange
self.unreadablePaths = unreadablePaths
}
}
/// Reads the host's current allowlist with this process's own privileges.
///
/// Best effort and never fatal: an absent preferences file reads as "no
/// allowlist", and one that exists but cannot be opened is recorded in
/// ``Status/unreadablePaths`` rather than being mistaken for absent.
public static func status() -> Status {
var sources: [(path: String, data: Data)] = []
var unreadable: [String] = []
for path in preferenceCandidates() {
if let data = FileManager.default.contents(atPath: path) {
sources.append((path, data))
} else if access(path, R_OK) != 0, errno == EACCES {
// Refused, not missing — including when the refusal is on a
// parent directory, which is exactly the /var/root case.
unreadable.append(path)
}
}
return status(fromContentsOf: sources, unreadablePaths: unreadable)
}
/// Builds a ``Status`` from preferences files already read, however they
/// were obtained.
///
/// Split out from ``status()`` so the privileged read-back in `RunnerHost`
/// — which has to shell out to `sudo` to see root's copy — shares this
/// parsing rather than reimplementing it.
public static func status(
fromContentsOf sources: [(path: String, data: Data)],
unreadablePaths: [String] = []
) -> Status {
var found: [String] = []
var paths: [String] = []
for source in sources {
let fresh = entries(inPreferences: source.data).filter { !found.contains($0) }
guard !fresh.isEmpty else { continue }
found.append(contentsOf: fresh)
paths.append(source.path)
}
return Status(
allowlist: found,
sourcePaths: paths,
coversGuestRange: found.contains(where: coversVMNetRange),
unreadablePaths: unreadablePaths
)
}
/// Every allowlist entry in one preferences file, across both keys, in the
/// order encountered and without duplicates. Unparseable data reads empty.
public static func entries(inPreferences data: Data) -> [String] {
guard
let plist = try? PropertyListSerialization.propertyList(
from: data, options: [], format: nil) as? [String: Any]
else { return [] }
var found: [String] = []
for key in keys {
for entry in (plist[key] as? [String] ?? []) where !found.contains(entry) {
found.append(entry)
}
}
return found
}
/// Subnets pre-authorized for local network access on this host, if any.
public static func allowlist() -> [String] { status().allowlist }
// MARK: - Coverage arithmetic
/// The span of addresses a vmnet NAT link can plausibly use.
///
/// `192.168.64.0/24` is only the *first* choice: the subnet is picked at
/// runtime and steps to the next free /24 when that one is already in use,
/// which is why a host that worked yesterday can hand out `192.168.65.x`
/// today. Everything from 192.168.64.0 to 192.168.127.255 — a /18 — is
/// treated as guest territory so the allowlist survives that drift.
public static let vmNetFirstAddress: UInt32 = 0xC0A8_4000 // 192.168.64.0
public static let vmNetLastAddress: UInt32 = 0xC0A8_7FFF // 192.168.127.255
/// Whether one allowlist entry covers the whole guest range.
///
/// Deliberately all-or-nothing: partial cover is the failure mode being
/// warned about, so an entry that contains today's subnet but not
/// tomorrow's is not treated as good enough.
public static func coversVMNetRange(_ entry: String) -> Bool {
guard let (network, broadcast) = range(of: entry) else { return false }
return network <= vmNetFirstAddress && broadcast >= vmNetLastAddress
}
/// Whether `entry` is a well-formed IPv4 address or CIDR block.
///
/// Used to reject `--subnet` typos at parse time. An entry macOS cannot
/// understand is silently ignored by the filter, which would leave the
/// operator with a configured-looking allowlist that grants nothing.
public static func isValidSubnet(_ entry: String) -> Bool { range(of: entry) != nil }
/// The first and last address of an IPv4 CIDR entry; nil for anything that
/// is not one (an IPv6 entry, a hostname, a typo).
///
/// A bare address is treated as a /32, matching `defaults`' own reading.
static func range(of entry: String) -> (network: UInt32, broadcast: UInt32)? {
// Empty components are kept, so a trailing slash is a parse failure
// rather than silently reading "192.168.64.0/" as a bare /32 host.
let parts = entry.split(separator: "/", maxSplits: 1, omittingEmptySubsequences: false)
guard let first = parts.first, let base = ipv4Value(String(first)) else { return nil }
let prefix = parts.count == 2 ? Int(parts[1]) : 32
guard let prefix, (0...32).contains(prefix) else { return nil }
let mask: UInt32 = prefix == 0 ? 0 : ~UInt32(0) << (32 - prefix)
let network = base & mask
return (network, network | ~mask)
}
/// Packs dotted-quad IPv4 into a comparable integer; nil for anything else
/// (an IPv6 entry, a hostname, a typo).
public static func ipv4Value(_ text: String) -> UInt32? {
let octets = text.split(separator: ".", omittingEmptySubsequences: false)
guard octets.count == 4 else { return nil }
var value: UInt32 = 0
for octet in octets {
guard let number = UInt32(octet), number <= 255 else { return nil }
value = value << 8 | number
}
return value
}
}
+101 -26
View File
@@ -179,7 +179,7 @@ public final class SSHExecutor: GuestExecutor, @unchecked Sendable {
/// otherwise read from the terminal exits instead of hanging.
/// - timeout: Wall-clock ceiling on the whole exchange.
/// - Throws: ``SSHTransportError`` for connect/auth problems (which
/// ``waitForSSH(host:port:username:password:timeout:pollInterval:)`` needs
/// ``waitForSSH(host:port:username:password:timeout:pollInterval:reportInterval:onAttemptFailure:)`` needs
/// to tell apart), or ``CoreError/timeout(_:)`` when the ceiling elapses.
func execute(_ command: String, stdin: Data?, timeout: Duration) async throws -> SSHCommandResult {
let group = MultiThreadedEventLoopGroup.singleton
@@ -189,7 +189,18 @@ public final class SSHExecutor: GuestExecutor, @unchecked Sendable {
let host = self.host
let port = self.port
// The `timeout` argument below cannot bound the connect: its watchdog is
// scheduled on the channel's event loop, which does not exist until the
// connect has already succeeded. A booting guest answers ARP long before
// it answers SYNs, so without an explicit ceiling here each probe hangs
// for the platform default — around 75 s — and `waitForSSH` gets a
// handful of attempts inside its budget instead of one every couple of
// seconds. Bounded by `timeout` so a caller asking for less than the
// default gets what it asked for.
let connectTimeout = min(timeout, Self.defaultConnectTimeout)
let bootstrap = ClientBootstrap(group: group)
.connectTimeout(.nanoseconds(Self.nanoseconds(connectTimeout)))
.channelOption(ChannelOptions.socketOption(.tcp_nodelay), value: 1)
.channelInitializer { channel in
channel.eventLoop.makeCompletedFuture {
@@ -288,7 +299,14 @@ public final class SSHExecutor: GuestExecutor, @unchecked Sendable {
"'" + value.replacingOccurrences(of: "'", with: "'\\''") + "'"
}
private static func nanoseconds(_ duration: Duration) -> Int64 {
/// Ceiling on the TCP connect alone.
///
/// Long enough that a loaded host's slow-but-working connect is not cut
/// short, short enough that a silently dropped SYN costs one poll interval
/// rather than the platform's ~75 s.
static let defaultConnectTimeout = Duration.seconds(10)
static func nanoseconds(_ duration: Duration) -> Int64 {
let components = duration.components
let seconds = components.seconds.multipliedReportingOverflow(by: 1_000_000_000)
guard !seconds.overflow else { return .max }
@@ -302,7 +320,7 @@ public final class SSHExecutor: GuestExecutor, @unchecked Sendable {
// MARK: - Transport failures
/// Connection-level failures, kept distinct from ``CoreError`` so that
/// ``waitForSSH(host:port:username:password:timeout:pollInterval:)`` can tell
/// ``waitForSSH(host:port:username:password:timeout:pollInterval:reportInterval:onAttemptFailure:)`` can tell
/// "sshd is not up yet" (retry) from "the password is wrong" (give up now).
enum SSHTransportError: Error {
/// No TCP connection could be established.
@@ -333,12 +351,16 @@ enum SSHTransportError: Error {
/// that errno is more often the privacy filter than a routing failure —
/// worth naming rather than leaving the operator to guess.
///
/// It matters most right after a rebuild. Per
/// On an **ad-hoc signed** build it matters most right after a rebuild. Per
/// [TN3179](https://developer.apple.com/documentation/technotes/tn3179-understanding-local-network-privacy)
/// the grant "uses your main executable UUID", and the linker mints a fresh
/// `LC_UUID` on essentially every build — so `make install` can present a
/// program macOS has never seen, whose permission is undetermined again,
/// even though the previous binary worked minutes earlier.
/// the grant "uses your main executable UUID" when there is no stable
/// designated requirement to key on, and the linker mints a fresh `LC_UUID`
/// on essentially every build — so `make install` can present a program
/// macOS has never seen, whose permission is undetermined again, even
/// though the previous binary worked minutes earlier. A Developer ID
/// signature is anchored to the team instead and does not have this
/// problem; the hint is unconditional because this layer cannot see which
/// kind of signature it is running under.
static func localNetworkHint(for underlying: any Error) -> String {
let text = "\(underlying)".lowercased()
guard text.contains("errno: 65") || text.contains("no route to host")
@@ -347,11 +369,8 @@ enum SSHTransportError: Error {
return """
(on macOS 15+ this is also what Local Network privacy returns when \
it blocks an app — and the grant is keyed on the executable's UUID, \
so every rebuild withdraws it. Pre-authorize the guest subnet \
instead: sudo defaults write com.apple.network.local-network \
AllowedEthernetLocalNetworkAddresses -array "192.168.64.0/18" — same \
for AllowedWiFiLocalNetworkAddresses — then reboot. \
it blocks an app. Check and fix it with `gitea-macos-runner \
permissions status` / `permissions grant`. \
See docs/troubleshooting.md)
"""
}
@@ -568,12 +587,42 @@ final class ExecChannelHandler: ChannelInboundHandler {
}
}
/// One failed probe, handed to ``waitForSSH(host:port:username:password:timeout:pollInterval:reportInterval:onAttemptFailure:)``'s
/// reporting callback.
///
/// Carries a rendered `error` rather than the `Error` itself so the whole value
/// is `Sendable` and can cross into a logger on another isolation domain.
public struct SSHWaitAttempt: Sendable {
/// 1-based probe count.
public let attempt: Int
/// Time since the wait began.
public let elapsed: Duration
/// The failure, rendered through `asCoreError` where applicable so the
/// Local Network privacy hint survives.
public let error: String
public init(attempt: Int, elapsed: Duration, error: String) {
self.attempt = attempt
self.elapsed = elapsed
self.error = error
}
}
/// Blocks until a guest accepts an authenticated SSH session, or the deadline
/// passes.
///
/// Called after a DHCP lease appears but before any provisioning: a fresh guest
/// answers on port 22 only once `launchd` has started `sshd`, which lags the
/// lease by tens of seconds.
/// lease by tens of seconds — and by minutes on a host running several guests
/// at once.
///
/// - Important: A caller whose own supervisor also enforces a deadline must pass
/// a `timeout` strictly smaller than the supervisor's *remaining* budget.
/// Otherwise the supervisor always fires first, this function is cancelled
/// mid-`Task.sleep`, and the `CoreError.timeout` below — the only place the
/// last error is ever rendered — is never thrown. That is why failures are
/// also reported as they happen via `onAttemptFailure` rather than solely at
/// the end.
///
/// - Parameters:
/// - host: Guest IP.
@@ -582,6 +631,11 @@ final class ExecChannelHandler: ChannelInboundHandler {
/// - password: Guest password.
/// - timeout: Overall ceiling.
/// - pollInterval: Delay between attempts. Defaults to 2 s.
/// - reportInterval: Floor on the gap between `onAttemptFailure` calls.
/// Defaults to 30 s. The first failure is always reported.
/// - onAttemptFailure: Called for the first failure and then no more often
/// than `reportInterval`, so a boot that is merely slow is visible while it
/// is happening instead of only in the post-mortem.
/// - Throws: ``CoreError/timeout(_:)`` if the guest never answers.
public func waitForSSH(
host: String,
@@ -589,13 +643,18 @@ public func waitForSSH(
username: String,
password: String,
timeout: Duration,
pollInterval: Duration = .seconds(2)
pollInterval: Duration = .seconds(2),
reportInterval: Duration = .seconds(30),
onAttemptFailure: (@Sendable (SSHWaitAttempt) -> Void)? = nil
) async throws {
let executor = SSHExecutor(host: host, port: port, username: username, password: password)
let started = ContinuousClock.now
var lastError: Error?
var attempt = 0
var lastReport: ContinuousClock.Instant?
while true {
attempt += 1
do {
// A real authenticated session running a trivial command, not a bare
// TCP probe: sshd binds the port before it is ready to authenticate,
@@ -616,20 +675,36 @@ public func waitForSSH(
lastError = error
}
if let onAttemptFailure, let lastError {
let now = ContinuousClock.now
// Every probe of a guest that is still booting fails, so reporting
// each one would bury the log. First one, then a heartbeat.
if lastReport.map({ now - $0 >= reportInterval }) ?? true {
lastReport = now
onAttemptFailure(
SSHWaitAttempt(
attempt: attempt,
elapsed: now - started,
error: renderSSHWaitError(lastError)
)
)
}
}
guard ContinuousClock.now - started < timeout else { break }
try await Task.sleep(for: pollInterval)
guard ContinuousClock.now - started < timeout else { break }
}
// Rendered through `asCoreError` rather than interpolated raw: a connect
// failure is where the Local Network privacy hint lives, and the timeout
// message is the *only* place most operators will ever see the last error.
let detail: String
if let lastError {
let rendered = (lastError as? SSHTransportError).map { "\($0.asCoreError)" } ?? "\(lastError)"
detail = "; last error: \(rendered)"
} else {
detail = ""
}
throw CoreError.timeout("ssh on \(host):\(port)\(detail)")
let detail = lastError.map { "; last error: \(renderSSHWaitError($0))" } ?? ""
throw CoreError.timeout("ssh on \(host):\(port) after \(attempt) attempts\(detail)")
}
/// Renders a probe failure for humans.
///
/// Goes through `asCoreError` rather than interpolating raw: a connect failure
/// is where the Local Network privacy hint lives, and these strings are the only
/// place most operators will ever see why a boot stalled.
private func renderSSHWaitError(_ error: Error) -> String {
(error as? SSHTransportError).map { "\($0.asCoreError)" } ?? "\(error)"
}
+285 -103
View File
@@ -78,21 +78,29 @@ public enum Doctor {
/// running binary, read with `codesign -d --entitlements - <path>`.
/// Running from `.build/` instead of the signed `.app` is the single most
/// common setup mistake, and this is what catches it.
/// 5. **Free disk ≥ `storage.minFreeDiskGB`.** CoW clones grow as guests
/// 5. **Code identity is stable**, i.e. the bundle is signed with a real
/// team-anchored certificate rather than ad-hoc. Warns on ad-hoc,
/// because that is what makes Local Network grants evaporate on every
/// rebuild (check 12).
/// 6. **Free disk ≥ `storage.minFreeDiskGB`.** CoW clones grow as guests
/// write.
/// 6. **`login.keychain` unlocked**, via `security show-keychain-info
/// 7. **`login.keychain` unlocked**, via `security show-keychain-info
/// login.keychain`. macOS 15+ refuses to start a VM otherwise — the
/// reason the daemon must be a LaunchAgent in a logged-in session.
/// 7. **Gitea reachable and the token has admin scope**, probed with
/// 8. **Gitea reachable and the token has admin scope**, probed with
/// ``GiteaClient/listRunners()``. A non-admin token fails here rather
/// than at the first poll.
/// 8. **Registration token resolvable** from file, inline value, or (if
/// 9. **Registration token resolvable** from file, inline value, or (if
/// enabled) the API.
/// 9. **Runner download URL is live**, via a one-byte ranged `GET` — the
/// same verb the real download uses, because the presigned redirect
/// target is signed per method. Catches a version bump that no longer
/// has a darwin-arm64 asset.
/// 10. **Local Network privacy**. Passes when a subnet allowlist is set in
/// 10. **Runner download URL is live**, via a one-byte ranged `GET` — the
/// same verb the real download uses, because the presigned redirect
/// target is signed per method. Catches a version bump that no longer
/// has a darwin-arm64 asset.
/// 11. **Guest SSH**, against whichever slot currently holds a DHCP lease —
/// the one check that exercises host → vmnet → guest `sshd` → password
/// auth end to end. Informational when no guest is up, since `doctor`
/// will not boot one.
/// 12. **Local Network privacy**. Passes when a subnet allowlist is set in
/// `com.apple.network.local-network`; otherwise informational. On
/// macOS 15+ the first attempt to reach a guest over the NAT link can
/// be blocked by the Local Network permission prompt, which a
@@ -107,6 +115,7 @@ public enum Doctor {
checks.append(checkLoginKeychain())
checks.append(contentsOf: await checkGitea(config: config))
checks.append(await checkRunnerDownloadURL(config: config))
checks.append(await checkGuestSSH(config: config))
checks.append(localNetworkNote())
return checks
}
@@ -148,18 +157,20 @@ public enum Doctor {
checks.append(checkLoginKeychain())
checks.append(contentsOf: await checkGitea(config: loaded))
checks.append(await checkRunnerDownloadURL(config: loaded))
checks.append(await checkGuestSSH(config: loaded))
checks.append(contentsOf: checkTokenFilePermissions(config: loaded))
checks.append(localNetworkNote())
return checks
}
/// The configuration-independent host checks: architecture, OS version,
/// framework support, entitlement.
/// framework support, entitlement, code identity.
public static func hostChecks() -> [DoctorCheck] {
[
checkHostCapability(),
checkVirtualizationSupported(),
checkVirtualizationEntitlement(),
checkCodeSignature(),
]
}
@@ -187,11 +198,7 @@ public enum Doctor {
// The entitlement lives on the signature, so a bare binary copied out of
// the bundle loses it. Report where we are as well as what we found.
let inAppBundle = executable.contains(".app/Contents/MacOS/")
let signedTarget = inAppBundle
? String(executable.prefix(upTo: executable.range(of: ".app/Contents/MacOS/")!.upperBound)
.dropLast("/Contents/MacOS/".count))
: executable
let (signedTarget, inAppBundle) = signableTarget(for: executable)
let result = DoctorShell.run(
"/usr/bin/codesign",
@@ -226,6 +233,112 @@ public enum Doctor {
)
}
/// Whether the bundle's code identity is stable across rebuilds.
///
/// This is not a cosmetic "is it properly signed" check — it is the root
/// cause of the project's most confusing failure. A Developer ID signature
/// carries a designated requirement anchored to the team
/// (`certificate leaf[subject.OU] = "…"`), so macOS recognises every later
/// build as the same program and the app's Local Network grant persists. An
/// **ad-hoc** signature has no such anchor, so per
/// [TN3179](https://developer.apple.com/documentation/technotes/tn3179-understanding-local-network-privacy)
/// the system identifies the app by its main executable's Mach-O UUID —
/// which the linker regenerates on essentially every link. Each
/// `make install` therefore presents a program macOS has never seen, its
/// permission reverts to undetermined, and guest SSH starts failing with
/// `No route to host` minutes after a build that worked.
///
/// Ad-hoc is a `warn`, not a `fail`: everything still runs, and it is the
/// only option on a host without a certificate (CI signs this way
/// deliberately). It just needs the subnet allowlist to compensate.
///
/// - Parameter binaryPath: Defaults to the current executable.
/// - Returns: The check result.
public static func checkCodeSignature(
binaryPath: String = CommandLine.arguments.first ?? ""
) -> DoctorCheck {
let name = "code identity"
guard let executable = resolveExecutablePath(binaryPath) else {
return DoctorCheck(
name: name,
result: .warn,
detail: "could not locate the running executable to inspect",
remediation: "build and install the signed bundle: `make install`"
)
}
let (target, inAppBundle) = signableTarget(for: executable)
let result = DoctorShell.run("/usr/bin/codesign", ["-dv", target])
guard result.exitCode == 0 else {
return DoctorCheck(
name: name,
result: inAppBundle ? .fail : .warn,
detail: "\(target) carries no code signature",
remediation: "sign the bundle: `make sign` (or `make install`)"
)
}
let team = value(of: "TeamIdentifier", in: result.output)
let identifier = value(of: "Identifier", in: result.output) ?? "?"
let hardened = result.output.contains("flags=") && result.output.contains("runtime")
guard let team, team != "not set" else {
return DoctorCheck(
name: name,
result: .warn,
detail: "\(identifier) is ad-hoc signed (no team identifier)",
remediation: """
An ad-hoc signature has no stable designated requirement, so macOS falls back \
to identifying this app by its Mach-O UUID — regenerated on every build. Any \
Local Network grant is withdrawn by the next `make install`, and guests then \
fail with "No route to host". Sign with a Developer ID certificate \
(`make sign TEAM_ID=<team>`), or set the subnet allowlist so no grant is \
needed at all — see the "local network access" check.
"""
)
}
return DoctorCheck(
name: name,
result: .pass,
detail: "\(identifier), team \(team)"
+ (hardened ? ", hardened runtime" : "")
)
}
/// Reads a `Key=value` line out of `codesign -dv` output.
///
/// `codesign` writes this block to stderr, one `Key=value` per line, and
/// repeats some keys (`Authority`); the first match is the one that matters.
private static func value(of key: String, in output: String) -> String? {
for line in output.split(separator: "\n") {
let trimmed = line.trimmingCharacters(in: .whitespaces)
guard trimmed.hasPrefix("\(key)=") else { continue }
return String(trimmed.dropFirst(key.count + 1))
}
return nil
}
/// The artifact `codesign` should be pointed at: the enclosing `.app` when
/// the executable lives inside one, otherwise the executable itself.
///
/// Signatures and entitlements are sealed on the bundle, so querying the
/// bare Mach-O inside it — or one copied out of it — answers the wrong
/// question.
///
/// - Parameter executable: An absolute, symlink-resolved executable path.
/// - Returns: The path to query, and whether it is an `.app` bundle.
private static func signableTarget(for executable: String) -> (path: String, inAppBundle: Bool) {
guard let marker = executable.range(of: ".app/Contents/MacOS/") else {
return (executable, false)
}
let bundle = executable.prefix(upTo: marker.upperBound)
.dropLast("/Contents/MacOS/".count)
return (String(bundle), true)
}
/// Whether `login.keychain` is currently unlocked.
public static func checkLoginKeychain() -> DoctorCheck {
let name = "login.keychain unlocked"
@@ -523,6 +636,129 @@ public enum Doctor {
]
}
/// Whether a guest that is up right now actually accepts an SSH session.
///
/// Every other check in this file inspects the host. This one exercises the
/// exact path the boot sequence depends on and that nothing else proves:
/// host → vmnet → guest `sshd` → password auth with `guest.username` /
/// `guest.password`. It is the difference between "the daemon never got a
/// runner online" and a named cause — wrong credentials, Local Network
/// privacy blocking the link, or a guest image whose Remote Login is off.
///
/// Read-only with respect to host state: it uses the MACs already persisted
/// in `state.json` and never generates them, so running `doctor` on a fresh
/// host does not quietly create the slot address table.
///
/// - Note: Informational when no guest is currently leased. `doctor` must not
/// boot a VM — that costs minutes and a slot out of the host's hard cap of
/// two — so with nothing running there is simply nothing to probe. To make
/// this check meaningful, leave a guest up (`gitea-macos-runner vm boot`)
/// and run `doctor` again.
public static func checkGuestSSH(config: RunnerConfig) async -> DoctorCheck {
let name = "guest ssh"
let macs: [String]
do {
macs = try VMStore(config: config).loadState().slotMACAddresses
} catch {
return DoctorCheck(
name: name,
result: .warn,
detail: "could not read host state: \(error)",
remediation: "check that \(config.storeDirectoryURL.path) is readable"
)
}
guard !macs.isEmpty else {
return DoctorCheck(
name: name,
result: .info,
detail: "no slot MAC addresses assigned yet; skipped",
remediation: nil
)
}
// Newest lease wins if a slot somehow holds more than one: that is the
// guest currently on the link.
let leases = DHCPLeaseParser.parseFile()
guard let (mac, lease) = macs.lazy
.compactMap({ mac in DHCPLeaseParser.lease(forMAC: mac, in: leases).map { (mac, $0) } })
.first
else {
return DoctorCheck(
name: name,
result: .info,
detail: "no guest currently holds a DHCP lease; skipped",
remediation: """
this check only runs against a guest that is already up. To exercise the \
host→guest SSH path, run `gitea-macos-runner vm boot --image default` and \
then `doctor` again.
"""
)
}
// Short and fixed rather than derived from `scheduler.bootTimeoutSeconds`:
// this probes a guest that is already booted, so a slow answer is a
// finding, not something to wait fifteen minutes for.
do {
try await waitForSSH(
host: lease.ipAddress,
username: config.guest.username,
password: config.guest.password,
timeout: .seconds(20),
pollInterval: .seconds(2)
)
return DoctorCheck(
name: name,
result: .pass,
detail: "authenticated to \(config.guest.username)@\(lease.ipAddress) (\(mac))"
)
} catch let error as CoreError {
let detail = "\(config.guest.username)@\(lease.ipAddress) (\(mac)): \(error)"
switch error {
case .timeout:
// Nothing answered. bootpd leases last 24 hours and the slot MACs
// are persistent, so on any host that has ever run the daemon the
// most likely explanation is a lease outliving the guest that held
// it — not a broken host. Calling that `.fail` would make `doctor`
// cry wolf on a perfectly healthy idle machine.
return DoctorCheck(
name: name,
result: .warn,
detail: detail,
remediation: """
most likely a stale lease: bootpd keeps leases for 24 hours, so this \
address may belong to a guest that has already been torn down. If a \
guest really is up at this address, the daemon cannot reach it either — \
check that the host's Local Network permission is not dropping the \
connection (see the "local network access" check).
"""
)
default:
// Authentication reached the guest and was refused: the guest is up
// and the credentials are wrong. Nothing about that improves on its
// own, and every boot will fail the same way.
return DoctorCheck(
name: name,
result: .fail,
detail: detail,
remediation: """
the daemon authenticates over this exact path, so no boot can succeed \
while it fails. Check that guest.username and guest.password match an \
account in the guest image, and that Remote Login is enabled there.
"""
)
}
} catch {
return DoctorCheck(
name: name,
result: .warn,
detail: "\(config.guest.username)@\(lease.ipAddress) (\(mac)): \(error)",
remediation: nil
)
}
}
/// The macOS 15+ Local Network permission note.
///
/// Reports `.pass` when the host carries a subnet allowlist that actually
@@ -535,119 +771,65 @@ public enum Doctor {
/// does not apply (Apple, TN3179).
public static func localNetworkNote() -> DoctorCheck {
let name = "local network access"
let allowed = localNetworkAllowlist()
if !allowed.isEmpty {
if allowed.contains(where: coversVMNetRange) {
let status = LocalNetworkPermission.observedStatus()
if status.isConfigured {
if status.coversGuestRange {
return DoctorCheck(
name: name,
result: .pass,
detail: "subnet allowlist set: \(allowed.joined(separator: ", "))"
detail: "subnet allowlist set: \(status.allowlist.joined(separator: ", "))"
)
}
return DoctorCheck(
name: name,
result: .warn,
detail: "subnet allowlist set but does not cover the guest range: "
+ allowed.joined(separator: ", "),
+ status.allowlist.joined(separator: ", "),
remediation: """
Virtualization.framework's NAT does not stay on 192.168.64.0/24 — it moves \
to the next free /24 (192.168.65.x, .66.x, …) when one is taken, so \
an allowlist pinned to a single /24 stops working the day the subnet shifts \
and every guest connection then fails with "No route to host". Widen it to \
cover the whole span: sudo defaults write com.apple.network.local-network \
AllowedEthernetLocalNetworkAddresses -array "192.168.64.0/18" (same for \
AllowedWiFiLocalNetworkAddresses), then reboot. See docs/setup.md §2.6.
and every guest connection then fails with "No route to host". Widen it: \
`gitea-macos-runner permissions grant`, then reboot. See docs/setup.md §2.6.
"""
)
}
// Nothing found. On all but an unusual host that means *not visible*
// rather than *not set*: the allowlist is written as root and lands in
// /var/root, which is mode 700. Say which of the two this is, because
// "no allowlist" would otherwise be asserted on a host that has one.
let caveat =
status.isIndeterminate
? """
This cannot see the setting itself — it lives in \
\(status.unreadablePaths[0]), which only root can read — so treat the above as \
"not visible", not "not set". `sudo gitea-macos-runner permissions status` \
answers definitively, and `permissions grant` verifies its own write.
"""
: ""
return DoctorCheck(
name: name,
result: .info,
detail: "guests are reached over the host-private NAT link",
detail: status.isIndeterminate
? "no allowlist visible; guests are reached over the host-private NAT link"
: "guests are reached over the host-private NAT link",
remediation: """
on macOS 15+ the first connection to a guest can be blocked by the Local Network \
privacy prompt, which a background LaunchAgent cannot answer. The app cannot be \
pre-approved: it only appears under System Settings → Privacy & Security → Local \
Network once it has actually attempted a guest connection. To trigger and answer \
the prompt by hand, run `gitea-macos-runner vm boot --image default` once from a \
Terminal in the GUI session. On an unattended CI host prefer the subnet \
allowlist, which needs no prompt and survives rebuilds: sudo defaults write \
com.apple.network.local-network AllowedEthernetLocalNetworkAddresses -array \
"192.168.64.0/18" (then reboot). See docs/setup.md §2.6.
"""
privacy prompt, and frequently there is nothing able to answer it. A LaunchAgent \
has no UI to show it in; a run started from a shell is attributed to the \
*responsible* process, so both the prompt and the System Settings → Privacy & \
Security → Local Network row belong to Terminal rather than to this app — and \
granting it to Terminal does not carry over to the LaunchAgent. Grant it with \
`gitea-macos-runner permissions grant`, which writes a subnet allowlist that \
needs no prompt, covers every process, and survives rebuilds — then reboot. \
`permissions status` explains both routes. See docs/setup.md §2.6.
""" + caveat
)
}
/// The span of addresses a vmnet NAT link can plausibly use.
///
/// `192.168.64.0/24` is only the *first* choice: the subnet is picked at
/// runtime and steps to the next free /24 when that one is already in use,
/// which is why a host that worked yesterday can hand out `192.168.65.x`
/// today. Everything from 192.168.64.0 to 192.168.127.255 — a /18 — is
/// treated as guest territory so the allowlist survives that drift.
static let vmNetFirstAddress: UInt32 = 0xC0A8_4000 // 192.168.64.0
static let vmNetLastAddress: UInt32 = 0xC0A8_7FFF // 192.168.127.255
/// Whether one allowlist entry covers the whole guest range.
///
/// Deliberately all-or-nothing: partial cover is the failure mode being
/// warned about, so an entry that contains today's subnet but not
/// tomorrow's is not treated as good enough.
static func coversVMNetRange(_ entry: String) -> Bool {
let parts = entry.split(separator: "/", maxSplits: 1)
guard let base = ipv4Value(String(parts[0])) else { return false }
let prefix = parts.count == 2 ? Int(parts[1]) : 32
guard let prefix, (0...32).contains(prefix) else { return false }
let mask: UInt32 = prefix == 0 ? 0 : ~UInt32(0) << (32 - prefix)
let network = base & mask
let broadcast = network | ~mask
return network <= vmNetFirstAddress && broadcast >= vmNetLastAddress
}
/// Packs dotted-quad IPv4 into a comparable integer; nil for anything else
/// (an IPv6 entry, a hostname, a typo).
static func ipv4Value(_ text: String) -> UInt32? {
let octets = text.split(separator: ".", omittingEmptySubsequences: false)
guard octets.count == 4 else { return nil }
var value: UInt32 = 0
for octet in octets {
guard let number = UInt32(octet), number <= 255 else { return nil }
value = value << 8 | number
}
return value
}
/// Subnets pre-authorized for local network access on this host, if any.
///
/// Best effort and never fatal: an unreadable or absent preferences file
/// simply reads as "no allowlist". The domain is written with `sudo`, so
/// which preferences directory it lands in depends on whether that `sudo`
/// preserved `HOME` — check each candidate rather than guess.
static func localNetworkAllowlist() -> [String] {
let keys = ["AllowedEthernetLocalNetworkAddresses", "AllowedWiFiLocalNetworkAddresses"]
let candidates = [
"/var/root/Library/Preferences/com.apple.network.local-network.plist",
"/Library/Preferences/com.apple.network.local-network.plist",
NSHomeDirectory() + "/Library/Preferences/com.apple.network.local-network.plist",
]
var found: [String] = []
for path in candidates {
guard let data = FileManager.default.contents(atPath: path),
let plist = try? PropertyListSerialization.propertyList(
from: data, options: [], format: nil) as? [String: Any]
else { continue }
for key in keys {
for entry in (plist[key] as? [String] ?? []) where !found.contains(entry) {
found.append(entry)
}
}
}
return found
}
/// Renders checks as aligned, human-readable lines for the CLI.
public static func format(_ checks: [DoctorCheck]) -> String {
let width = checks.map(\.name.count).max() ?? 0
+57 -2
View File
@@ -47,10 +47,25 @@ public struct ServiceStatus: Sendable, Equatable {
/// because this is the failure people hit first.
public enum LaunchdService {
/// The `launchd` label, matching `CFBundleIdentifier`.
public static let label = "xyz.blakeslee.gitea-macos-runner"
public static let label = "xyz.blakeslee.gitea-macos-vm-orchestrator"
/// `~/Library/LaunchAgents/xyz.blakeslee.gitea-macos-runner.plist`.
/// Labels this service used to install under.
///
/// Renaming the label renames the plist, so an upgrade that only wrote the
/// new one would leave the old job bootstrapped and still running the old
/// binary — two daemons polling the same Gitea instance, racing to claim
/// the same queued jobs, with no hint in the logs that a second one exists.
/// ``install(executablePath:configPath:)`` and ``uninstall()`` therefore
/// evict these first. Append, never edit, when the label changes again.
public static let legacyLabels = ["xyz.blakeslee.gitea-macos-runner"]
/// `~/Library/LaunchAgents/xyz.blakeslee.gitea-macos-vm-orchestrator.plist`.
public static var agentPlistURL: URL {
agentPlistURL(for: label)
}
/// The LaunchAgent plist path for an arbitrary label.
public static func agentPlistURL(for label: String) -> URL {
URL(fileURLWithPath: RunnerConfig.expandTilde("~/Library/LaunchAgents/\(label).plist"))
}
@@ -106,6 +121,11 @@ public enum LaunchdService {
withIntermediateDirectories: true
)
// Upgrading from a build that installed under an older label: evict it
// before bootstrapping this one, or both run at once. See
// ``legacyLabels``.
removeLegacyAgents()
// A reinstall over a loaded job is the common case (upgrade, config
// change), so unload before rewriting rather than failing on "already
// bootstrapped".
@@ -140,13 +160,48 @@ public enum LaunchdService {
}
/// Unloads the job and removes the plist. Safe when not installed.
///
/// Also evicts any ``legacyLabels`` job, so `service uninstall` leaves
/// nothing of this project loaded regardless of which version installed it.
public static func uninstall() throws {
removeLegacyAgents()
_ = try? uninstallJobOnly()
if FileManager.default.fileExists(atPath: agentPlistURL.path) {
try FileManager.default.removeItem(at: agentPlistURL)
}
}
/// Boots out and deletes any LaunchAgent installed under a ``legacyLabels``
/// entry.
///
/// Best effort by design: a legacy job that was never installed, is not
/// loaded, or whose plist is already gone is not an error, and failing to
/// evict one must not block installing the current job.
///
/// - Returns: The legacy labels that were actually found and removed, for
/// callers that want to tell the operator a migration happened.
@discardableResult
public static func removeLegacyAgents() -> [String] {
var removed: [String] = []
for legacy in legacyLabels {
let plist = agentPlistURL(for: legacy)
let bootout = LaunchdShell.run(
"/bin/launchctl", ["bootout", "\(domainTarget)/\(legacy)"])
if bootout.exitCode != 0 {
_ = LaunchdShell.run("/bin/launchctl", ["unload", "-w", plist.path])
}
if FileManager.default.fileExists(atPath: plist.path) {
try? FileManager.default.removeItem(at: plist)
removed.append(legacy)
} else if bootout.exitCode == 0 {
// Loaded, but from a plist that is no longer on disk.
removed.append(legacy)
}
}
return removed
}
/// Unloads the job but leaves the plist on disk.
private static func uninstallJobOnly() throws {
let bootout = LaunchdShell.run("/bin/launchctl", ["bootout", serviceTarget])
@@ -0,0 +1,441 @@
import AppKit
import Darwin
import Foundation
import RunnerCore
/// Grants the host's Local Network access, so the operator does not have to
/// paste `sudo defaults write` incantations and work out for themselves that a
/// reboot is required.
///
/// Two routes, with genuinely different trade-offs — see ``Method``:
///
/// - ``grantViaAllowlist(subnets:allowPasswordPrompt:)`` writes the subnet
/// allowlist. Deterministic and process-independent, but inert until reboot.
/// - ``triggerPrompt(timeout:)`` provokes the real system prompt, attributed to
/// *this app* rather than to Terminal. Takes effect immediately, but depends
/// on macOS actually presenting the alert.
///
/// The pure parts — where the setting lives, what covers the guest range, what
/// to write — are `RunnerCore`'s ``LocalNetworkPolicy``. This type is the half
/// that runs processes.
public enum LocalNetworkPermission {
/// How to obtain the grant.
public enum Method: String, CaseIterable, Sendable {
/// Write the subnet allowlist. Needs `sudo` and a reboot.
case allowlist
/// Provoke the system prompt via LaunchServices. Needs a GUI session.
case prompt
}
/// The bundle identifier of the installed app.
///
/// Must match `Resources/Info.plist`. It is the same string as
/// ``LaunchdService/label`` by convention — the agent is named after the
/// bundle it launches — but they are read by different subsystems, so this
/// spells it out rather than aliasing.
public static let bundleIdentifier = "xyz.blakeslee.gitea-macos-vm-orchestrator"
// MARK: - Errors
public enum PermissionError: Error, CustomStringConvertible {
/// `sudo` could not be run non-interactively and there is no terminal
/// to prompt on. Carries the commands to run by hand.
case needsPassword(commands: [String])
/// A `defaults write` exited non-zero.
case writeFailed(command: String, exitCode: Int32)
/// `open -b` could not find the app.
case bundleNotRegistered
/// The probe process ran but left no report behind.
case probeProducedNoReport
/// A subnet argument is not an IPv4 address or CIDR block.
case invalidSubnet(String)
/// A child process could not be started or waited on.
case spawnFailed(command: String, code: Int32)
public var description: String {
switch self {
case .needsPassword(let commands):
return """
this needs administrator rights and stdin is not a terminal, so there is \
nowhere to prompt for a password. Run these by hand, then reboot:
""" + commands.map { "\n " + $0 }.joined()
case .writeFailed(let command, let exitCode):
return "`\(command)` exited \(exitCode)"
case .bundleNotRegistered:
return """
the signed app bundle is not installed, so it cannot be launched as its own \
responsible process — which is the entire point of this method. Install it \
with `make install`, or use --method allowlist instead.
"""
case .probeProducedNoReport:
return "the probe exited without writing a result"
case .invalidSubnet(let entry):
return """
"\(entry)" is not an IPv4 address or CIDR block. macOS silently ignores \
entries it cannot parse, which would leave the allowlist looking configured \
while granting nothing.
"""
case .spawnFailed(let command, let code):
return "could not run \(command): \(String(cString: strerror(code))) (\(code))"
}
}
}
// MARK: - Headless: the subnet allowlist
/// What ``grantViaAllowlist(subnets:allowPasswordPrompt:)`` actually achieved.
public struct AllowlistResult: Sendable {
/// The subnets we asked for.
public let requested: [String]
/// What reading the preferences back afterwards found.
public let observed: LocalNetworkPolicy.Status
/// Whether every requested subnet is now readable on disk.
public var verified: Bool {
requested.allSatisfy(observed.allowlist.contains)
}
}
/// Writes the subnet allowlist, then reads it back to prove it landed.
///
/// The read-back is not ceremony. `sudo defaults write <domain>` resolves
/// the domain relative to whichever `HOME` survived `sudo`'s `env_reset`,
/// which differs between hosts — so the only way to know where the file
/// went is to look. A write that succeeds but leaves nothing readable is
/// reported as unverified rather than as success.
///
/// - Parameters:
/// - subnets: CIDR entries to authorize.
/// - allowPasswordPrompt: When true, `sudo` inherits this process's
/// terminal and may ask for a password. When false it runs `-n` and
/// fails rather than blocking — the right behaviour under `launchd` or
/// in a pipeline.
/// - Returns: The requested subnets and what is now on disk.
public static func grantViaAllowlist(
subnets: [String] = LocalNetworkPolicy.defaultSubnets,
allowPasswordPrompt: Bool
) throws -> AllowlistResult {
for subnet in subnets where !LocalNetworkPolicy.isValidSubnet(subnet) {
throw PermissionError.invalidSubnet(subnet)
}
let commands = LocalNetworkPolicy.writeCommandLines(subnets: subnets)
for (index, arguments) in LocalNetworkPolicy.writeArguments(subnets: subnets).enumerated() {
let sudoArguments = (allowPasswordPrompt ? [] : ["-n"]) + ["/usr/bin/defaults"] + arguments
let exitCode = try runInForeground("/usr/bin/sudo", sudoArguments)
guard exitCode == 0 else {
if !allowPasswordPrompt {
throw PermissionError.needsPassword(commands: commands)
}
throw PermissionError.writeFailed(command: commands[index], exitCode: exitCode)
}
}
return AllowlistResult(requested: subnets, observed: observedStatus())
}
/// The host's allowlist, read with root's privileges when this process's
/// own are not enough.
///
/// `sudo defaults write <domain>` lands in `/var/root/Library/Preferences`
/// on a stock host, and that directory is mode 700 — so the plain read in
/// ``LocalNetworkPolicy/status()`` is refused and a write that worked
/// perfectly looks like it vanished. Re-read the refused candidates as
/// root instead.
///
/// Always `sudo -n`, so this can never turn a status query into a password
/// prompt. Right after a write the credentials are still cached and it
/// simply works; later — after a reboot, say — it fails and the result
/// stays ``LocalNetworkPolicy/Status/isIndeterminate``, which callers
/// report as "cannot tell without root" rather than as "not configured".
public static func observedStatus() -> LocalNetworkPolicy.Status {
let unprivileged = LocalNetworkPolicy.status()
guard !unprivileged.unreadablePaths.isEmpty else { return unprivileged }
var sources: [(path: String, data: Data)] = []
for path in LocalNetworkPolicy.preferenceCandidates() {
if let data = FileManager.default.contents(atPath: path) {
sources.append((path, data))
} else if let data = readAsRoot(path) {
sources.append((path, data))
}
}
let recovered = LocalNetworkPolicy.status(fromContentsOf: sources)
// Nothing came back from the privileged read either: keep the
// unprivileged answer, which still carries why it could not tell.
guard recovered.isConfigured else { return unprivileged }
return recovered
}
/// `sudo -n cat <path>`, or nil if that fails for any reason.
///
/// Both failure modes are ordinary rather than exceptional — the candidate
/// usually does not exist, and `sudo -n` legitimately refuses when no
/// credentials are cached — so stderr is discarded instead of being shown
/// to the operator. `Process` is fine here, unlike in ``runInForeground``:
/// `-n` never touches the terminal.
private static func readAsRoot(_ path: String) -> Data? {
let process = Process()
process.executableURL = URL(fileURLWithPath: "/usr/bin/sudo")
process.arguments = ["-n", "/bin/cat", path]
let output = Pipe()
process.standardOutput = output
process.standardError = FileHandle.nullDevice
process.standardInput = FileHandle.nullDevice
guard (try? process.run()) != nil else { return nil }
let data = output.fileHandleForReading.readDataToEndOfFile()
process.waitUntilExit()
guard process.terminationStatus == 0, !data.isEmpty else { return nil }
return data
}
/// Reboots the host. Only ever called from an explicit confirmation — the
/// allowlist is read at boot, so nothing else makes it take effect.
public static func reboot() throws {
_ = try runInForeground("/usr/bin/sudo", ["/sbin/shutdown", "-r", "now"])
}
/// Runs a command with this process's stdio *and its process group*, and
/// returns its exit status.
///
/// The process group is the whole reason this is not `Foundation.Process`.
/// `Process` starts the child as its own process-group leader, so for the
/// controlling terminal the child is a *background* job — and the terminal
/// driver defends itself against those. `sudo`'s `tcsetattr` to turn echo
/// off raises `SIGTTOU` and fails, so the password is typed in the clear;
/// its read of the tty raises `SIGTTIN`, so Return never reaches `sudo` and
/// the line editor just echoes a newline. Both symptoms, one cause.
///
/// `posix_spawn` with no `POSIX_SPAWN_SETPGROUP` leaves the child in our
/// process group, which is the terminal's foreground group, so `sudo` gets
/// the terminal it expects. Stdio is inherited for the same reason it
/// always was: the prompt and any "not in the sudoers file" complaint
/// belong in front of the operator, not captured and paraphrased.
private static func runInForeground(_ executable: String, _ arguments: [String]) throws -> Int32
{
var argv: [UnsafeMutablePointer<CChar>?] = ([executable] + arguments).map { strdup($0) }
argv.append(nil)
var envp: [UnsafeMutablePointer<CChar>?] = ProcessInfo.processInfo.environment.map {
strdup("\($0.key)=\($0.value)")
}
envp.append(nil)
defer {
for pointer in argv { free(pointer) }
for pointer in envp { free(pointer) }
}
var pid: pid_t = 0
let spawned = posix_spawn(&pid, executable, nil, nil, argv, envp)
guard spawned == 0 else {
throw PermissionError.spawnFailed(command: executable, code: spawned)
}
var status: Int32 = 0
while waitpid(pid, &status, 0) < 0 {
guard errno == EINTR else {
throw PermissionError.spawnFailed(command: executable, code: errno)
}
}
// WIFEXITED and friends are C macros, so Swift does not import them.
let terminatingSignal = status & 0x7F
return terminatingSignal == 0 ? (status >> 8) & 0xFF : 128 + terminatingSignal
}
// MARK: - Interactive: the system prompt
/// What a probe observed.
public enum ProbeOutcome: String, Codable, Sendable {
/// Datagrams left the host. Either the app is allowed, or the system is
/// still deciding — macOS drops packets silently while the prompt is up
/// rather than failing the send, so this is "not blocked", not proof.
case permitted
/// Every send came back `EHOSTUNREACH`. That is what Local Network
/// privacy returns when it blocks an app.
case blocked
/// The socket failed for some unrelated reason.
case inconclusive
}
/// A probe result, serialized through a temp file because the probe runs in
/// a separate process launched by LaunchServices.
public struct ProbeReport: Codable, Sendable {
public let outcome: ProbeOutcome
public let detail: String
public init(outcome: ProbeOutcome, detail: String) {
self.outcome = outcome
self.detail = detail
}
}
/// Launches the installed bundle so it provokes the Local Network prompt
/// **as itself**, then reports what the launched process observed.
///
/// The launch is the whole trick. Running this binary from a shell makes
/// Terminal the *responsible process*, so the prompt and the System
/// Settings row name Terminal — and a grant to Terminal does nothing for
/// the LaunchAgent. Going through LaunchServices (`open -b`) makes the app
/// its own responsible process, so the grant attaches to the app's code
/// identity and the agent inherits it.
///
/// That only holds because the bundle is Developer ID signed: a team
/// anchored designated requirement is a stable identity across rebuilds.
/// Under an ad-hoc signature macOS falls back to the Mach-O UUID, which the
/// linker regenerates on every link, and the grant would not survive the
/// next `make install`.
///
/// - Parameter timeout: How long to let the child wait for a verdict. It
/// needs to outlast a human reading the alert.
public static func triggerPrompt(timeout: TimeInterval = 90) throws -> ProbeReport {
let reportURL = FileManager.default.temporaryDirectory
.appendingPathComponent("gmr-probe-\(UUID().uuidString).json")
defer { try? FileManager.default.removeItem(at: reportURL) }
let process = Process()
process.executableURL = URL(fileURLWithPath: "/usr/bin/open")
process.arguments = [
"-n", // a fresh instance; an already-running daemon must not be reused
"-b", bundleIdentifier,
"--wait-apps",
"--args", "permissions", "probe",
"--report", reportURL.path,
"--timeout", String(Int(timeout)),
]
// `open` reports "Unable to find application" on stderr; let it through.
try process.run()
process.waitUntilExit()
guard process.terminationStatus == 0 else { throw PermissionError.bundleNotRegistered }
guard let data = FileManager.default.contents(atPath: reportURL.path),
let report = try? JSONDecoder().decode(ProbeReport.self, from: data)
else { throw PermissionError.probeProducedNoReport }
return report
}
/// The child side of ``triggerPrompt(timeout:)``: touch the local network
/// and report whether the packets got out.
///
/// Sends to the broadcast address and to mDNS multicast, which is what
/// makes macOS classify this as local-network traffic and raise the prompt.
/// Deliberately does not boot a VM — no guest is needed to trigger the
/// check, and this path therefore needs none of the `NSApplication`
/// plumbing `VZAppRuntime` exists for.
///
/// Retries until `deadline` because the verdict is not synchronous: while
/// the alert is on screen the system neither fails the send nor delivers
/// the packet, so a single attempt cannot distinguish "allowed" from "still
/// asking". Looping until the operator answers is what turns it into a
/// usable signal.
public static func probe(timeout: TimeInterval = 90) async -> ProbeReport {
await activateForPrompt()
let deadline = Date().addingTimeInterval(timeout)
var lastErrno: Int32 = 0
var attempts = 0
repeat {
attempts += 1
guard let code = sendLocalNetworkDatagrams() else {
return ProbeReport(
outcome: .permitted,
detail: attempts == 1
? "local network traffic was not blocked"
: "local network traffic was allowed after \(attempts) attempts"
)
}
lastErrno = code
// Anything other than the privacy filter's answer is a real socket
// problem; retrying will not change it.
guard code == EHOSTUNREACH else {
return ProbeReport(
outcome: .inconclusive,
detail: "socket error \(code): \(describeErrno(code))"
)
}
// Deliberately not Thread.sleep: activateForPrompt just put this
// process in the foreground, and a main thread wedged in a sleep is
// a process macOS will show as unresponsive while the alert it is
// waiting on is on screen.
try? await Task.sleep(nanoseconds: 1_000_000_000)
} while Date() < deadline
return ProbeReport(
outcome: .blocked,
detail: "every send over \(attempts) attempts returned EHOSTUNREACH (errno \(lastErrno))"
)
}
/// Sends one datagram to the broadcast address and one to mDNS multicast.
///
/// - Returns: `nil` if either got out, otherwise the last `errno`.
private static func sendLocalNetworkDatagrams() -> Int32? {
// Port 9 is discard; 5353 is mDNS. Nothing has to be listening — the
// privacy filter makes its decision on the send, not on a reply.
let targets: [(address: String, port: UInt16)] = [
("255.255.255.255", 9),
("224.0.0.251", 5353),
]
var lastErrno: Int32 = EINVAL
for target in targets {
let handle = socket(AF_INET, SOCK_DGRAM, 0)
guard handle >= 0 else {
lastErrno = errno
continue
}
defer { close(handle) }
var enable: Int32 = 1
setsockopt(handle, SOL_SOCKET, SO_BROADCAST, &enable, socklen_t(MemoryLayout<Int32>.size))
var destination = sockaddr_in()
destination.sin_family = sa_family_t(AF_INET)
destination.sin_port = target.port.bigEndian
destination.sin_addr.s_addr = inet_addr(target.address)
let payload: [UInt8] = [0]
let sent = withUnsafePointer(to: &destination) { pointer in
pointer.withMemoryRebound(to: sockaddr.self, capacity: 1) { address in
sendto(handle, payload, payload.count, 0, address, socklen_t(MemoryLayout<sockaddr_in>.size))
}
}
if sent >= 0 { return nil }
lastErrno = errno
}
return lastErrno
}
/// `strerror`, with the optionality unwrapped.
private static func describeErrno(_ code: Int32) -> String {
guard let text = strerror(code) else { return "unknown error" }
return String(cString: text)
}
/// Brings the probe process forward so the system alert has a frontmost app
/// to attach to.
///
/// `LSUIElement` in `Info.plist` would otherwise leave this at `.accessory`.
/// The daemon wants that — it goes further and sets `.prohibited` — but a
/// prompt nobody can see is the exact failure this command exists to fix,
/// so the probe opts back in. It starts no VM, so it is not bound by the
/// activation policy `VZAppRuntime` needs.
@MainActor
private static func activateForPrompt() {
let app = NSApplication.shared
app.setActivationPolicy(.regular)
app.activate(ignoringOtherApps: true)
}
}
+89 -6
View File
@@ -50,7 +50,7 @@ public struct LiveVM: Sendable {
///
/// `ensureFreeSpace` → `cloneImage(named:slotMAC:)` → ``VMInstance/start(options:)``
/// → poll `/var/db/dhcpd_leases` for the slot MAC until `bootTimeout` →
/// ``waitForSSH(host:port:username:password:timeout:pollInterval:)`` → write the
/// ``waitForSSH(host:port:username:password:timeout:pollInterval:reportInterval:onAttemptFailure:)`` → write the
/// registration token into a guest file with mode `0600` → over SSH:
///
/// ```sh
@@ -105,6 +105,15 @@ public actor Orchestrator {
/// The supervising task per slot: clone → boot → register → run → teardown.
private var slotTasks: [Int: Task<Void, Never>] = [:]
/// Why a slot's supervising task was cancelled, left for that task to find.
///
/// `Task.cancel()` carries no payload and `CancellationError` no detail, so
/// a lifecycle that catches one knows only *that* it was stopped. Everything
/// worth reading — "boot timeout: provisioning for 312s (limit 300s)" — is
/// known only to the canceller. Without this hand-off the operator sees
/// `reason=cancelled`, which names the mechanism and hides the cause.
private var slotCancelReasons: [Int: String] = [:]
/// The "the guest stopped on its own" watcher per slot.
private var deathWatchTasks: [Int: Task<Void, Never>] = [:]
@@ -252,6 +261,7 @@ public actor Orchestrator {
// let them run their own teardown; whatever they miss we clean up below.
let tasks = slotTasks
slotTasks.removeAll()
for (slot, _) in tasks { slotCancelReasons[slot] = "shutting down" }
for (_, task) in tasks { task.cancel() }
for (_, task) in tasks { await task.value }
@@ -307,6 +317,9 @@ public actor Orchestrator {
// are about to kill, so it would otherwise clear that entry only
// after the boot had already been refused.
if let task = slotTasks.removeValue(forKey: slot) {
// Before the cancel, not after: the task may reach its
// `catch` the instant it is cancelled.
slotCancelReasons[slot] = reason
task.cancel()
// Its own teardown runs to completion here, which also means
// it cannot race a successor booted later in this pass.
@@ -404,6 +417,9 @@ public actor Orchestrator {
let generation = (slotGeneration[slot] ?? 0) + 1
slotGeneration[slot] = generation
// Taken alongside the `Date()` handed to the planner, so the lifecycle
// can measure against the same deadline the planner will enforce.
let provisioningStarted = ContinuousClock.now
state = SchedulerCore.markProvisioning(state: state, slot: slot, jobHint: jobHint, now: Date())
logger.info(
@@ -417,7 +433,13 @@ public actor Orchestrator {
slotTasks[slot] = Task { [weak self] in
guard let self else { return }
await self.runSlotLifecycle(slot: slot, jobHint: jobHint, runnerName: runnerName, generation: generation)
await self.runSlotLifecycle(
slot: slot,
jobHint: jobHint,
runnerName: runnerName,
generation: generation,
provisioningStarted: provisioningStarted
)
}
}
@@ -425,11 +447,40 @@ public actor Orchestrator {
///
/// Every failure path funnels into the same teardown, because a slot that is
/// neither live nor idle is a slot leaked for the process's lifetime.
private func runSlotLifecycle(slot: Int, jobHint: Int64, runnerName: String, generation: Int) async {
///
/// - Parameter provisioningStarted: When the *planner's* boot deadline began
/// ticking — earlier than this function's own first instruction. Used to
/// budget the readiness waits against the deadline that will actually be
/// enforced rather than against a fresh copy of it.
private func runSlotLifecycle(
slot: Int,
jobHint: Int64,
runnerName: String,
generation: Int,
provisioningStarted: ContinuousClock.Instant
) async {
let bootTimeout = Duration.seconds(max(30, config.scheduler.bootTimeoutSeconds))
let jobTimeout = Duration.seconds(max(60, config.scheduler.jobTimeoutMinutes * 60))
var teardownReason = "job finished"
// Both readiness waits below are already supervised by the planner's
// boot deadline, which started ticking at `provisioningStarted` — before
// the clone, the boot and the DHCP lease had spent any of it. Handing
// either wait the full `bootTimeout` puts its deadline strictly *after*
// the planner's, so the planner always wins the race: this task is
// cancelled mid-wait and the specific error the wait was about to throw
// ("ssh on 192.168.65.233:22 after 41 attempts; last error: …") is
// discarded in favour of a bare cancellation. Budget from what is left
// and the wait gets to speak first.
func remainingBootBudget() -> Duration {
let spent = ContinuousClock.now - provisioningStarted
// Landing a little before the planner, so its next tick finds the
// slot already failing for a stated reason. The floor keeps an
// already-overrun budget from collapsing to zero attempts, which
// would trade one useless message for another.
return max(.seconds(15), bootTimeout - spent - .seconds(5))
}
do {
let mac = try store.macAddress(forSlot: slot, slotCount: slotCount)
// Whatever lease this MAC already holds belongs to the *previous*
@@ -454,15 +505,40 @@ public actor Orchestrator {
await self?.vmStoppedUnexpectedly(slot: slot, generation: generation, reason: reason)
}
let ip = try await waitForLease(mac: mac, timeout: bootTimeout, replacing: priorLease)
let ip = try await waitForLease(
mac: mac,
timeout: remainingBootBudget(),
replacing: priorLease
)
live[slot]?.ipAddress = ip
logger.info("guest leased address", metadata: ["slot": .stringConvertible(slot), "ip": .string(ip)])
// A `Logger` is a value type, so the callback below gets its own
// copy and never touches the actor — which is what lets it be a
// plain synchronous closure called from inside the poll loop.
let log = logger
try await waitForSSH(
host: ip,
username: config.guest.username,
password: config.guest.password,
timeout: bootTimeout
timeout: remainingBootBudget(),
onAttemptFailure: { attempt in
// A guest sharing a host with other Virtualization guests
// can take minutes to start `sshd`. Without this the wait is
// indistinguishable from a hang: the log goes quiet between
// "guest leased address" and teardown, which is exactly the
// window an operator most wants to see into.
log.info(
"waiting for guest ssh",
metadata: [
"slot": .stringConvertible(slot),
"host": .string(ip),
"attempt": .stringConvertible(attempt.attempt),
"elapsed": .string("\(attempt.elapsed.components.seconds)s"),
"error": .string(attempt.error),
]
)
}
)
let token = try await registrationToken()
@@ -511,7 +587,9 @@ public actor Orchestrator {
)
}
} catch is CancellationError {
teardownReason = "cancelled"
// Whoever cancelled us knows why; `CancellationError` does not.
// Falling back to "cancelled" only when nobody left a note.
teardownReason = slotCancelReasons[slot] ?? "cancelled"
} catch {
teardownReason = "\(error)"
logger.error(
@@ -539,6 +617,10 @@ public actor Orchestrator {
state = SchedulerCore.releaseJob(state: state, jobID: jobHint)
}
// A note left for a cancel that arrived after the lifecycle had already
// finished on its own would otherwise be read by the *next* occupant of
// this slot, mislabelling its teardown.
slotCancelReasons[slot] = nil
slotTasks[slot] = nil
}
@@ -551,6 +633,7 @@ public actor Orchestrator {
"guest stopped unexpectedly",
metadata: ["slot": .stringConvertible(slot), "reason": .string("\(reason)")]
)
slotCancelReasons[slot] = "guest stopped: \(reason)"
slotTasks[slot]?.cancel()
await teardownSlot(slot, reason: "guest stopped: \(reason)")
}
@@ -133,7 +133,7 @@ enum VZAppRuntime {
/// Signals land here rather than on `.main`. See ``run(onSignal:body:)``.
private static let signalQueue = DispatchQueue(
label: "xyz.blakeslee.gitea-macos-runner.signals")
label: "xyz.blakeslee.gitea-macos-vm-orchestrator.signals")
/// Starts the run loop and runs `body` alongside it. Never returns.
///
@@ -0,0 +1,282 @@
import ArgumentParser
import Foundation
import RunnerCore
import RunnerHost
/// `gitea-macos-runner permissions …` — inspect and grant the macOS 15+ Local
/// Network access the runner needs to reach its guests.
///
/// This exists because the alternative was a paragraph of documentation asking
/// the operator to paste two `sudo defaults write` lines and reboot. That is
/// the single most common way a freshly installed runner fails — every guest
/// boots, no job ever starts, and the only symptom is `No route to host`.
struct PermissionsCommand: AsyncParsableCommand {
static let configuration = CommandConfiguration(
commandName: "permissions",
abstract: "Inspect and grant the macOS Local Network access guests are reached over.",
discussion: """
macOS 15 and newer filter local-network traffic per app. When the runner is \
blocked the connection fails with "No route to host", which looks exactly \
like a guest that is off the network — so this is worth checking before \
debugging anything else.
`permissions grant` offers two routes. The default subnet allowlist is \
deterministic and applies to every process, but is read at boot, so it \
needs a reboot. `--method prompt` provokes the real system prompt and \
applies immediately, but needs a GUI session and an installed app bundle.
""",
subcommands: [Status.self, Grant.self, Probe.self]
)
/// `permissions status` — what is configured, and what to do about it.
struct Status: AsyncParsableCommand {
static let configuration = CommandConfiguration(
commandName: "status",
abstract: "Report whether local network access is configured."
)
@OptionGroup var options: GlobalOptions
func run() async throws {
// The same two checks `doctor` runs, rendered the same way. Code
// identity belongs here because it decides whether an interactive
// grant survives the next build — an ad-hoc signature makes
// --method prompt a waste of the operator's time.
print(Doctor.format([
Doctor.localNetworkNote(),
Doctor.checkCodeSignature(),
]))
let status = LocalNetworkPermission.observedStatus()
if !status.sourcePaths.isEmpty {
print("")
for path in status.sourcePaths {
print("allowlist read from: \(path)")
}
}
if status.isIndeterminate {
print("")
print("The allowlist lives in root's preferences, which only root can read, so")
print("this cannot tell whether it is already set. For a definitive answer:")
print(" sudo gitea-macos-runner permissions status")
}
guard !status.coversGuestRange else { return }
print("")
// Not visible is not the same as not set, and an unconfigured host
// looks identical to a configured one from an ordinary login — so
// offer the commands without asserting anything is broken.
print(status.isIndeterminate ? "if it is not set, either of these sets it:" : "to fix:")
print(" gitea-macos-runner permissions grant # subnet allowlist, needs a reboot")
print(" gitea-macos-runner permissions grant --method prompt # system prompt, takes effect at once")
}
}
/// `permissions grant` — actually configure it.
struct Grant: AsyncParsableCommand {
static let configuration = CommandConfiguration(
commandName: "grant",
abstract: "Grant local network access to the guest subnets.",
discussion: """
The default `allowlist` method writes com.apple.network.local-network with \
sudo, so it will ask for your password, and the values are only read at \
boot — nothing changes until you reboot.
`--method prompt` instead launches the installed app bundle through \
LaunchServices so it becomes its own responsible process, and provokes the \
system prompt as *this app* rather than as Terminal. That distinction is \
the whole point: a grant given to Terminal does not carry over to the \
LaunchAgent. It takes effect immediately, but needs `make install` to have \
run and a GUI session to show the alert in.
"""
)
@OptionGroup var options: GlobalOptions
@Option(name: .long, help: "How to grant it: allowlist (default) or prompt.")
var method: LocalNetworkPermission.Method = .allowlist
@Option(
name: .long,
parsing: .singleValue,
help: ArgumentHelp(
"Subnet to authorize, repeatable. Defaults to all of RFC 1918.",
valueName: "cidr"
))
var subnet: [String] = []
@Flag(
inversion: .prefixedNo,
help: "Reboot when the allowlist is written. Default: ask, when on a terminal.")
var reboot: Bool?
func run() async throws {
try LocalNetworkGrantFlow.run(method: method, subnets: subnet, reboot: reboot)
}
}
/// `permissions probe` — the child half of `grant --method prompt`.
///
/// Hidden because it is not something to run directly: invoked from a shell
/// it is attributed to Terminal, which is precisely the attribution the
/// prompt method exists to avoid. It is only meaningful when LaunchServices
/// started it.
struct Probe: AsyncParsableCommand {
static let configuration = CommandConfiguration(
commandName: "probe",
abstract: "Internal: touch the local network and report whether it was blocked.",
shouldDisplay: false
)
@Option(name: .long, help: "Where to write the JSON result.")
var report: String?
@Option(name: .long, help: "Seconds to wait for a verdict.")
var timeout: Int = 90
func run() async throws {
let result = await LocalNetworkPermission.probe(timeout: TimeInterval(timeout))
guard let report else {
print("\(result.outcome.rawValue): \(result.detail)")
return
}
try JSONEncoder().encode(result).write(to: URL(fileURLWithPath: report))
}
}
}
extension LocalNetworkPermission.Method: ExpressibleByArgument {}
/// The operator-facing grant flow, shared by `permissions grant` and the
/// `service install` hook.
///
/// It lives outside both so `service install` does not have to construct
/// another command's `ParsableCommand` and mutate its parsed properties, which
/// works only by accident of how ArgumentParser synthesizes initializers.
enum LocalNetworkGrantFlow {
/// Runs one grant, end to end, printing what happened.
///
/// - Parameters:
/// - method: Allowlist or system prompt.
/// - subnets: Empty means the RFC 1918 default. Allowlist only.
/// - reboot: `nil` asks, when there is a terminal to ask on.
static func run(
method: LocalNetworkPermission.Method,
subnets: [String] = [],
reboot: Bool? = nil
) throws {
switch method {
case .allowlist: try grantAllowlist(subnets: subnets, reboot: reboot)
case .prompt: try grantByPrompt(subnets: subnets)
}
}
private static func grantAllowlist(subnets requested: [String], reboot: Bool?) throws {
let subnets = requested.isEmpty ? LocalNetworkPolicy.defaultSubnets : requested
let interactive = isatty(fileno(stdin)) == 1
CLI.note("authorizing \(subnets.joined(separator: ", ")) for local network access")
if interactive {
CLI.note("this needs administrator rights; sudo may ask for your password")
}
let result: LocalNetworkPermission.AllowlistResult
do {
result = try LocalNetworkPermission.grantViaAllowlist(
subnets: subnets, allowPasswordPrompt: interactive)
} catch let error as LocalNetworkPermission.PermissionError {
CLI.error("\(error)")
throw ExitCode(1)
}
// `defaults` reports success regardless of which preferences directory
// the write landed in, so report what was read back rather than what
// was asked for. See LocalNetworkPermission.grantViaAllowlist.
if result.verified {
print("granted: \(result.observed.allowlist.joined(separator: ", "))")
for path in result.observed.sourcePaths {
print("written to: \(path)")
}
if !result.observed.coversGuestRange {
CLI.note("""
warning: none of these cover the whole guest range (192.168.64.0/18), \
so guests will still be blocked once the NAT subnet shifts
""")
}
} else if result.observed.isIndeterminate {
// The write succeeded but there is no way to look: the allowlist
// lands in root's preferences, and this host does not keep sudo
// credentials cached long enough for the read-back to use them.
// Unknown is not failure — say so plainly rather than either
// claiming success or crying wolf.
CLI.note("""
wrote \(subnets.joined(separator: ", ")), but could not read it back to \
confirm — that needs administrator rights this process no longer holds. \
Check it with: sudo defaults read \(LocalNetworkPolicy.domain)
""")
} else {
CLI.error("""
the write reported success but the values could not be read back. \
Check by hand: sudo defaults read \(LocalNetworkPolicy.domain)
""")
throw ExitCode(1)
}
print("")
print("This is read at boot, so it does nothing until the host reboots.")
guard reboot ?? CLI.confirm("reboot now?") else {
CLI.note("not rebooting; run `sudo shutdown -r now` when convenient")
return
}
try LocalNetworkPermission.reboot()
}
private static func grantByPrompt(subnets: [String]) throws {
guard subnets.isEmpty else {
CLI.error("--subnet applies to --method allowlist only; the system prompt is not per-subnet")
throw ExitCode(2)
}
CLI.note("launching the app bundle so the prompt is attributed to it, not to Terminal")
CLI.note("answer \"Allow\" in the alert that appears")
let report: LocalNetworkPermission.ProbeReport
do {
report = try LocalNetworkPermission.triggerPrompt()
} catch let error as LocalNetworkPermission.PermissionError {
CLI.error("\(error)")
throw ExitCode(1)
}
switch report.outcome {
case .permitted:
print("local network access is not blocked (\(report.detail))")
print("")
print("""
This applies immediately — no reboot. It is tied to the app's code \
identity, so it survives rebuilds only while the bundle keeps a stable \
Developer ID signature; `permissions status` reports that.
""")
case .blocked:
CLI.error("still blocked after the prompt (\(report.detail))")
CLI.note("""
Either the alert was declined, or macOS already has a decision on file for \
this app — it does not ask twice, and there is no way to reset one. Look in \
System Settings > Privacy & Security > Local Network: if there is a row for \
Gitea macOS Runner, switch it on.
""")
CLI.note("""
Otherwise use the allowlist, which needs no prompt at all: \
gitea-macos-runner permissions grant
""")
throw ExitCode(1)
case .inconclusive:
CLI.error("could not tell: \(report.detail)")
throw ExitCode(1)
}
}
}
@@ -21,7 +21,7 @@ struct ServiceCommand: AsyncParsableCommand {
struct Install: AsyncParsableCommand {
static let configuration = CommandConfiguration(
commandName: "install",
abstract: "Write ~/Library/LaunchAgents/xyz.blakeslee.gitea-macos-runner.plist and load it.",
abstract: "Write ~/Library/LaunchAgents/\(LaunchdService.label).plist and load it.",
discussion: """
Points the agent at the installed, signed .app bundle — not at a bare \
binary. The com.apple.security.virtualization entitlement only survives \
@@ -35,6 +35,16 @@ struct ServiceCommand: AsyncParsableCommand {
@Option(name: .long, help: "Path to the installed executable (default: ~/Applications/GiteaMacosRunner.app/Contents/MacOS/gitea-macos-runner).")
var executable: String?
/// How to configure Local Network access, if it is not already.
///
/// Unset means "decide at run time": ask on a terminal, skip with a
/// pointer otherwise. `none` suppresses the question outright, for a
/// scripted install that has its own arrangements.
@Option(
name: .customLong("grant-local-network"),
help: "Configure macOS Local Network access during install: allowlist, prompt, or none.")
var grantLocalNetwork: LocalNetworkGrantChoice?
func run() async throws {
let executablePath = executable ?? LaunchdService.defaultExecutablePath
@@ -46,14 +56,94 @@ struct ServiceCommand: AsyncParsableCommand {
CLI.note("warning: \(RunnerConfig.expandTilde(options.configPath)) is missing or invalid; the agent will fail to start until it is fixed")
}
// Done before install (which also does it) purely so the operator
// is told: an agent silently vanishing from launchctl is alarming
// if you do not know a rename happened.
for legacy in LaunchdService.removeLegacyAgents() {
CLI.note("removed legacy agent \(legacy) (renamed to \(LaunchdService.label))")
}
try LaunchdService.install(executablePath: executablePath, configPath: configPath)
print("installed \(LaunchdService.agentPlistURL.path)")
print("program: \(RunnerConfig.expandTilde(executablePath)) daemon")
print("logs: \(LaunchdService.logDirectoryURL.path)")
print("")
offerLocalNetworkGrant()
print("check it with: gitea-macos-runner service status")
}
/// Offers to configure Local Network access, if it is not already.
///
/// This is where the question belongs. The agent that was just
/// installed is the process that will be blocked, it has no UI to ask
/// with, and the symptom when it is blocked — every guest boots, no job
/// starts, `No route to host` — points nowhere near the cause. Asking
/// now costs one prompt; not asking costs a debugging session.
///
/// Never fatal: a failed or declined grant leaves a perfectly good
/// installed agent, so this reports and returns rather than throwing.
private func offerLocalNetworkGrant() {
guard grantLocalNetwork != .skip else { return }
guard !LocalNetworkPolicy.status().coversGuestRange else { return }
let method: LocalNetworkPermission.Method
switch grantLocalNetwork {
case .allowlist: method = .allowlist
case .prompt: method = .prompt
case .skip: return // handled above; here for exhaustiveness
case nil:
// Not asked for either way: decide from the terminal. A piped
// or launchd-driven install must not stop on a question, so it
// gets the pointer and carries on.
guard isatty(fileno(stdin)) == 1 else {
CLI.note("""
note: macOS Local Network access is not configured. Until it is, guests \
boot but SSH fails with "No route to host". Configure it with \
`gitea-macos-runner permissions grant`.
""")
print("")
return
}
CLI.note("""
macOS Local Network access is not configured. Without it the agent starts \
guests fine but cannot reach them, and every job fails with "No route to \
host". Granting it writes a subnet allowlist with sudo and needs a reboot.
""")
guard CLI.confirm("configure it now?") else {
CLI.note("skipped; run `gitea-macos-runner permissions grant` later")
print("")
return
}
method = .allowlist
}
// Deliberately swallowed. The agent is installed and correct at
// this point; a declined sudo password should not turn a successful
// install into a failure.
do {
try LocalNetworkGrantFlow.run(method: method)
} catch {
CLI.note("could not configure it: \(error)")
CLI.note("the agent is installed; run `gitea-macos-runner permissions grant` to retry")
}
print("")
}
}
/// `--grant-local-network`'s values: the two grant methods plus an explicit
/// opt-out, which the method enum itself has no business carrying.
///
/// The opt-out case is spelled `skip` rather than `none` so that
/// `choice == .skip` cannot be read as `Optional.none` — the option is
/// itself optional, and "not passed" means something different from
/// "passed `none`".
enum LocalNetworkGrantChoice: String, ExpressibleByArgument, CaseIterable {
case allowlist
case prompt
case skip = "none"
}
/// `service uninstall` — unload and remove the plist.
@@ -68,7 +158,11 @@ struct ServiceCommand: AsyncParsableCommand {
func run() async throws {
let path = LaunchdService.agentPlistURL.path
let existed = FileManager.default.fileExists(atPath: path)
let legacy = LaunchdService.removeLegacyAgents()
try LaunchdService.uninstall()
for label in legacy {
print("removed legacy agent \(label)")
}
print(existed ? "removed \(path)" : "not installed (\(path))")
}
}
+1
View File
@@ -45,6 +45,7 @@ struct GiteaMacOSRunner: AsyncParsableCommand {
ServiceCommand.self,
DoctorCommand.self,
ConfigCommand.self,
PermissionsCommand.self,
],
defaultSubcommand: nil
)
+1 -1
View File
@@ -64,7 +64,7 @@ struct ConfigTests {
#expect(c.scheduler.pollIntervalSeconds == 5)
#expect(c.scheduler.reconcileIntervalSeconds == 300)
#expect(c.scheduler.jobTimeoutMinutes == 120)
#expect(c.scheduler.bootTimeoutSeconds == 300)
#expect(c.scheduler.bootTimeoutSeconds == 900)
#expect(c.guest.username == "admin")
#expect(c.guest.cpuCount == 4)
#expect(c.guest.memoryGB == 8)
@@ -0,0 +1,211 @@
import Foundation
import Testing
@testable import RunnerCore
/// Tests for ``LocalNetworkPolicy`` — the arithmetic behind the Local Network
/// subnet allowlist.
///
/// The bug these exist for: an allowlist entry that covers *today's* guest
/// subnet but not tomorrow's. vmnet picks its NAT subnet at runtime and steps
/// to the next free /24 when one is taken, so `192.168.64.0/24` works right up
/// until the day a second VM host appears on the machine — and then every guest
/// connection fails with "No route to host" with the allowlist still looking
/// perfectly configured.
@Suite("LocalNetworkPolicy")
struct LocalNetworkPolicyTests {
// MARK: - Coverage
@Test("an entry must cover the whole vmnet span, not just its first /24")
func coverageIsAllOrNothing() {
// The exact span, and anything wider.
#expect(LocalNetworkPolicy.coversVMNetRange("192.168.64.0/18"))
#expect(LocalNetworkPolicy.coversVMNetRange("192.168.0.0/16"))
#expect(LocalNetworkPolicy.coversVMNetRange("192.168.0.0/8"))
#expect(LocalNetworkPolicy.coversVMNetRange("0.0.0.0/0"))
// The trap: contains 192.168.64.x, but not 192.168.65.x.
#expect(!LocalNetworkPolicy.coversVMNetRange("192.168.64.0/24"))
#expect(!LocalNetworkPolicy.coversVMNetRange("192.168.64.0/19"))
// Adjacent but disjoint.
#expect(!LocalNetworkPolicy.coversVMNetRange("192.168.128.0/18"))
#expect(!LocalNetworkPolicy.coversVMNetRange("10.0.0.0/8"))
}
@Test("the RFC 1918 default covers the guest range")
func defaultSubnetsCoverGuests() {
#expect(LocalNetworkPolicy.defaultSubnets.contains(where: LocalNetworkPolicy.coversVMNetRange))
}
@Test("a prefix is required for coverage; a bare address is a /32")
func bareAddressIsASingleHost() {
#expect(!LocalNetworkPolicy.coversVMNetRange("192.168.64.1"))
#expect(!LocalNetworkPolicy.coversVMNetRange("192.168.64.1/32"))
}
@Test("host bits below the prefix do not change the block")
func hostBitsAreMaskedOff() {
// 192.168.70.5/18 and 192.168.64.0/18 are the same block.
#expect(LocalNetworkPolicy.coversVMNetRange("192.168.70.5/18"))
}
// MARK: - Rejecting what macOS would silently ignore
@Test("malformed, IPv6 and hostname entries are rejected, not crashed on")
func garbageIsRejected() {
for entry in [
"", "/", "/24", "192.168.64.0/", "192.168.64.0/33", "192.168.64.0/-1",
"192.168.64", "192.168.64.0.1", "192.168.256.0/18", "192.168.64.x/18",
"fd00::/8", "::/0", "localhost", "example.com/24", "192.168.64.0/18/24",
] {
#expect(!LocalNetworkPolicy.isValidSubnet(entry), "\(entry) should be rejected")
#expect(!LocalNetworkPolicy.coversVMNetRange(entry), "\(entry) should not cover")
}
}
@Test("well-formed entries validate")
func goodEntriesValidate() {
for entry in ["0.0.0.0/0", "10.0.0.0/8", "192.168.64.0/24", "192.168.64.1", "255.255.255.255/32"] {
#expect(LocalNetworkPolicy.isValidSubnet(entry), "\(entry) should validate")
}
}
@Test("ipv4Value packs octets most-significant first")
func addressPacking() {
#expect(LocalNetworkPolicy.ipv4Value("0.0.0.0") == 0)
#expect(LocalNetworkPolicy.ipv4Value("192.168.64.0") == 0xC0A8_4000)
#expect(LocalNetworkPolicy.ipv4Value("192.168.127.255") == 0xC0A8_7FFF)
#expect(LocalNetworkPolicy.ipv4Value("255.255.255.255") == 0xFFFF_FFFF)
#expect(LocalNetworkPolicy.ipv4Value("192.168.64") == nil)
#expect(LocalNetworkPolicy.ipv4Value("192.168.64.256") == nil)
}
// MARK: - What gets written
@Test("both interface keys are written, each with the subnets as separate arguments")
func writeArgumentsCoverBothKeys() {
let subnets = ["10.0.0.0/8", "192.168.0.0/16"]
let commands = LocalNetworkPolicy.writeArguments(subnets: subnets)
#expect(commands.count == 2)
#expect(commands[0] == ["write", LocalNetworkPolicy.domain, LocalNetworkPolicy.ethernetKey,
"-array", "10.0.0.0/8", "192.168.0.0/16"])
#expect(commands[1] == ["write", LocalNetworkPolicy.domain, LocalNetworkPolicy.wifiKey,
"-array", "10.0.0.0/8", "192.168.0.0/16"])
// Each subnet is its own argv element. Joined into one string, macOS
// would read the whole thing as a single unparseable entry and grant
// nothing — while `defaults read` still showed something plausible.
for command in commands {
#expect(!command.contains { $0.contains(" ") })
}
}
@Test("the shell rendering quotes anything a shell would reinterpret")
func shellRenderingIsSafe() {
let lines = LocalNetworkPolicy.writeCommandLines(subnets: ["10.0.0.0/8", "a b; rm -rf /"])
#expect(lines.count == 2)
for line in lines {
#expect(line.hasPrefix("sudo defaults write \(LocalNetworkPolicy.domain) "))
// Plain CIDR stays readable; the hostile entry gets quoted.
#expect(line.contains(" 10.0.0.0/8 "))
#expect(line.contains("'a b; rm -rf /'"))
}
}
// MARK: - Reading the host back
@Test("status reads the live host without throwing and stays self-consistent")
func statusIsSelfConsistent() {
// Cannot assert the host's actual configuration — this suite runs on
// developer machines and in CI guests alike. What must hold either way
// is that the derived flags agree with the entries.
let status = LocalNetworkPolicy.status()
#expect(status.isConfigured == !status.allowlist.isEmpty)
#expect(status.coversGuestRange == status.allowlist.contains(where: LocalNetworkPolicy.coversVMNetRange))
if status.allowlist.isEmpty { #expect(status.sourcePaths.isEmpty) }
#expect(Set(status.allowlist).count == status.allowlist.count, "entries should be deduplicated")
}
@Test("all three candidate preference paths are checked")
func candidatePathsCoverBothSudoOutcomes() {
let candidates = LocalNetworkPolicy.preferenceCandidates()
// `sudo defaults write` lands in root's preferences or the invoking
// user's depending on whether sudo preserved HOME, so both must be
// checked — plus the system-wide location.
#expect(candidates.contains("/var/root/Library/Preferences/\(LocalNetworkPolicy.domain).plist"))
#expect(candidates.contains("/Library/Preferences/\(LocalNetworkPolicy.domain).plist"))
#expect(candidates.contains(NSHomeDirectory() + "/Library/Preferences/\(LocalNetworkPolicy.domain).plist"))
}
// MARK: - Parsing preferences
/// A preferences file carrying `entries` under both allowlist keys.
private func preferences(_ entries: [String]) throws -> Data {
try PropertyListSerialization.data(
fromPropertyList: [
LocalNetworkPolicy.ethernetKey: entries,
LocalNetworkPolicy.wifiKey: entries,
"UnrelatedKey": "ignored",
],
format: .xml,
options: 0)
}
@Test("both keys are read, and the same entry in both is not counted twice")
func entriesAreUnionedAcrossKeys() throws {
let data = try preferences(["10.0.0.0/8", "192.168.0.0/16"])
#expect(LocalNetworkPolicy.entries(inPreferences: data) == ["10.0.0.0/8", "192.168.0.0/16"])
}
@Test("data that is not a preferences file reads as empty rather than throwing")
func unparseablePreferencesReadEmpty() {
#expect(LocalNetworkPolicy.entries(inPreferences: Data("not a plist".utf8)).isEmpty)
#expect(LocalNetworkPolicy.entries(inPreferences: Data()).isEmpty)
}
@Test("only files that contribute an entry are named as sources")
func sourcePathsNameOnlyContributingFiles() throws {
let empty = try preferences([])
let real = try preferences(["192.168.0.0/16"])
// The same entries again: a second copy of a value already seen adds
// nothing, so its path must not be reported as a source.
let duplicate = try preferences(["192.168.0.0/16"])
let status = LocalNetworkPolicy.status(fromContentsOf: [
("/first.plist", empty),
("/second.plist", real),
("/third.plist", duplicate),
])
#expect(status.allowlist == ["192.168.0.0/16"])
#expect(status.sourcePaths == ["/second.plist"])
#expect(status.coversGuestRange)
}
@Test("an unreadable candidate makes the answer unknown, not unconfigured")
func unreadableCandidatesAreIndeterminate() throws {
// The real case: `sudo defaults write` lands in /var/root, which is
// mode 700, so an ordinary user is refused before it can learn whether
// the file is even there. Reporting that as "no allowlist" is how a
// successful grant gets called a failure.
let blind = LocalNetworkPolicy.status(
fromContentsOf: [], unreadablePaths: ["/var/root/Library/Preferences/x.plist"])
#expect(!blind.isConfigured)
#expect(blind.isIndeterminate)
// Nothing found and nothing refused really is unconfigured.
let empty = LocalNetworkPolicy.status(fromContentsOf: [])
#expect(!empty.isConfigured)
#expect(!empty.isIndeterminate)
// Something found outweighs a refusal elsewhere: the answer is known.
let found = LocalNetworkPolicy.status(
fromContentsOf: [("/a.plist", try preferences(["10.0.0.0/8"]))],
unreadablePaths: ["/var/root/Library/Preferences/x.plist"])
#expect(found.isConfigured)
#expect(!found.isIndeterminate)
}
}
+85 -9
View File
@@ -21,8 +21,9 @@ Three constraints shape everything below:
`VZError.virtualMachineLimitExceeded` from `start()`. Concurrency is therefore
2, permanently, and the config value is clamped rather than trusted.
2. **Virtualization needs a GUI session and a signed bundle.** The daemon runs as
a LaunchAgent in a logged-in user session, from inside an ad-hoc-signed `.app`
carrying `com.apple.security.virtualization`.
a LaunchAgent in a logged-in user session, from inside a signed `.app` carrying
`com.apple.security.virtualization` — Developer ID when a certificate is
available, ad-hoc otherwise (see "Verified facts", item 10).
3. **Gitea decides which job a runner claims, not us.** We supply capacity; the
server matches. Trying to pin a specific job to a specific VM would mean
reimplementing Gitea's matching rules, and would be wrong the moment they
@@ -45,7 +46,7 @@ Three constraints shape everything below:
┌──────────────────────────────── Host (Apple Silicon Mac, macOS 26+) ─────────────────────────────┐
│ │
│ LaunchAgent (user session, auto-login, login.keychain unlocked) │
│ └── GiteaMacosRunner.app (ad-hoc signed, com.apple.security.virtualization, LSUIElement) │
│ └── GiteaMacosRunner.app (signed, com.apple.security.virtualization, LSUIElement) │
│ │ │
│ │ NSApplication(.prohibited).run() ── main thread, required by Virtualization │
│ │ │
@@ -284,8 +285,18 @@ scheduler treats as transient back-pressure rather than a failure.
### Timeouts
* A slot in `.provisioning(since:)` longer than `bootTimeoutSeconds` (default
300) is torn down. Covers a guest that never gets a lease, never starts `sshd`,
or hangs in Setup Assistant.
900) is torn down. Covers a guest that never gets a lease, never starts `sshd`,
or hangs in Setup Assistant. The default is deliberately generous: several
Virtualization guests sharing one host push a boot from tens of seconds into
minutes, and a limit below the worst case does not time out a bad boot, it
livelocks — each replacement clone starts from zero and adds load, so the next
boot is slower still and no runner ever registers.
* The lifecycle's own `waitForLease` and `waitForSSH` budgets are derived from
what is *left* of that deadline, not from a fresh copy of it. Given the full
`bootTimeoutSeconds` their deadlines would fall after the planner's, so the
planner would always cancel first and the specific error — which host, how
many attempts, what the last one said — would be discarded in favour of a bare
cancellation.
* A slot in `.running(jobHint:since:)` longer than `jobTimeoutMinutes` (default
120) is torn down. Covers a job that hangs. This is comfortably below Gitea's
own `ABANDONED_JOB_TIMEOUT` (24 h), so our teardown always happens first and
@@ -372,7 +383,8 @@ disposable and isolated, not on the job being constrained inside it.
and nothing else.
* Networking is **NAT**, not bridged. Guests can reach the LAN and Gitea, but are
not first-class hosts on it. Bridged networking would require the restricted
`com.apple.vm.networking` entitlement, which ad-hoc signing cannot grant — a
`com.apple.vm.networking` entitlement, which needs an Apple-approved
provisioning profile and which ad-hoc signing cannot grant at all — a
constraint that happens to align with what we want anyway.
* **SSH host keys are not verified.** The peer is a VM this process booted
moments ago on a link no other machine shares; pinning would break on every
@@ -517,9 +529,10 @@ timeout. The `--manual-setup` fallback is out of v1 scope (§4).
**10. Headless Virtualization requires an `NSApplication` run loop with
`.prohibited` activation policy, inside a signed `.app` bundle** carrying
`com.apple.security.virtualization`. Ad-hoc signing (`codesign -s -`) suffices.
Bridged networking would additionally need a restricted entitlement; NAT does
not.
`com.apple.security.virtualization`. That entitlement is unrestricted: ad-hoc
signing (`codesign -s -`) grants it, and a Developer ID certificate grants it
with no provisioning profile. Bridged networking would additionally need a
restricted entitlement; NAT does not.
→ *Consequence:* `CommandDaemon` starts `NSApplication` and runs the orchestrator
in a detached `Task`. This applies to **every** command that starts a VM, not
just the daemon: `vm boot`, `image build`, and `image provision` all go through
@@ -540,6 +553,24 @@ downgrade that only surfaces as a failed VM start. And `bundle` must copy
`.app` without them is a working binary with a broken `image build`,
`service install`, and `config init`.
**10a. Which signature is used decides whether the app's code identity is stable
across rebuilds.** A Developer ID signature's designated requirement is anchored
to the team (`… and certificate leaf[subject.OU] = <TEAM_ID>`), so every build
is the same program to macOS. An ad-hoc signature has no anchor, so identity
falls back to the main executable's Mach-O UUID, which the linker regenerates on
essentially every link.
→ *Consequence:* this is not cosmetic, because macOS Local Network privacy is
keyed on exactly that UUID (Fact 16). Under ad-hoc signing a grant is
withdrawn by the next `make install`; under Developer ID it persists. `make sign`
therefore selects a `Developer ID Application` identity matching `TEAM_ID` when
the keychain has one and falls back to ad-hoc with a warning when it does not —
the fallback is required because CI builds inside a throwaway guest with neither
keychain nor certificate. The Developer ID path also passes `--options runtime
--timestamp`, so the bundle is notarizable later without re-signing; notarization
itself is skipped, since it governs distribution to other Macs and this app is
built and installed in place. `Doctor.checkCodeSignature` reports which path was
taken and warns on ad-hoc.
**11. macOS 15+ requires an unlocked `login.keychain` to start a VM.**
→ *Consequence:* the service **must** be a LaunchAgent in the auto-logged-in
user's session, never a LaunchDaemon (which has no session and no unlocked
@@ -579,3 +610,48 @@ changing the MAC address or ECID.**
prohibition collides directly with the per-slot MAC scheme from Fact 12, so
adopting it would require per-slot saved states and a careful look at DHCP lease
reuse — not a drop-in optimization.
**16. macOS 15+ Local Network privacy can block host→guest connections, and
granting it interactively takes deliberate work.** Per
[TN3179](https://developer.apple.com/documentation/technotes/tn3179-understanding-local-network-privacy)
it is not TCC — the check is a Network Extension packet filter, so it is absent
from `TCC.db`, cannot be queried, cannot be reset, and it "uses your main
executable UUID as part of its implementation". A denial returns `EHOSTUNREACH`
(errno 65), indistinguishable from a genuinely unreachable host. Three things
then conspire against the interactive grant: a LaunchAgent has no UI to show the
prompt in; a run started from a shell is attributed to the **responsible
process**, so the prompt and the System Settings row belong to Terminal rather
than to this app, and granting it to Terminal does not carry to the agent; and
under ad-hoc signing the UUID keying (Fact 10a) withdraws the grant on the next
rebuild.
→ *Consequence:* the deterministic fix is the subnet allowlist
(`com.apple.network.local-network`, keys `AllowedEthernetLocalNetworkAddresses`
and `AllowedWiFiLocalNetworkAddresses`), which is keyed on the network rather
than the app and is read at boot — so it needs a reboot, not a service restart.
`LocalNetworkPolicy` owns the arithmetic and requires coverage of
`192.168.64.0/18`, not a single /24, because the NAT subnet is chosen at runtime
and slides to the next free /24; `Doctor.localNetworkNote` and
`SSHExec.localNetworkHint` both report against it, since errno 65 gives the
operator nothing to go on by itself.
→ *Consequence:* both routes are commands rather than documentation.
`permissions grant` writes the allowlist and verifies it read back (`sudo
defaults write` lands in root's or the invoking user's preferences depending on
whether sudo preserved `HOME`, so where it went is not assumable).
`permissions grant --method prompt` addresses the attribution problem head-on:
launching the installed bundle through LaunchServices (`open -n -b …`) makes the
app its **own** responsible process, so the prompt and the Settings row belong to
it rather than to Terminal — and because the LaunchAgent runs the same signed
identity, the grant carries. That only became worth building once the bundle was
Developer ID signed; under ad-hoc signing the UUID churn withdraws it on the next
rebuild, which is why `permissions status` reports code identity alongside the
allowlist.
→ *Consequence:* the prompt route is best-effort and says so. Observed on a host
where the decision was already recorded: `UserEventAgent` resolves the flow to
the bundle ID on every attempt — so the attribution works — but presents no
alert, because macOS asks once per app identity and then answers from that
record, silently, forever. There is no supported reset. So `--method prompt`
verifies by *probing* rather than by trusting the launch, and on a denial says
plainly that it did not take and points at the allowlist, which is not subject
to the per-app check at all. The allowlist stays the recommendation.
+127 -39
View File
@@ -162,8 +162,9 @@ SSH server, so the build appears to hang. See
Virtualization.framework refuses to run unless the calling binary carries the
`com.apple.security.virtualization` entitlement, and entitlements are only honoured on a signed
binary inside a proper `.app` bundle. Ad-hoc signing (`codesign -s -`) satisfies this — **no paid
Apple developer account is needed.**
binary inside a proper `.app` bundle. That entitlement is not restricted, so ad-hoc signing
(`codesign -s -`) satisfies it — **the runner works with no Apple developer account.** A Developer
ID certificate buys something different and worth having; see [Code signing](#code-signing) below.
```sh
git clone <this repo> && cd gitea-macos-runner
@@ -176,7 +177,7 @@ make install
| --- | --- |
| `make build` | `swift build -c release --arch arm64` |
| `make bundle` | Assemble `GiteaMacosRunner.app` around the binary: `Contents/MacOS/gitea-macos-runner`, `Contents/Info.plist`, and `Contents/Resources/` (`provision.sh`, `launchd.plist.template`, `config.example.json`) |
| `make sign` | `codesign --sign - --entitlements …` (ad-hoc) and print the resulting entitlements |
| `make sign` | Sign the bundle with the virtualization entitlement — Developer ID when a matching certificate is in the keychain, ad-hoc otherwise — then print the entitlements and the resulting identity |
| `make all` | `build` + `bundle` + `sign`. The default target. |
| `make install` | Runs `all`, copies the app to `~/Applications`, and symlinks the CLI to `/usr/local/bin/gitea-macos-runner` |
| `make dev` | Debug build + bundle + sign, for fast iteration. Does **not** install. |
@@ -192,6 +193,58 @@ code looks in `Contents/Resources` first and only then falls back to
repo-relative paths, so an installed `.app` missing them is a working binary
with three broken commands.
### Code signing
`make sign` picks its identity automatically:
| Keychain state | What you get |
| --- | --- |
| A `Developer ID Application` certificate whose team matches `TEAM_ID` | Developer ID signature, hardened runtime (`--options runtime`), trusted timestamp (`--timestamp`) |
| No matching certificate | Ad-hoc signature (`codesign --sign -`), with a warning |
Both produce a bundle that boots VMs — the virtualization entitlement is not restricted, and needs
no provisioning profile on either path. What differs is **code identity stability**, and that is the
whole reason to prefer Developer ID:
- A Developer ID signature carries a designated requirement anchored to your team
(`… and certificate leaf[subject.OU] = L7UDTQ6F5W`). Every subsequent build satisfies it, so macOS
recognises rebuild after rebuild as *the same program*.
- An ad-hoc signature has no anchor, so the system falls back to the main executable's Mach-O UUID —
which the linker regenerates on essentially every link. Each `make install` presents a program
macOS has never seen before.
The practical consequence is Local Network privacy (§2.6): per
[TN3179](https://developer.apple.com/documentation/technotes/tn3179-understanding-local-network-privacy)
the grant "uses your main executable UUID as part of its implementation", so under ad-hoc signing it
is silently withdrawn by the next rebuild. Under Developer ID it survives.
The team is baked into the `Makefile` as a default; override it for your own certificate:
```sh
make install TEAM_ID=ABCDE12345 # your Developer ID team
make install TEAM_ID= # force ad-hoc even if a certificate exists
```
Confirm what actually landed — `doctor`'s `code identity` check reports it, or ask `codesign`:
```sh
codesign -dvv ~/Applications/GiteaMacosRunner.app
# Identifier=xyz.blakeslee.gitea-macos-vm-orchestrator
# CodeDirectory v=20500 … flags=0x10000(runtime)
# Authority=Developer ID Application: Your Name (ABCDE12345)
# TeamIdentifier=ABCDE12345
```
`TeamIdentifier=not set` and `flags=0x2(adhoc)` mean the ad-hoc path was taken.
Two things this deliberately does **not** do. The bundle is not **notarized**: notarization matters
for software distributed to other Macs, where Gatekeeper checks the quarantine bit, and this app is
built and installed in place. `spctl -a` therefore reports `rejected: Unnotarized Developer ID`,
which is expected and does not stop anything here. Signing does require network access for
`--timestamp`, so an offline host falls back to ad-hoc. And the entitlements list stays minimal:
`com.apple.vm.networking` — needed only for bridged networking, and genuinely restricted — is not
requested. See [DESIGN.md](DESIGN.md).
If `/usr/local/bin` isn't writable, `make install` says so and prints the `sudo ln -sf …` command to
run yourself.
@@ -277,7 +330,7 @@ is required; the file form wins over the inline form when both are present.
| `pollIntervalSeconds` | `5` | How often to poll the queued-jobs API. |
| `reconcileIntervalSeconds` | `300` | How often to sweep Gitea for orphaned runner registrations from uncleanly-killed VMs. |
| `jobTimeoutMinutes` | `120` | Wall-clock limit for one job; the VM is destroyed when exceeded. |
| `bootTimeoutSeconds` | `300` | Time allowed from VM start to a usable SSH connection. |
| `bootTimeoutSeconds` | `900` | Time allowed from VM start to a usable SSH connection. |
**`guest`**
@@ -388,6 +441,11 @@ disabling sleep (`sudo pmset -a sleep 0 disablesleep 1`) so the Mac comes back i
after a power event without a human present. This does mean the disk is effectively unlocked at
boot — appropriate for a dedicated CI machine, not for a shared workstation.
On a terminal, `service install` also asks whether to configure Local Network access (§2.6) when it
is not already, defaulting to no. It never blocks: a scripted install with no terminal prints a
pointer and carries on. `--grant-local-network allowlist|prompt|none` decides it up front instead of
being asked.
`service uninstall` removes the LaunchAgent; it does not delete images or config.
### 2.6 macOS 15+ Local Network privacy prompt
@@ -396,49 +454,77 @@ Starting with macOS 15, a process that contacts other hosts on the local network
one-time Local Network permission prompt. A LaunchAgent that is denied (or that never gets a human
to click Allow) cannot reach the guest's NAT address, so VMs boot but SSH never connects.
**On a CI host, use the subnet allowlist.** It is the only deterministic option — no prompt, no GUI
session, and nothing to redo after a rebuild:
`service install` offers to configure this, and it can also be done at any time:
```sh
sudo defaults write com.apple.network.local-network \
AllowedEthernetLocalNetworkAddresses -array "192.168.64.0/18"
sudo defaults write com.apple.network.local-network \
AllowedWiFiLocalNetworkAddresses -array "192.168.64.0/18"
gitea-macos-runner permissions status # what is configured, and what to do about it
gitea-macos-runner permissions grant # configure it
```
Then **reboot** — these are read at boot, so restarting the service alone is not enough.
One asymmetry to know about before reading any of this output: the allowlist is written as root and
lands in `/var/root/Library/Preferences/`, which is mode 700. An ordinary login cannot read it back —
so `permissions status` and `doctor` report `no allowlist visible`, which means *not visible*, not
*not set*. `sudo gitea-macos-runner permissions status` answers definitively. `permissions grant`
does not have this problem: it re-reads the file with the sudo credentials it just used, so it
confirms its own write.
**Do not pin this to a single /24.** Virtualization.framework's NAT starts at `192.168.64.0/24` but
picks the subnet at runtime and steps to the next free one when that range is already in use, so the
same host can hand out `192.168.65.x` tomorrow. An allowlist naming only `192.168.64.0/24` then
looks configured while silently blocking every guest — the failure surfaces as `No route to host`
(errno 65) on the SSH connection, not as a permission error. The `/18` above spans
`192.168.64.0`–`192.168.127.255`, which covers the drift; if you would rather not think about
ranges at all, the RFC 1918 set `"10.0.0.0/8" "172.16.0.0/12" "192.168.0.0/16"` also works. `doctor`
reports `local network access` as a **pass** once it sees an allowlist that covers that span, and as
a **warning** when an allowlist exists but does not. Both keys are documented by Apple in
[TN3179](https://developer.apple.com/documentation/technotes/tn3179-understanding-local-network-privacy)
and are the workaround [Tart's FAQ](https://tart.run/faq/) recommends for the same problem.
`grant` has two methods. Both are one command; neither needs anything pasted.
**Approving interactively instead.** The app cannot be pre-approved: it appears under **System
Settings → Privacy & Security → Local Network** only *after* it has actually attempted a connection
to a guest. An empty list is expected on a fresh install and does not mean anything is broken. To
create the entry and answer the prompt, run one boot by hand from a Terminal in the GUI session:
**`--method allowlist` (the default) is what a CI host wants.** It writes a subnet allowlist — the
one deterministic option: no prompt, no GUI session, and nothing to redo after a rebuild. It asks
for your sudo password, reports which preferences file the write actually landed in, and then offers
to **reboot**, which is required: these values are read at boot, so restarting the service alone is
not enough. Pass `--no-reboot` to defer that.
```sh
gitea-macos-runner vm boot --image default
```
The default grant is all of RFC 1918 — `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16` — the same
set [Tart](https://tart.run/faq/) and orchard use. Narrow it with `--subnet`, repeatable, but **do
not pin it to a single /24**: Virtualization.framework's NAT starts at `192.168.64.0/24` but picks
the subnet at runtime and steps to the next free one when that range is already in use, so the same
host can hand out `192.168.65.x` tomorrow. An allowlist naming only `192.168.64.0/24` then looks
configured while silently blocking every guest — and the failure surfaces as `No route to host`
(errno 65) on the SSH connection, not as a permission error. `192.168.64.0/18` spans
`192.168.64.0`–`192.168.127.255`, which is the narrowest entry that covers the drift. `doctor`
reports `local network access` as a **pass** once it sees an allowlist covering that span, and as a
**warning** when an allowlist exists but does not — but only when it can see it at all, which means
running under `sudo` or straight after a grant. Both keys are documented by Apple in
[TN3179](https://developer.apple.com/documentation/technotes/tn3179-understanding-local-network-privacy).
and click **Allow**. Do not wait for the LaunchAgent to hit it — a background agent has no way to
answer the prompt.
**`--method prompt` takes effect immediately, with no reboot**, and is the better choice on a Mac
you are sitting in front of. It launches the installed `.app` through LaunchServices — which is what
makes the app its own responsible process — provokes the real system alert, and reports whether the
grant took. It needs `make install` to have run, a GUI session to show the alert in, and a Developer
ID signature for the grant to survive the next rebuild; `permissions status` reports that last one.
> **Caveat with ad-hoc signing.** An interactive grant is not durable for this project's ad-hoc
> signed bundle. Local Network privacy does not use TCC; per TN3179 it "uses your main executable
> UUID as part of its implementation", and the linker mints a fresh `LC_UUID` on essentially every
> rebuild. So `make install` after a code change is liable to present as a new app that must be
> approved again — and macOS offers no way to reset a Local Network decision back to undetermined,
> so the stale entries accumulate. This is why the allowlist above, which is keyed on the subnet
> rather than on the app, is the recommendation for an unattended machine.
**Why the prompt needs that much machinery.** Left to itself, on this host there is usually nothing
able to show it, and when something does, it is attributed to the wrong program.
An app appears under **System Settings → Privacy & Security → Local Network** only *after* it has
attempted a connection to a guest, so an empty list on a fresh install is expected and means
nothing is broken. It cannot be pre-approved. But the two obvious ways to trigger the prompt both
miss:
- **From the LaunchAgent.** A background agent has no UI, so the prompt has nowhere to appear. The
connection is simply denied, and it surfaces as `No route to host` (errno 65) — not as a
permission error.
- **By hand from a Terminal**, e.g. `gitea-macos-runner vm boot --image default`. macOS assigns the
privacy decision to the *responsible process*, and a binary exec'd from a shell is Terminal's
responsibility, not its own. So both the prompt and the Settings row belong to **Terminal**, and
approving it there does not carry over to the LaunchAgent. (If you are hunting for a row that
seems missing, look for Terminal rather than for "Gitea macOS Runner".)
`permissions grant --method prompt` exists to thread that needle: it starts the app through
LaunchServices rather than from the shell, so the app is its own responsible process and the
decision is recorded against *its* identity — the same identity the LaunchAgent runs under.
> **Under an ad-hoc signature it is not durable anyway.** Local Network privacy does not use TCC;
> per TN3179 it "uses your main executable UUID as part of its implementation", and the linker mints
> a fresh `LC_UUID` on essentially every rebuild. So `make install` after a code change presents as
> a new app that must be approved again — and macOS offers no way to reset a Local Network decision
> back to undetermined, so stale entries accumulate. A Developer ID signature fixes the churn, since
> the identity is then anchored to the certificate rather than to the binary (see
> [Code signing](#code-signing)) — that is what makes `--method prompt` worth using at all. The
> subnet allowlist, keyed on the network rather than on the app, sidesteps the whole mechanism and
> remains the recommendation for an unattended machine.
---
@@ -463,6 +549,7 @@ The checks, in order:
| `host capability` | Apple Silicon, and host macOS ≥ 26 |
| `Virtualization.framework` | `VZVirtualMachine.isSupported` |
| `virtualization entitlement` | `com.apple.security.virtualization` on the *running* executable — this is the check that catches running from `.build/` instead of the signed `.app` |
| `code identity` | The bundle's signature. Passes naming the identifier, team, and hardened runtime; warns on an ad-hoc signature, because that is what makes Local Network grants evaporate on every rebuild ([Code signing](#code-signing)) |
| `configuration` | The config file loads, parses, and passes validation |
| `free disk space` | Free space on the `storeDir` volume against `storage.minFreeDiskGB` |
| `login.keychain unlocked` | `security show-keychain-info login.keychain` |
@@ -470,7 +557,8 @@ The checks, in order:
| `registration token` | A static token resolves, or one can be fetched when `fetchRegistrationTokenViaAPI` is on |
| `runner download url` | The `gitea-runner` release asset is reachable |
| `token file permissions` | Warns — not fails — when a token file is group- or world-readable |
| `local network access` | Passes when a subnet allowlist covers `192.168.64.0/18`; warns when an allowlist exists but is scoped too narrowly; otherwise an informational note about the macOS 15+ Local Network prompt (§2.6) |
| `guest ssh` | Reachability of the most recent guest lease, when there is one. Warns on a timeout, which is most often a stale 24-hour lease for a guest that is already gone |
| `local network access` | Passes when a subnet allowlist covers `192.168.64.0/18`; warns when an allowlist exists but is scoped too narrowly; otherwise an informational note about the macOS 15+ Local Network prompt. Fix either with `permissions grant` (§2.6) |
If the config file is missing or invalid, the host checks still run and the rest
are skipped — which is exactly the state a first-time operator is in. Resolve
+160 -58
View File
@@ -23,11 +23,12 @@ gitea-macos-runner service status
| --- | --- | --- |
| VM won't start; entitlement / `com.apple.security.virtualization` error | Running an unsigned binary, or one outside the signed `.app` bundle | `make sign` (or re-run `make install`); invoke the installed bundle, never `.build/release/…` |
| `virtualMachineLimitExceeded` at boot | macOS allows at most **2** concurrent macOS VMs | Set `scheduler.maxConcurrentVMs` ≤ 2; kill stray VMs from earlier runs |
| VM boots but never gets an IP | DHCP lease not yet written, or Local Network privacy denial (macOS 15+) | Check `/var/db/dhcpd_leases`; grant Local Network permission or pre-authorize the subnet |
| Runner not listed under Privacy & Security → Local Network | Expected — the list is populated only after the app first attempts a local connection; it cannot be pre-approved | Boot one VM by hand from a GUI Terminal to create the entry, or (better on CI) allowlist the subnet with `defaults write com.apple.network.local-network` |
| VM boots but never gets an IP | DHCP lease not yet written, or Local Network privacy denial (macOS 15+) | Check `/var/db/dhcpd_leases`; `gitea-macos-runner permissions grant` |
| Runner not listed under Privacy & Security → Local Network | Expected — the list is populated only after the app first attempts a local connection, and a run started from a shell is attributed to Terminal, not to the app | `gitea-macos-runner permissions grant` (allowlist, then reboot), or `--method prompt` to raise the alert as the app rather than as Terminal |
| SSH times out on a freshly built image | Guest macOS < 27, so provisioning options were ignored and Setup Assistant is waiting | Rebuild the image from a macOS **27+** IPSW |
| `ssh failed: cannot connect … No route to host) (errno: 65)` part-way through provisioning | macOS 15+ Local Network privacy blocking the app — the grant is keyed on the executable's UUID, so `make install` withdraws it | Allowlist the subnet (`192.168.64.0/18`) and **reboot**; see [SSH fails with "No route to host" mid-run](#ssh-fails-with-no-route-to-host-errno-65-mid-run) |
| Allowlist is set but guests are still unreachable | It names `192.168.64.0/24` while the NAT has moved to `192.168.65.x` | Widen it to `192.168.64.0/18` and reboot; `doctor` now warns about too-narrow allowlists |
| VMs boot in a loop; every teardown says `reason=cancelled` and nothing is logged between the lease and the teardown | `scheduler.bootTimeoutSeconds` is below the guest's *worst-case* boot on a contended host, so each clone is killed while still starting — and each replacement makes the next one slower | Raise `scheduler.bootTimeoutSeconds` (default 900) and reduce the number of concurrent guests; see [The daemon boots VMs forever](#the-daemon-boots-vms-forever-and-every-teardown-says-reasoncancelled) |
| `ssh failed: cannot connect … No route to host) (errno: 65)` part-way through provisioning | macOS 15+ Local Network privacy blocking the app — the grant is keyed on the executable's UUID, so `make install` withdraws it | `gitea-macos-runner permissions grant`, then **reboot**; see [SSH fails with "No route to host" mid-run](#ssh-fails-with-no-route-to-host-errno-65-mid-run) |
| Allowlist is set but guests are still unreachable | It names `192.168.64.0/24` while the NAT has moved to `192.168.65.x` | `gitea-macos-runner permissions grant` (defaults to all of RFC 1918) and reboot; `doctor` warns about too-narrow allowlists |
| `SecKeyCreateRandomKey` / "Interaction is not allowed" | `login.keychain` is locked — no GUI session | Run as a LaunchAgent in an unlocked GUI session; enable auto-login |
| Job stays queued forever | Label mismatch, or the daemon isn't running/reaching Gitea | Use bare label names in `runs-on`; match `runner.labels`; check daemon logs |
| `actions/checkout` fails instantly | Node.js missing from the guest image | `gitea-macos-runner image provision <name>` |
@@ -62,8 +63,9 @@ codesign -d --entitlements - ~/Applications/GiteaMacosRunner.app # verify
The output must list `com.apple.security.virtualization`. Then confirm the command you're running
resolves to the installed bundle's binary — `which -a gitea-macos-runner` should point at
`~/Applications/GiteaMacosRunner.app/Contents/MacOS/gitea-macos-runner`, not at
`.build/release/gitea-macos-runner`. Ad-hoc signing is sufficient; you do not need a paid developer
account.
`.build/release/gitea-macos-runner`. Ad-hoc signing is sufficient for *this* error; you do not need
a developer account to start VMs. (A Developer ID certificate solves a different problem — Local
Network grants evaporating on rebuild. See [setup.md](setup.md#code-signing).)
---
@@ -103,25 +105,28 @@ prompt**: a LaunchAgent that was never granted permission (or was denied) cannot
An entry with a recent `lease` timestamp and the guest's MAC means networking is fine and the
problem is timing — raise `scheduler.bootTimeoutSeconds`.
2. No entry at all: pre-authorize the VM subnet, then **reboot** (these are read at boot):
2. No entry at all: check and fix the permission.
```sh
sudo defaults write com.apple.network.local-network \
AllowedEthernetLocalNetworkAddresses -array "192.168.64.0/18"
sudo defaults write com.apple.network.local-network \
AllowedWiFiLocalNetworkAddresses -array "192.168.64.0/18"
sudo gitea-macos-runner permissions status # sudo: the allowlist lives in root's preferences
gitea-macos-runner permissions grant # then reboot when it offers
```
The `/18` is deliberate: the NAT subnet is chosen at runtime and slides to the next free /24
(192.168.65.x, .66.x, …) when one is taken, so a pinned `192.168.64.0/24` breaks the day it
moves. `doctor` reports `local network access` as a pass once it sees an allowlist covering that
span. This is the deterministic fix for an unattended host —
see [Local Network: the app is not listed in System Settings](#local-network-the-app-is-not-listed-in-system-settings)
for why the interactive grant is not.
Without `sudo`, `status` reports `no allowlist visible` on every host — it is written as root into
`/var/root/Library/Preferences/`, which an ordinary login cannot read. That is not evidence the
grant is missing. `grant` itself does not have the problem: it verifies its own write.
3. Or grant it interactively: run `gitea-macos-runner vm boot --image default` from a Terminal in
the GUI session and click **Allow**. The app is not listed under **System Settings → Privacy &
Security → Local Network** until it has made that first attempt.
`grant` pre-authorizes the VM subnets and offers to reboot, which is required — the allowlist is
read at boot. It grants all of RFC 1918 by default; `--subnet` narrows it, but nothing narrower
than `192.168.64.0/18` is safe, because the NAT subnet is chosen at runtime and slides to the next
free /24 (192.168.65.x, .66.x, …) when one is taken, so a pinned `192.168.64.0/24` breaks the day
it moves. `doctor` reports `local network access` as a pass once it sees an allowlist covering
that span.
This is the deterministic fix for an unattended host. On a Mac with someone in front of it,
`permissions grant --method prompt` applies immediately with no reboot —
see [Local Network: the app is not listed in System Settings](#local-network-the-app-is-not-listed-in-system-settings)
for what it does and why granting the prompt by hand does not work.
---
@@ -141,35 +146,61 @@ privacy controls, Local Network privacy is not stored in TCC — per
the checks live "deep in the networking stack" as a Network Extension packet filter, so the
permission is absent from `TCC.db` and `tccutil reset` does not apply to it.
**Fix — interactive.** Make the app connect once, from a GUI session where a human can answer:
**Why you will probably never see the app's own row.** macOS assigns a privacy decision to the
*responsible process*, not necessarily to the binary that opened the socket. Launching the runner
the obvious way —
```sh
gitea-macos-runner vm boot --image default
```
Click **Allow**. The entry now exists and can be toggled later. Do not wait for the LaunchAgent to
trigger it; a background agent cannot answer the prompt, so it simply fails to reach the guest.
— execs the bundle's binary from a shell, so the system holds **Terminal** responsible. Both the
prompt and the Settings row belong to Terminal, and allowing it there does **not** carry over to the
LaunchAgent, which is the process that actually needs it. Meanwhile the LaunchAgent itself has no UI
to show a prompt in, so from it the connection is denied outright and surfaces as `No route to host`
(errno 65) rather than as a permission error.
**Fix — deterministic, and what to use on a CI box.** Allowlist the VM subnet instead. It is keyed
on the network rather than on the app, so no prompt is involved and nothing needs redoing:
**Fix — deterministic, and what to use on any host running the LaunchAgent.** Allowlist the VM
subnets instead. The allowlist is keyed on the network rather than on the app, so no prompt is
involved and nothing needs redoing:
```sh
sudo defaults write com.apple.network.local-network \
AllowedEthernetLocalNetworkAddresses -array "192.168.64.0/18"
sudo defaults write com.apple.network.local-network \
AllowedWiFiLocalNetworkAddresses -array "192.168.64.0/18"
gitea-macos-runner permissions grant
```
Reboot afterwards. `doctor` then reports `local network access` as a **pass**. Both keys are
Apple's, documented in TN3179; the same pair is what [Tart's FAQ](https://tart.run/faq/) recommends
for this exact problem on CI hosts.
That asks for your sudo password, writes both of Apple's keys (documented in TN3179 — the same pair
[Tart's FAQ](https://tart.run/faq/) recommends for this exact problem on CI hosts), reports which
preferences file the write landed in, and offers to reboot. Reboot is required: the values are read
at boot. `doctor` then reports `local network access` as a **pass**.
> **Why the interactive grant does not stick here.** This project ships an **ad-hoc signed** bundle
> (`codesign --sign -`), and TN3179 notes that "local network privacy uses your main executable UUID
> as part of its implementation". The linker writes a new `LC_UUID` on essentially every rebuild, so
> a rebuilt-and-reinstalled runner can read as a *different* program and prompt again — while the
> old row lingers, since macOS provides no way to reset a Local Network decision to undetermined.
> Expect duplicate entries after a few upgrades. The subnet allowlist avoids all of this.
**On a Mac you are sitting at,** `permissions grant --method prompt` is the alternative, and it
needs no reboot. It launches the installed `.app` through LaunchServices instead of exec'ing it from
the shell, which is exactly what makes the app its own responsible process — so the alert, and the
Settings row it creates, belong to the app rather than to Terminal, and the decision applies to the
LaunchAgent. It requires `make install` to have run, a GUI session, and a Developer ID signature to
be durable (see below); `permissions status` reports all three.
**If `--method prompt` reports "still blocked" and you never saw an alert,** macOS most likely
already has a decision on file for the app. It prompts exactly once per app identity and then
answers from that record forever — silently, with `EHOSTUNREACH`, and with no supported way to reset
it back to undetermined. The app *is* being evaluated under its own identity at that point (you can
confirm with `log show --last 2m --predicate 'subsystem == "com.apple.networkextension"'`, which
names the bundle ID on every attempt); the system simply is not asking. Switch the row on in
**System Settings → Privacy & Security → Local Network**, or use the allowlist, which bypasses the
per-app check entirely.
> **And an ad-hoc signed bundle cannot hold the grant anyway.** TN3179 notes that "local network
> privacy uses your main executable UUID as part of its implementation". An ad-hoc signature has no
> team anchor, so that UUID *is* the app's identity — and the linker writes a new `LC_UUID` on
> essentially every rebuild, so a rebuilt-and-reinstalled runner reads as a *different* program and
> prompts again, while the old row lingers (macOS provides no way to reset a Local Network decision
> to undetermined). Expect duplicate entries after a few upgrades.
>
> Signing with a **Developer ID Application** certificate fixes the churn: its designated
> requirement is anchored to your team, so every build is recognised as the same program. `make sign`
> uses one automatically when it is in the keychain — check with `codesign -dvv` or `doctor`'s
> `code identity` line, and see [setup.md](setup.md#code-signing). The subnet allowlist avoids the
> mechanism entirely and is still the right answer for an unattended host.
---
@@ -195,6 +226,63 @@ with this builder.
---
## The daemon boots VMs forever and every teardown says `reason=cancelled`
**Symptom.** A job is queued, the daemon is running, and the log repeats the same three lines with a
new runner name each time — but no runner ever appears in Gitea:
```
info orchestrator: job=1 runner=macos-vm-1cd8e83f… slot=0 booting VM
info orchestrator: ip=192.168.65.233 slot=0 guest leased address
info orchestrator: reason=cancelled slot=0 tearing down slot
```
Note what is missing: nothing between the lease and the teardown, and a teardown reason that names
no cause.
**Cause.** The guest takes longer to reach `sshd` than `scheduler.bootTimeoutSeconds` allows, so the
scheduler tears the slot down while it is still coming up — usually seconds before it would have
succeeded. This is not a timeout that fires once; it is a **livelock**. The replacement clone starts
from zero *and* adds load to an already contended host, so the next boot is slower still and the
loop never converges.
Several Virtualization guests on one Mac is enough to cause it: a guest that reaches SSH in 40
seconds on an idle host can take four or five minutes when it is sharing the machine, and each slot
holds 4 vCPU and 8 GB for the whole attempt. Check with `uptime` inside a guest — a load average in
the tens means the guest is starved, not broken.
**Fix.**
1. Raise `scheduler.bootTimeoutSeconds`. The default is 900; treat it as a ceiling on the guest's
*worst* case, not its typical one. Timing out too early costs far more than noticing a genuinely
wedged guest late.
2. Reduce contention. Count what is actually running:
```sh
ps -Ao pid,rss,etime,comm | grep -i -e virtual -e vmnet
```
Virtualization guests belonging to *other* tools compete for the same cores and the same two-VM
macOS limit. Shut down what you are not using, or lower `scheduler.maxConcurrentVMs`.
3. Confirm the guest itself is fine, independently of the daemon, with `doctor` — its `guest ssh`
check authenticates against whichever slot currently holds a lease:
```sh
gitea-macos-runner doctor
```
**If you are on an older build**, upgrade: the empty gap in that log was three bugs, all now fixed.
`waitForSSH` was handed the full `bootTimeoutSeconds` even though the scheduler's clock had started
before the clone — so the scheduler always fired first and `waitForSSH`'s own error was unreachable;
its per-attempt failures were only ever reported in that unreachable error; and the teardown
overwrote the planner's reason with `cancelled`. Current builds log `waiting for guest ssh` with the
attempt count and the last error while it is happening, and report
`reason="boot timeout: provisioning for 312s (limit 300s)"`.
---
## `SecKeyCreateRandomKey` / "Interaction is not allowed"
**Symptom.** The daemon starts but fails during VM setup with a Security-framework error mentioning
@@ -533,45 +621,59 @@ error: provisioning failed: could not upload provision.sh from …/provision.sh:
ssh failed: cannot connect to 192.168.65.232:22: … No route to host) (errno: 65)
```
**First, check whether it is actually failing.** A single errno 65 at `attempt=1 elapsed=0s`,
seconds after `guest leased address`, is normal and not worth chasing. `waitForSSH` logs its first
probe unconditionally and then heartbeats every 30 s, and that first probe usually lands before the
host has an ARP entry for the guest — which is also `EHOSTUNREACH`. What matters is whether the
message *repeats* at `elapsed=30s`, `60s`, `90s`, … If it does not, boot is proceeding normally. If
it does, read on.
**Cause.** On macOS 15 and newer, an app that has not been granted Local Network access does not get
a "permission denied": the packet filter answers **`EHOSTUNREACH` — errno 65, "No route to host"**,
which is indistinguishable from a guest that is genuinely off the network. Guests here live on a
host-private NAT link that is reachable whenever the VM is up, so on this path errno 65 is far more
often the privacy filter than a routing problem.
host-private NAT link that is reachable whenever the VM is up, so on this path a *persistent* errno
65 is far more often the privacy filter than a routing problem.
Two details make it look intermittent rather than like a permission problem:
- **The grant is keyed on the executable's UUID.** Per
- **Under an ad-hoc signature the grant is keyed on the executable's UUID.** Per
[TN3179](https://developer.apple.com/documentation/technotes/tn3179-understanding-local-network-privacy),
Local Network privacy "uses your main executable UUID as part of its implementation", and the
linker mints a fresh `LC_UUID` on essentially every build. A `make install` after a code change
therefore presents a program macOS has never seen, whose permission is undetermined again — even
though the binary you ran ten minutes ago worked.
though the binary you ran ten minutes ago worked. A Developer ID signature anchors identity to the
team instead and does not drift; `doctor`'s `code identity` check tells you which you have, and
[setup.md](setup.md#code-signing) covers switching.
- **Processes started over SSH are exempt.** Running the same command through `ssh you@host …`
succeeds while running it from a GUI Terminal fails. A remote-shell success proves nothing about
the interactive path.
- **A shell-launched run is attributed to Terminal.** macOS charges the privacy decision to the
responsible process, so the app's own identity is not what is being evaluated when you launch it
by hand — and a grant given to Terminal does nothing for the LaunchAgent.
**Fix.** Allowlist the subnet — it is keyed on the network, not on the app, so no rebuild can
withdraw it and no prompt has to be answered:
**Fix.** Allowlist the subnets — the allowlist is keyed on the network, not on the app, so no
rebuild can withdraw it and no prompt has to be answered:
```sh
sudo defaults write com.apple.network.local-network \
AllowedEthernetLocalNetworkAddresses -array "192.168.64.0/18"
sudo defaults write com.apple.network.local-network \
AllowedWiFiLocalNetworkAddresses -array "192.168.64.0/18"
sudo reboot
gitea-macos-runner permissions grant
```
The values are only read at boot, so **the reboot is not optional** — until it happens, `defaults
read com.apple.network.local-network` shows the new setting while the filter still behaves as
before.
It writes both of Apple's keys with sudo, verifies the values read back, and offers to reboot. The
values are only read at boot, so **the reboot is not optional** — until it happens, `defaults read
com.apple.network.local-network` shows the new setting while the filter still behaves as before.
Use `/18`, not `/24`. Virtualization.framework's NAT starts at `192.168.64.0/24` but chooses the
subnet at runtime and steps to the next free /24 when that one is in use, so hosts drift to
`192.168.65.x` and beyond. An allowlist naming a single /24 that the NAT has since moved off is the
worst case: it reads as configured, `doctor` used to call it a pass, and every guest connection
still fails with errno 65. `doctor` now warns instead when the allowlist does not cover
`192.168.64.0`–`192.168.127.255`.
The default grant is all of RFC 1918. If you narrow it with `--subnet`, use `/18`, not `/24`.
Virtualization.framework's NAT starts at `192.168.64.0/24` but chooses the subnet at runtime and
steps to the next free /24 when that one is in use, so hosts drift to `192.168.65.x` and beyond. An
allowlist naming a single /24 that the NAT has since moved off is the worst case: it reads as
configured, `doctor` used to call it a pass, and every guest connection still fails with errno 65.
`doctor` now warns instead when the allowlist does not cover `192.168.64.0`–`192.168.127.255`, and
`permissions grant` warns at the point you ask for something that narrow.
**If you are at the machine and would rather not reboot,** `permissions grant --method prompt`
launches the installed app through LaunchServices so the system alert is attributed to the app
rather than to Terminal, and takes effect immediately. It needs a GUI session and a Developer ID
signature to stick across rebuilds.
**Verifying.** After the reboot, `gitea-macos-runner doctor` should show `local network access` as a
pass naming the range. Re-run the command that failed; nothing else needs redoing, and