Build the custom vminit image locally via make vminit-image; drop the CI workflow
The GitHub-hosted macos runners can't build the host framework (needs the macOS 26+ Virtualization SDK), so publish the custom vminit guest image from a local macOS 26/27 machine instead. Adds root-Makefile targets: - vminit-image-prep : one-time swiftly + musl static SDK install - vminit-image : build cctl + cross-build vminitd, package the image - vminit-image-push : push to GHCR (REGISTRY_* env creds) Forces WARNINGS_AS_ERRORS=false (Xcode Swift 6.4 rejects -warnings-as-errors alongside SwiftPM's -suppress-warnings). Removes .github/workflows/vminit-image.yml and repoints vminitReference + PATCHES.md docs at the Makefile. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Claude Opus 4.8
@@ -1,118 +0,0 @@
|
||||
name: Publish custom vminitd init image
|
||||
|
||||
# Builds and pushes Nucleic's **custom** vminitd guest image — the init filesystem (guest PID 1,
|
||||
# gRPC-over-vsock) that every Nucleic Control / sandbox container boots. Nucleic pulls this instead of
|
||||
# Apple's stock `ghcr.io/apple/containerization/vminit` (see ContainerEngine.vminitReference) because
|
||||
# we carry guest-side patches in third_party/containerization/vminitd (per-exec cgroups, event-loop
|
||||
# offload — see third_party/containerization/PATCHES.md).
|
||||
#
|
||||
# Built from the VENDORED source (third_party/containerization, pinned upstream commit 6b7b42ca), so
|
||||
# the guest vminitd and the host `Containerization` framework are compiled from the same tree and are
|
||||
# guaranteed protocol-compatible — a stock `vminit:0.34.0` could drift from our vendored commit.
|
||||
#
|
||||
# Runner: macOS 26 arm64. This must be macOS (not Linux like the rest of our CI) because `cctl images
|
||||
# push` is macOS-only (`#if os(macOS)` in cctl/ImageCommand.swift) and the Makefile `init` flow is a
|
||||
# Darwin flow (it codesigns cctl). It must be macOS *26* specifically: `make containerization` compiles
|
||||
# the whole host framework, and Sources/Containerization/{VmnetNetwork,NATNetworkInterface}.swift use
|
||||
# vmnet_network_ref / VZVmnetNetworkDeviceAttachment — symbols that exist only in the macOS 26 SDK, so
|
||||
# an older runner (macos-15) fails with "cannot find type 'vmnet_network_ref' in scope". The guest
|
||||
# binaries are cross-built for aarch64-linux-musl via the Swift static SDK; this job only packages +
|
||||
# pushes an image, it does not boot a VM.
|
||||
#
|
||||
# If GitHub-hosted `macos-26` isn't available in this org yet, switch to a self-hosted macOS 26 runner
|
||||
# (the dev machines already run macOS 26 / Darwin 27, which is why local `swift build` succeeds).
|
||||
#
|
||||
# Manual + on-change. FIRST publish only: set the `vminit` GHCR package to Public (GitHub → Packages)
|
||||
# so it can be pulled anonymously while the repo stays private — exactly like nucleic-kernel /
|
||||
# nucleic-sandbox / the linux-vm-agents artifacts.
|
||||
#
|
||||
# NOTE (unvalidated until first run): CI for this image has not executed yet. The build steps mirror
|
||||
# the documented `make init` + `cctl images push` flow and third_party/containerization/.github/
|
||||
# workflows/linux-build.yml, but the toolchain/SDK install and the push have to be confirmed on the
|
||||
# first dispatch. Keep `publish: false` on the first run to build-only and inspect the artifact.
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
tag:
|
||||
description: "GHCR tag for the vminit image (keep in lockstep with ContainerEngine.vminitReference)"
|
||||
required: true
|
||||
default: "0.34.0-nucleic1"
|
||||
publish:
|
||||
description: "Push to GHCR (else just build + upload the image tar as a workflow artifact)"
|
||||
type: boolean
|
||||
default: true
|
||||
push:
|
||||
paths:
|
||||
- "third_party/containerization/vminitd/**"
|
||||
- "third_party/containerization/Sources/Containerization/**"
|
||||
- ".github/workflows/vminit-image.yml"
|
||||
|
||||
jobs:
|
||||
vminit:
|
||||
name: Build + push custom vminit image
|
||||
runs-on: macos-26 # macOS 26 arm64 — needs the macOS 26 SDK (vmnet_network_* symbols); see header
|
||||
timeout-minutes: 45
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
defaults:
|
||||
run:
|
||||
working-directory: third_party/containerization
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
# Install the exact Swift the vendored tree pins (.swift-version = 6.3.0) via swiftly, plus the
|
||||
# aarch64 static-linux (musl) SDK the guest binaries cross-compile against. `cross-prep` is the
|
||||
# repo's own target for this (vminitd/Makefile: `cross-prep: swift linux-sdk`).
|
||||
- name: Install Swift 6.3 toolchain + musl static SDK
|
||||
run: make -C vminitd cross-prep
|
||||
|
||||
# Put swiftly's Swift on PATH so BOTH makefiles use 6.3.0 (the top Makefile defaults SWIFT to
|
||||
# /usr/bin/swift on Darwin, which may be an older Xcode Swift — force the swiftly one instead).
|
||||
- name: Select the swiftly toolchain
|
||||
run: |
|
||||
echo "$HOME/.swiftly/bin" >> "$GITHUB_PATH"
|
||||
|
||||
- name: Resolve GHCR reference
|
||||
id: ref
|
||||
run: |
|
||||
O=$(echo '${{ github.repository_owner }}' | tr '[:upper:]' '[:lower:]')
|
||||
echo "image=ghcr.io/$O/vminit:${{ inputs.tag }}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Build cctl (native macOS, ad-hoc codesigned by `make containerization`) and vminitd/vmexec
|
||||
# (aarch64-linux-musl static via the SDK), then assemble the init image into the local content
|
||||
# store under our GHCR reference. Mirrors the Makefile `init` target with `--image` set to our ref.
|
||||
- name: Build cctl + vminitd (musl) and assemble the init image
|
||||
env:
|
||||
SWIFT: swift # the swiftly 6.3 one, now first on PATH
|
||||
run: |
|
||||
set -euo pipefail
|
||||
make containerization
|
||||
make -C vminitd # default LIBC=musl → static aarch64-linux-musl vminitd + vmexec
|
||||
rm -f bin/init.rootfs.tar.gz bin/init.block bin/initfs.ext4
|
||||
./bin/cctl rootfs create \
|
||||
--vminitd vminitd/bin/vminitd \
|
||||
--vmexec vminitd/bin/vmexec \
|
||||
--ext4 ./bin/initfs.ext4 \
|
||||
--label org.opencontainers.image.source=https://github.com/apple/containerization \
|
||||
--image "${{ steps.ref.outputs.image }}" \
|
||||
bin/init.rootfs.tar.gz
|
||||
|
||||
# Build-only fallback: on the first (unvalidated) run keep publish=false and inspect this.
|
||||
- name: Upload init ext4 as a workflow artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: vminit-initfs-arm64
|
||||
path: third_party/containerization/bin/initfs.ext4
|
||||
if-no-files-found: error
|
||||
|
||||
# `cctl images push` (macOS-only) reads registry creds from REGISTRY_HOST/USERNAME/TOKEN
|
||||
# (cctl/ImageCommand.swift authenticationFromEnv), so no docker login is needed.
|
||||
- name: Push init image to GHCR
|
||||
if: ${{ github.event_name == 'workflow_dispatch' && inputs.publish }}
|
||||
env:
|
||||
REGISTRY_HOST: ghcr.io
|
||||
REGISTRY_USERNAME: ${{ github.actor }}
|
||||
REGISTRY_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: ./bin/cctl images push "${{ steps.ref.outputs.image }}"
|
||||
@@ -213,3 +213,47 @@ runner-test: $(RUNNER_DIR)/node_modules
|
||||
## runner-deploy: build, then deploy the Worker to Cloudflare (wrangler deploy)
|
||||
runner-deploy: runner-build
|
||||
cd $(RUNNER_DIR) && npm run deploy
|
||||
|
||||
# --- custom vminitd guest image (third_party/containerization) ---
|
||||
# Nucleic boots a CUSTOM vminit initfs (guest PID 1, gRPC over vsock) instead of Apple's stock image,
|
||||
# because we carry guest-side patches in third_party/containerization/vminitd (see its PATCHES.md).
|
||||
# ContainerEngine.vminitReference points at NUCLEIC_VMINIT_REF — keep the two in lockstep, bumping the
|
||||
# `-nucleicN` suffix whenever a guest patch (or the vendored commit) changes. Built locally rather than
|
||||
# in CI: the host framework needs the macOS 26+ Virtualization SDK, which GitHub-hosted runners lack.
|
||||
#
|
||||
# One-time setup on a fresh machine (installs swiftly + the aarch64 musl static SDK the guest
|
||||
# cross-build needs; ~1 GB of downloads): make vminit-image-prep
|
||||
#
|
||||
# WARNINGS_AS_ERRORS=false is forced because Xcode's Swift 6.4 rejects upstream's `-warnings-as-errors`
|
||||
# once SwiftPM also passes `-suppress-warnings` to dependencies.
|
||||
|
||||
CTZN_DIR := third_party/containerization
|
||||
NUCLEIC_VMINIT_REF ?= ghcr.io/abkslm/vminit:0.34.0-nucleic1
|
||||
|
||||
.PHONY: vminit-image-prep vminit-image vminit-image-push
|
||||
|
||||
## vminit-image-prep: one-time — install swiftly + the musl static SDK for the guest cross-build
|
||||
vminit-image-prep:
|
||||
$(MAKE) -C $(CTZN_DIR)/vminitd cross-prep
|
||||
|
||||
## vminit-image: build the custom vminit guest image locally into the cctl content store
|
||||
vminit-image:
|
||||
$(MAKE) -C $(CTZN_DIR) containerization WARNINGS_AS_ERRORS=false
|
||||
$(MAKE) -C $(CTZN_DIR) vminitd WARNINGS_AS_ERRORS=false
|
||||
rm -f $(CTZN_DIR)/bin/init.rootfs.tar.gz $(CTZN_DIR)/bin/init.block $(CTZN_DIR)/bin/initfs.ext4
|
||||
cd $(CTZN_DIR) && ./bin/cctl rootfs create \
|
||||
--vminitd vminitd/bin/vminitd \
|
||||
--vmexec vminitd/bin/vmexec \
|
||||
--ext4 ./bin/initfs.ext4 \
|
||||
--label org.opencontainers.image.source=https://github.com/apple/containerization \
|
||||
--image $(NUCLEIC_VMINIT_REF) \
|
||||
bin/init.rootfs.tar.gz
|
||||
@echo ""
|
||||
@echo "Built $(NUCLEIC_VMINIT_REF) into the local cctl content store."
|
||||
@echo "Push it (needs a GHCR token with write:packages):"
|
||||
@echo " REGISTRY_HOST=ghcr.io REGISTRY_USERNAME=<user> REGISTRY_TOKEN=<PAT> make vminit-image-push"
|
||||
@echo "Then set the vminit GHCR package Public."
|
||||
|
||||
## vminit-image-push: push the built image to GHCR (set REGISTRY_HOST/USERNAME/TOKEN first)
|
||||
vminit-image-push:
|
||||
cd $(CTZN_DIR) && ./bin/cctl images push $(NUCLEIC_VMINIT_REF)
|
||||
|
||||
@@ -172,11 +172,13 @@ public actor ContainerEngine {
|
||||
///
|
||||
/// This is Nucleic's **custom** build (not Apple's stock `ghcr.io/apple/containerization/vminit`),
|
||||
/// because we carry guest-side patches in `third_party/containerization/vminitd` — see
|
||||
/// `third_party/containerization/PATCHES.md`. It is built and pushed by
|
||||
/// `.github/workflows/vminit-image.yml` FROM the vendored source, so the guest vminitd and the
|
||||
/// host `Containerization` framework compile from the same pinned commit and speak the same vsock
|
||||
/// protocol. **Keep this tag in lockstep with that workflow's `tag` input**; bump the `-nucleicN`
|
||||
/// suffix whenever a guest patch (or the vendored commit) changes, and re-run the workflow.
|
||||
/// `third_party/containerization/PATCHES.md`. It is built locally with `make vminit-image` (root
|
||||
/// Makefile) FROM the vendored source, so the guest vminitd and the host `Containerization`
|
||||
/// framework compile from the same pinned commit and speak the same vsock protocol. (Built locally
|
||||
/// rather than in CI because the host framework needs the macOS 26+ Virtualization SDK that
|
||||
/// GitHub-hosted runners lack.) **Keep this tag in lockstep with the Makefile's `NUCLEIC_VMINIT_REF`**;
|
||||
/// bump the `-nucleicN` suffix whenever a guest patch (or the vendored commit) changes, then re-run
|
||||
/// `make vminit-image` + `make vminit-image-push`.
|
||||
/// (`0.34.0` = the upstream vminit version this replaces; `-nucleicN` = our guest-patch revision.)
|
||||
public static let vminitReference = "ghcr.io/abkslm/vminit:0.34.0-nucleic1"
|
||||
|
||||
|
||||
+13
-8
@@ -36,9 +36,10 @@ in-tree means the patch can't be lost to a dependency re-resolve.
|
||||
stdout is never read (the "no output, just a spinner" symptom in Nucleic Control containers).
|
||||
Behavior is unchanged; it only surfaces the failing stream. Marked `[Nucleic vendored patch]`
|
||||
(the `import os`, the `nucleicIOLog` static, and the per-stream check in `setupIO`). All three are
|
||||
wrapped in `#if canImport(os)` — the swiftly toolchain used by `.github/workflows/vminit-image.yml`
|
||||
resolves Foundation/Virtualization but not the `os` overlay, so the diagnostic degrades to a no-op
|
||||
there instead of failing the build; Xcode (local) builds keep it.
|
||||
wrapped in `#if canImport(os)` — non-Xcode toolchains (e.g. a swiftly Swift used to cross-build the
|
||||
host framework) resolve Foundation/Virtualization but not the `os` overlay, so the diagnostic
|
||||
degrades to a no-op there instead of failing the build; Xcode (the local `make vminit-image` path)
|
||||
builds keep it.
|
||||
|
||||
4. **Trimmed for footprint (no behavior change).** `Tests/`, `docs/`, `examples/`, and `images/`
|
||||
were dropped, and the corresponding `.testTarget(...)` entries removed from `Package.swift`. The
|
||||
@@ -79,9 +80,13 @@ in-tree means the patch can't be lost to a dependency re-resolve.
|
||||
Patches #1–#7 are host-side (the `Containerization` library), shipped by a normal `swift build`.
|
||||
Patches #8+ live in `vminitd/` (the guest agent), which rides in the initfs OCI image. They are INERT
|
||||
until that image is rebuilt from this source and published, and `ContainerEngine.vminitReference`
|
||||
points at it. That is now automated: **`.github/workflows/vminit-image.yml`** builds vminitd from this
|
||||
vendored tree and pushes `ghcr.io/abkslm/vminit:<tag>`; `vminitReference` is pinned to that custom
|
||||
image. Bump the `-nucleicN` tag suffix and re-run the workflow whenever a guest patch changes.
|
||||
points at it. Build it with **`make vminit-image`** (root Makefile) — it builds cctl + the guest
|
||||
vminitd/vmexec from this vendored tree and packages `ghcr.io/abkslm/vminit:<tag>` into the local cctl
|
||||
store; `make vminit-image-push` (with GHCR creds in the environment) publishes it, and `vminitReference`
|
||||
is pinned to that custom image. First time on a machine, run `make vminit-image-prep` once (installs
|
||||
the swiftly toolchain + musl SDK the guest cross-build needs). Bump the `-nucleicN` tag suffix and
|
||||
rebuild whenever a guest patch changes. Built locally, not in CI: the host framework needs the macOS
|
||||
26+ Virtualization SDK that GitHub-hosted runners lack.
|
||||
|
||||
8. **`vminitd/Sources/VminitdCore/ManagedProcess.swift` — offload the blocking start off the event loop.**
|
||||
`ManagedProcess.start()` did synchronous, potentially slow pipe reads (waiting for `vmexec` to
|
||||
@@ -124,7 +129,7 @@ image. Bump the `-nucleicN` tag suffix and re-run the workflow whenever a guest
|
||||
stdio-or-abort guard in `start()`), patch #7 (the bounded `deleteProcess` timeout in
|
||||
`Vminitd.swift`), and patch #8 (the `ManagedProcess.start` event-loop offload in `vminitd/`). Grep
|
||||
for `[Nucleic vendored patch]` to find every site. Patch #9 (per-exec cgroups) is design-only so
|
||||
far — see its entry. After re-applying any `vminitd/` patch, re-run `.github/workflows/vminit-image.yml`
|
||||
to rebuild + publish the custom init image, and bump `ContainerEngine.vminitReference`.
|
||||
far — see its entry. After re-applying any `vminitd/` patch, rebuild + publish the custom init image
|
||||
with `make vminit-image` + `make vminit-image-push`, and bump `ContainerEngine.vminitReference`.
|
||||
5. Update the commit hash above and in the root `Package.swift` comment.
|
||||
6. `swift build` and run the balloon tests.
|
||||
|
||||
Reference in New Issue
Block a user