This directory is the reproducible data and training pipeline for docs/PURPOSE_CLASSIFIER.md. The current slice covers work item 2 and the first part of work item 3: deterministic curation/splitting, a frozen v1 eval set, MiniLM fine-tuning, temperature calibration, shared confidence thresholds, and the frozen-set accuracy, recall, hard-slice, calibration, and latency report.

Data contract

The canonical generated sources are listed in data/generation-manifest.json. round2-NN.jsonl files are retained generation batches and intentionally duplicate purpose-prompts-round2.jsonl; they are provenance, not additional training input.

prepare_data.py:

validates the strict generated-record schema;
removes exact and high-overlap word-trigram duplicates;
fails for review if a high-overlap pair has conflicting labels;
keeps shipped fixtures completely outside source data;
holds every vague-eval record out of training;
optionally applies a completed, versioned human-review ledger before splitting;
stratifies by primary purpose, slice, and primary language; and
verifies that the deterministic test partition still matches the versioned data/frozen-test-v1.jsonl.

The frozen test set is the synthetic JSONL plus the 87 classifiable records in Tests/NucleicCoreTests/Fixtures/purpose-prompts.json. The fixture file's five general records are excluded because general is deliberately not a model label. The exact membership and hashes are locked in data/dataset-v1-manifest.json.

Full Sol-high label reset

rebuild_sol_high.py is the end-to-end workflow for intentionally invalidating the old labels and rebuilding both classifiers from one teacher. It freezes and relabels every canonical public prompt, every shipped fixture, local Nucleic first-prompt history, and the pinned SWE-chat candidates with gpt-5.6-sol at high reasoning effort. The teacher settings are not configurable in this workflow. Batches default to eight prompts and failed structured responses are retried up to ten times.

The reset is staged below the ignored .artifacts/sol-high-reset/ directory. Snapshot hashes prevent a resumed run from silently mixing input revisions or teacher settings. Run these commands from the repository root with the purpose-classifier environment:

"$PY" ml/purpose-classifier/rebuild_sol_high.py snapshot

"$PY" ml/purpose-classifier/rebuild_sol_high.py label \
  --python "$PY"

"$PY" ml/purpose-classifier/rebuild_sol_high.py status

Rerunning label resumes every existing decision log and does not relabel completed lines. Once status reports "complete": true, promote the staged result explicitly:

"$PY" ml/purpose-classifier/rebuild_sol_high.py promote \
  --confirm overwrite-all-labels-with-sol-high

If promotion reports a near-duplicate label conflict in the combined real-session population, inspect both prompts and exclude the reviewed bad/ambiguous copy explicitly. The 1-based line is relative to the reported combined-real.labeled.jsonl; repeat the option for multiple decisions:

"$PY" ml/purpose-classifier/rebuild_sol_high.py promote \
  --confirm overwrite-all-labels-with-sol-high \
  --exclude-real-line <line>

Each exclusion is bound to the prompt hash, prior purpose, and review reason in the combined dataset manifest and promotion result. This option does not change a teacher label or relax duplicate detection.

Promotion first validates and curates the complete staged population. It then replaces the two canonical source files, regenerates all round-two mirrors, relabels shipped fixtures (teacher-rejected fixtures become the runtime general fallback), rebuilds the frozen public evaluation split, and replaces .artifacts/dataset-v1/ with the combined training dataset. Nucleic history and SWE-chat are deduplicated against public evaluation data and added to training only; vague-eval records remain optimization exclusions. The prior files are moved into a timestamped ignored backup before replacement.

The old human/semantic review assertions are marked superseded because they were tied to the invalidated label population. Consequently, Sol-high v2 evaluation measures agreement with the new teacher and is not directly comparable with the old v1 scores.

Print the two clean, from-pretrained-base training commands after promotion:

"$PY" ml/purpose-classifier/rebuild_sol_high.py train-commands

By default the generated commands use the Python interpreter that invoked rebuild_sol_high.py, so running this from an activated virtual environment emits that environment's concrete Python path. Pass --python <path> only to override it.

The purpose-lite command starts from the pinned MiniLM revision. The purpose-deep command starts from the pinned ModernBERT base variant; neither command supplies a prior classifier checkpoint or continuation flag.

Label local Nucleic history

export_nucleic_prompts.py extracts the first userText event from every local transcript. label_nucleic_prompts.py then removes malformed, empty, NUL-containing, and normalized-duplicate lines before asking gpt-5.6-terra to reject semantic junk and label the retained prompts. The result uses the exact canonical seven-field source-data contract. All generated files stay under the gitignored .artifacts/ directory because they contain private prompt history.

python3 ml/purpose-classifier/export_nucleic_prompts.py \
  --sessions-dir "$HOME/Library/Application Support/Nucleic/sessions" \
  --output ml/purpose-classifier/.artifacts/nucleic-history-first-prompts.unlabeled.jsonl \
  --manifest ml/purpose-classifier/.artifacts/nucleic-history-first-prompts.manifest.json

python3 ml/purpose-classifier/label_nucleic_prompts.py

The labeler writes the dataset, a rejection audit, and an append-only state file. If a Codex call or the process stops partway through, continue without re-labeling completed batches:

python3 ml/purpose-classifier/label_nucleic_prompts.py --resume

By default Codex runs ephemerally at low reasoning effort, ignores user configuration and project rules, and is instructed not to use tools. --codex-isolation auto uses Codex's read-only isolation on a host and the existing outer isolation when the script runs in a Nucleic managed container.

SWE-chat v2 import (gated source)

The SWE-chat source is not downloaded by this repository. After accepting the dataset's Hugging Face conditions, place a pinned Parquet snapshot below the ignored .artifacts/swe-chat/raw/ directory, record its immutable revision, then run the streaming extractor. It reads only the needed columns, takes the first qualifying human prompt plus its conversational agent response, and writes the prompt plus a response hash. Do not use main as a revision.

ml/purpose-classifier/.venv/bin/pip install -r \
  ml/purpose-classifier/requirements-swe-chat.txt
ml/purpose-classifier/.venv/bin/python \
  ml/purpose-classifier/export_swe_chat.py \
  --revision <accepted-immutable-hf-revision>

The export and manifest remain ignored because candidate JSONL temporarily contains the prompt and agent response. Run the one-record schema/availability canary before the 100-session dry run; both use Luna through subscription-backed codex exec, not an API key. The labeler writes only the first message to canonical source JSONL; state and audit sidecars retain the response solely as a hash and source ID.

The labeler sends all remaining explanatory response prose to Luna. It replaces fenced code and structured tool/reasoning payloads with size-marked placeholders, and fails on oversized remaining prose rather than silently truncating it. The default response limit is 48,000 characters; raise --max-response-chars only after inspecting an outlier.

ml/purpose-classifier/.venv/bin/python \
  ml/purpose-classifier/label_swe_chat_prompts.py --limit-sessions 1
ml/purpose-classifier/.venv/bin/python \
  ml/purpose-classifier/label_swe_chat_prompts.py --limit-sessions 100

Use a fresh --output path for the dry run, then manually audit it before invoking the full resumable run. Cases marked recoverableFromFirst=false remain vague-eval abstention evidence and are excluded from optimization.

Prepare

From the repository root:

python3 ml/purpose-classifier/validate-data.py
python3 ml/purpose-classifier/prepare_data.py
python3 -m unittest discover -s ml/purpose-classifier/tests -p 'test_*.py'

For a newly generated raw 200-record batch, enable batch-shape checks explicitly with validate-data.py path/to/batch.jsonl --batch-size 200 --expected-total 200.

The generated train/validation copies land under .artifacts/dataset-v1/ and are gitignored. A source, curation, seed, or split-policy change that moves the frozen test set fails closed. After reviewing such a change, intentionally version it with:

python3 ml/purpose-classifier/prepare_data.py --refresh-frozen-test

Train purpose-lite

Use a dedicated virtual environment. The base model is pinned to a specific sentence-transformers/all-MiniLM-L6-v2 commit: a 6-layer, 384-dimensional encoder. The training collator always pads/truncates to 128 tokens so the later ONNX/Core ML export can expose a fixed 1 x 128 runtime shape. Long prompts preserve both ends as [CLS] + 63 head tokens + [SEP] + 62 tail tokens + [SEP]; this keeps the ask when it follows a pasted log or stack trace while retaining enough leading context to interpret it.

python3 -m venv ml/purpose-classifier/.venv
ml/purpose-classifier/.venv/bin/pip install -r ml/purpose-classifier/requirements.txt
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/train.py

The default requirements use PyTorch's CPU-only wheel on Linux, avoiding an accidental multi-gigabyte CUDA install in CI and development containers. For NVIDIA, AMD, or Intel accelerator training, install the platform's torch==2.13.0 build using PyTorch's platform selector, then install requirements-base.txt.

Training writes a local checkpoint, calibration.json, and metrics.json under outputs/purpose-lite-v1/. It selects checkpoints and fits temperature on label-scorable validation records. When deriving nested HIGH/MEDIUM/LOW cutoffs, every vague-eval record counts as an abstention miss even if its synthetic label happens to match. The incoming checkpoint is scored and retained as epoch zero, so a continuation run cannot silently replace it with a regression. Validation early stopping defaults to two epochs without an improvement greater than 0.05 points.

Continuation training accepts a local checkpoint. --boundary-weight is an opt-in, validation-selected loss weight for the measured weakest slice; it does not add held-out fixtures to training:

ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/train.py \
  --model ml/purpose-classifier/outputs/purpose-lite-v1/model \
  --epochs 3 --learning-rate 3e-6 --warmup-ratio 0 \
  --boundary-weight 2 \
  --output-dir ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune \
  --overwrite-output

For QAT, --quantization-aware replaces the model's linear and embedding forwards with straight-through fake quantization matching the shipping QDQ graph: per-tensor uint8 embeddings, per-channel symmetric int8 linear weights, and per-tensor uint8 activations. Parameter names remain unchanged, so the selected checkpoint reopens as an ordinary Transformers model and uses the same export.py path. Keep the incoming checkpoint as epoch zero and select QAT only on validation. Training logs progress every 50 batches by default (--progress-steps 0 disables it), so a long CPU run remains observable:

ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/train.py \
  --model ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune/model \
  --epochs 2 --learning-rate 1e-6 --warmup-ratio 0 \
  --early-stopping-patience 1 --boundary-weight 2 --quantization-aware \
  --output-dir ml/purpose-classifier/outputs/purpose-lite-v1-qat1 \
  --overwrite-output

On dataset v1, that validation-selected run produced a 23,148,500-byte int8 graph at 94.88% frozen accuracy (889/937), 94.46% scored-hard accuracy, and 98.19% scorable PyTorch↔ONNX agreement. It was the pre-distillation quantized candidate and remained two correct predictions below the 95% gate. A subsequent validation-selected 5e-7 epoch improved int8 validation accuracy from 93.31% to 93.71% but regressed frozen accuracy to 94.34%; it is rejected. Do not continue optimizer-only QAT sweeps on this split. The next model iteration should incorporate reviewed boundary data and be selected on a revised validation/frozen dataset version.

To target only the remaining float→int8 decision drift, cache the float teacher in a separate inference process and use its logits for QAT distillation. Keeping teacher and student models out of the same process avoids doubling peak resident memory:

ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/cache_teacher.py \
  --model ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune/model \
  --output ml/purpose-classifier/outputs/purpose-lite-v1-boundary-teacher.pt \
  --overwrite-output
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/train.py \
  --model ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune/model \
  --distillation-cache \
    ml/purpose-classifier/outputs/purpose-lite-v1-boundary-teacher.pt \
  --distillation-weight 0.9 --distillation-temperature 2 \
  --distillation-selection-weight 0.5 --quantization-aware \
  --epochs 2 --learning-rate 1e-6 --warmup-ratio 0 --boundary-weight 1 \
  --output-dir ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat \
  --overwrite-output

The cache binds each logit row to normalized prompt hash plus expected label. Training fails closed if either split changes. Selection combines label accuracy with float-teacher agreement, retains the incoming checkpoint as epoch zero, and logs label/distillation loss separately. A 64-record wiring run exercised cache loading, shuffled row alignment, backpropagation, selection, and ordinary checkpoint reload. The current shared CPU runtime then showed severe post-batch throttling, so no full candidate result is claimed from that canary.

Native Apple Silicon training with MLX

Use the MLX backend when training on Apple Silicon. It implements the same six-layer BERT classifier, fixed head-tail tokenization, export-matched QAT graph, cached-teacher distillation, validation selection, and early stopping with native MLX arrays. Fake quantization is decomposed into Metal-supported round, clip, and straight-through-gradient operations, avoiding PyTorch's unsupported MPS fake-quant operator. Selected weights are written back with the original Hugging Face parameter names, so the existing PyTorch export.py and eval.py paths remain unchanged.

Install the additional pinned dependency into the macOS virtual environment:

ml/purpose-classifier/venv/bin/python -m pip install \
  -r ml/purpose-classifier/requirements-mlx.txt

Before the first full run on a new MLX or Transformers version, run the fail-closed parity check. It requires exact fake-quant primitives, float-logit parity, matching QAT predictions with bounded backend drift, healthy QAT gradients, and an exact Hugging Face → MLX → Hugging Face weight round trip:

ml/purpose-classifier/venv/bin/python ml/purpose-classifier/verify_mlx.py \
  --model ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune/model

Then run the distilled QAT candidate natively on Metal:

ml/purpose-classifier/venv/bin/python -u ml/purpose-classifier/train_mlx.py \
  --model ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune/model \
  --distillation-cache \
    ml/purpose-classifier/outputs/purpose-lite-v1-boundary-teacher.pt \
  --distillation-weight 0.9 --distillation-temperature 2 \
  --distillation-selection-weight 0.5 --quantization-aware \
  --epochs 4 --early-stopping-patience 1 \
  --learning-rate 1e-6 --warmup-ratio 0 --boundary-weight 1 \
  --progress-steps 1 \
  --output-dir ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e \
  --overwrite-output

The full Metal run stopped after epoch three and selected epoch two at 94.75% fake-quant validation accuracy. Its 23,148,500-byte int8-QDQ export scores 95.20% frozen (892/937), 95.19% macro recall, 94.17% scored-hard accuracy, and 98.08% scorable PyTorch↔ONNX agreement. Every purpose recall is above 91%, and the vague-abstention and routing-tier-drift gates pass. This is the current accuracy-qualified shipping candidate; latency and energy/residency still require measurement on the target Apple and Windows accelerator runtimes.

First-prompt history augmentation experiment

prepare_history_experiment.py appends the labeled Nucleic first-prompt corpus to training only. It preserves validation and test byte-for-byte, excludes vague-eval records from optimization, removes exact base/evaluation overlap, and applies the canonical 0.92 near-duplicate guard against evaluation fixtures and earlier history records. Every exclusion is represented only by hashes and source line in history-exclusions.jsonl; manifest.json binds all input and output hashes.

Build the augmented split and its teacher cache:

ml/purpose-classifier/venv/bin/python \
  ml/purpose-classifier/prepare_history_experiment.py
ml/purpose-classifier/venv/bin/python -u \
  ml/purpose-classifier/cache_teacher.py \
  --dataset-dir \
    ml/purpose-classifier/.artifacts/dataset-v1-history-first-prompts \
  --model ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune/model \
  --output \
    ml/purpose-classifier/outputs/purpose-lite-v1-history-first-prompts-teacher.pt \
  --device mps --batch-size 16 --progress-steps 25

Then run the same validation-selected MLX recipe as the accepted baseline:

ml/purpose-classifier/venv/bin/python -u ml/purpose-classifier/train_mlx.py \
  --device metal \
  --dataset-dir \
    ml/purpose-classifier/.artifacts/dataset-v1-history-first-prompts \
  --model ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune/model \
  --distillation-cache \
    ml/purpose-classifier/outputs/purpose-lite-v1-history-first-prompts-teacher.pt \
  --distillation-weight 0.9 --distillation-temperature 2 \
  --distillation-selection-weight 0.5 --quantization-aware \
  --epochs 4 --early-stopping-patience 1 \
  --learning-rate 1e-6 --warmup-ratio 0 --boundary-weight 1 \
  --progress-steps 1 \
  --output-dir \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-history-v1

MLX remains Metal-first. --device cpu is an explicit diagnostic fallback for parity checks and bounded smoke tests; it is not an acceptable full-training path when Metal is available.

The completed Metal run stopped after epoch two and selected epoch one at 94.42% fake-quant validation accuracy and a 95.59% teacher-aware selection score. Its selected float checkpoint improves frozen v1 to 95.09% (891/937), two correct decisions above the accepted baseline's float checkpoint. That gain does not survive export: the matched 23,148,500-byte int8-QDQ graph scores 94.66% (887/937), 94.66% macro recall, and 94.17% scored-hard accuracy. It is five correct decisions behind the accepted int8 baseline and fails the 95% shipping gate. Preserve the artifact as a rejected experiment; purpose-lite-v1-distilled-qat-mlx-4e remains the candidate of record.

Train purpose-deep

The next classifier tier is an MLX-native ModernBERT multi-task model. It keeps the primary eight-way purpose output and jointly learns secondary purpose, a mixed-intent flag, and advisory difficulty. Both upstream rungs are immutable: base is ModernBERT 149M at revision 8949b909ec900327062f0ebf497f51aef5e6f0c8; large is ModernBERT 395M at revision 45bb4654a4d5aaff24dd11d4781fa46d39bf8c13.

Before the first run on a new MLX/Transformers version, compare the real pinned backbone against Hugging Face. The check crosses ModernBERT's local-attention window and fails if pooled-representation drift exceeds 5e-4:

ml/purpose-classifier/venv/bin/python ml/purpose-classifier/verify_deep_mlx.py \
  --variant base

Start with the base ablation rung and the validated first-prompt history augmentation. The trainer downloads the pinned checkpoint on first use, fixes every input at 512 tokens (255 head + 254 tail + three special tokens for long prompts), and uses gradient checkpointing by default. Checkpoint selection is half scored overall accuracy and half scored hard-slice accuracy; auxiliary heads are reported independently and cannot hide a primary-purpose regression.

ml/purpose-classifier/venv/bin/python -u \
  ml/purpose-classifier/train_deep_mlx.py \
  --variant base \
  --dataset-dir \
    ml/purpose-classifier/.artifacts/dataset-v1-history-first-prompts \
  --epochs 3 --early-stopping-patience 1 \
  --progress-steps 10 \
  --output-dir ml/purpose-classifier/outputs/purpose-deep-v1-base-mlx \
  --overwrite-output

The defaults use batch size 4 and learning rate 2e-5 for base (2 and 1e-5 for large). If unified memory is tight, lower --batch-size before disabling gradient checkpointing. --device cpu is diagnostic only: a real 512-token backward pass is expected to be extremely slow there. Each improved epoch atomically rewrites model/ and updates training-state.json, so progress is visible and an interrupted run retains the last selected checkpoint.

To continue a completed run without discarding its trained task heads, pass its selected model/ directory through --resume-from and write to a new output directory. Continuation restores the backbone and all four heads strictly, then starts a fresh optimizer and learning-rate schedule; --model remains reserved for an untrained local upstream checkpoint. The first base run was still improving when its three-epoch schedule ended, so its selected checkpoint was continued conservatively before changing architecture:

ml/purpose-classifier/venv/bin/python -u \
  ml/purpose-classifier/train_deep_mlx.py \
  --variant base \
  --resume-from \
    ml/purpose-classifier/outputs/purpose-deep-v1-base-mlx/model \
  --dataset-dir \
    ml/purpose-classifier/.artifacts/dataset-v1-history-first-prompts \
  --epochs 3 \
  --learning-rate 1e-5 \
  --early-stopping-patience 2 \
  --progress-steps 10 \
  --output-dir \
    ml/purpose-classifier/outputs/purpose-deep-v1-base-mlx-cont-3e \
  --overwrite-output

That continuation reached 80.31% primary and 78.28% hard-slice validation accuracy; calibrated mixed F1 reached 67.12%. It remained far below purpose-lite, while primary training loss and validation accuracy were still improving. Do not chain another plain continuation. The bounded next experiment distills the mature purpose-lite boundary teacher into the continued deep checkpoint while retaining direct primary labels and all three auxiliary losses.

Create a teacher cache bound to the history-augmented split:

ml/purpose-classifier/venv/bin/python -u \
  ml/purpose-classifier/cache_teacher.py \
  --dataset-dir \
    ml/purpose-classifier/.artifacts/dataset-v1-history-first-prompts \
  --model \
    ml/purpose-classifier/outputs/purpose-lite-v1-boundary-tune/model \
  --output \
    ml/purpose-classifier/outputs/purpose-lite-v1-history-first-prompts-teacher.pt \
  --device mps \
  --batch-size 16 \
  --progress-steps 25 \
  --overwrite-output

Then run two validation-selected distilled continuation epochs:

ml/purpose-classifier/venv/bin/python -u \
  ml/purpose-classifier/train_deep_mlx.py \
  --variant base \
  --resume-from \
    ml/purpose-classifier/outputs/purpose-deep-v1-base-mlx-cont-3e/model \
  --dataset-dir \
    ml/purpose-classifier/.artifacts/dataset-v1-history-first-prompts \
  --distillation-cache \
    ml/purpose-classifier/outputs/purpose-lite-v1-history-first-prompts-teacher.pt \
  --distillation-weight 0.5 \
  --distillation-temperature 2 \
  --epochs 2 \
  --learning-rate 1e-5 \
  --early-stopping-patience 1 \
  --progress-steps 10 \
  --output-dir \
    ml/purpose-classifier/outputs/purpose-deep-v1-base-mlx-distilled \
  --overwrite-output

The trainer records the resumed checkpoint as epoch zero before updating anything, so a distillation regression cannot overwrite the 80.31% candidate. Teacher agreement is reported for diagnosis but does not enter deep checkpoint selection; overall and hard primary label accuracy remain the only selection inputs.

Do not launch the large rung yet. It is justified only after base is evaluated on the frozen set; large must beat base by at least two hard-slice points, while deep itself must reach 97% scored overall and beat the shipping lite artifact by five hard-slice points.

Convert and validate Core ML

Core ML Tools no longer maintains the legacy ONNX converter, so the Apple artifact is converted directly from the selected Hugging Face checkpoint. convert_coreml.py uses a fixed-shape export-only BERT forward to avoid dynamic Transformers masking helpers, checks that forward against Transformers before conversion, writes an ML Program package, and records hashes for every package file.

Install the pinned converter in the macOS environment and create the package:

ml/purpose-classifier/venv/bin/python -m pip install \
  -r ml/purpose-classifier/requirements-coreml.txt
ml/purpose-classifier/venv/bin/python ml/purpose-classifier/convert_coreml.py \
  --model-dir \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/model \
  --output \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/coreml/purpose-lite-v1-fp16.mlpackage \
  --overwrite-output

The direct float16 package is the conversion baseline, not the accepted Apple artifact. On the first physical Apple-Silicon run it scored 94.98% (890/937), two correct decisions behind the accepted ONNX graph, with 97.97% scorable label agreement. Calibrate a Core ML-native W8A8 candidate with the same deterministic 256-record sample and QDQ policy as the ONNX exporter:

ml/purpose-classifier/venv/bin/python ml/purpose-classifier/quantize_coreml.py \
  --model \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/coreml/purpose-lite-v1-fp16.mlpackage \
  --model-dir \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/model \
  --output \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/coreml/purpose-lite-v1-w8a8.mlpackage \
  --overwrite-output

Activation calibration writes its temporary packages under the candidate output directory and removes each package immediately after prediction; this avoids Core ML Tools retaining one full weight copy per calibration step until process exit. It prints progress while it runs. The successfully rewritten A8 package is cached beside the W8A8 output and reused only when its source hash, Core ML Tools version, activation policy, calibration seed, and prompt hashes match exactly. This prevents a later weight-stage failure from forcing another calibration. The candidate uses per-tensor asymmetric uint8 activations, per-channel symmetric int8 linear weights, and per-tensor asymmetric uint8 embedding weights. Activation quantization is limited to floating-point linear operations; applying Core ML Tools' global policy also selects integer embedding-index additions and produces an invalid quantize operation. It fails the command if the resulting package exceeds 25 MiB.

Run the frozen gate with CPU+Neural Engine placement and compare labels directly with the accepted int8 ONNX artifact. Gated Core ML evaluation fails closed without --compare-onnx, and requires at least 99.5% scorable label agreement:

ml/purpose-classifier/venv/bin/python ml/purpose-classifier/eval.py \
  --model-dir \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/model \
  --calibration \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/calibration.json \
  --coreml-model \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/coreml/purpose-lite-v1-w8a8.mlpackage \
  --coreml-compute-units cpu-and-ne \
  --compare-onnx \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/export/purpose-lite-v1-int8-qdq.onnx \
  --report \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/coreml/ane-frozen-eval.json

Record the compute plan separately; this reports both operation-count and estimated-cost ANE shares. Repeat evaluation with --coreml-compute-units cpu-only --no-gate before the energy comparison in ENERGY_AND_RESIDENCY.md:

ml/purpose-classifier/venv/bin/python ml/purpose-classifier/inspect_coreml.py \
  --model \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/coreml/purpose-lite-v1-w8a8.mlpackage \
  --compute-units cpu-and-ne \
  --report \
    ml/purpose-classifier/outputs/purpose-lite-v1-distilled-qat-mlx-4e/coreml/ane-compute-plan.json

For a wiring smoke test, use a small deterministic prefix:

ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/train.py \
  --epochs 1 --max-train-records 64 --max-validation-records 64 \
  --output-dir ml/purpose-classifier/outputs/smoke --overwrite-output

Evaluate

ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/eval.py

The command returns failure unless label-scorable frozen accuracy is at least 95%, every purpose recall is at least 85%, at least 90% of the deliberately context-free vague-eval slice resolves LOW, every misroute stays within one routing cost tier, and measured batch-one p95 is at most 20 ms. Use --no-gate only for diagnostic runs.

Export and score ONNX

ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/export.py
ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/eval.py \
  --onnx-model ml/purpose-classifier/outputs/purpose-lite-v1/export/purpose-lite-v1-int8-qdq.onnx \
  --report ml/purpose-classifier/outputs/purpose-lite-v1/export/int8-frozen-eval.json

export.py emits fixed-shape opset-17 fp16 and int8-QDQ graphs, a tokenizer/ normalization contract, golden tokenizations, shared calibration config, graph checks, artifact hashes, and a size report. Its default 256-record quantization calibration sample is deterministic and stratified by purpose, slice, and primary language; the export report records the seed, distribution, and prompt hashes. The int8 graph is the ≤25 MiB shipping candidate; the fp16 graph remains the accelerator-oriented conversion input.

When scoring ONNX, add --compare-pytorch to measure artifact drift against --model-dir. The report then includes overall, label-scorable, and per-slice label agreement plus every correct→incorrect, incorrect→correct, and changed-wrong-label transition:

ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/eval.py \
  --onnx-model ml/purpose-classifier/outputs/purpose-lite-v1/export/purpose-lite-v1-int8-qdq.onnx \
  --compare-pytorch --no-gate

Audit curation

Run the semantic embedding duplicate audit. It also emits the deterministic, purpose/slice/language-stratified 10% human label-and-difficulty review CSV:

ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/audit_data.py

The semantic pass uses the same commit-pinned MiniLM encoder and fixed 128-token input as purpose-lite. Similarity only proposes review candidates; it never edits source data or the frozen split automatically. The 18 word-trigram exclusions and the human-review completion rule are recorded in data/curation-review-v1.json; the semantic report is versioned as data/semantic-audit-v1.json.

Optional human review

The dataset owner accepted the curated generated labels and difficulty metadata as-is on 2026-07-31, so the blank 1,219-row review sample is not a training or rollout blocker. It remains available as an optional future audit. Check its progress without running the embedding audit again:

ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/review_data.py

Mark each row accept, relabel, or reject. accept and reject leave the four reviewed* fields blank; reject requires notes. For relabel, blank reviewed fields retain their generated value, <none> clears a secondary purpose, and notes are required. If a secondary purpose is added or removed, set reviewedSlice consistently (mixed when a secondary is present). The validator rejects stale generated columns, missing or duplicate sample rows, invalid label combinations, and partially completed rows.

When every row has a human decision, write the versionable ledger:

ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/review_data.py --finalize

Build an isolated candidate split first:

ml/purpose-classifier/.venv/bin/python ml/purpose-classifier/prepare_data.py \
  --human-review ml/purpose-classifier/data/human-review-v1.json \
  --output-dir ml/purpose-classifier/.artifacts/reviewed-candidate \
  --frozen-test ml/purpose-classifier/.artifacts/reviewed-frozen-candidate.jsonl \
  --manifest ml/purpose-classifier/.artifacts/reviewed-manifest-candidate.json \
  --refresh-frozen-test

Inspect the ledger, decision summary, candidate manifest, and split diff. Only then rerun the same command with the three candidate-path overrides removed to intentionally replace the versioned frozen dataset and manifest.

--regenerate recreates a blank CSV in the current schema and is only appropriate before review begins.

The one-time, hardware-bound energy and accelerator-residency procedure is in ENERGY_AND_RESIDENCY.md.