Files
nucleic/docs/APP_INTENTS_OPPORTUNITIES.md
T
abkslmandnucleic 9dbb873449 App Intents Integration Report
Nucleic-Session: 93D678C7-1451-4F73-B3CE-67BF8B2EDE02
Co-authored-by: Nucleic <[email protected]>
2026-07-05 18:26:33 -07:00

20 KiB
Raw Blame History

Nucleic — App Intents Integration Opportunities

Scope. A survey of where Apple's App Intents framework could plug into Nucleic — across both the macOS host app (Sources/NucleicApp) and the iPhone companion (ios/NucleicRemote). Grounded in the shipping code as of this writing. Every opportunity is tied to an existing action or model type, with file:line anchors, so this doubles as an implementation map. Status notes are honest about feasibility and platform constraints.

TL;DR. Nucleic is unusually well-suited to App Intents: it is an attention-routing product ("which agent needs me?") whose whole value is answering a small set of high-stakes actions — approve/deny, unblock, start, ship — from wherever you are. That is exactly the surface Siri, Shortcuts, Spotlight, interactive widgets, Control Center, and the Action button are designed to expose. There is no App Intents code today (confirmed: zero import AppIntents, AppIntent, AppEntity, or AppShortcut in the tree). The domain model is already Sendable/Codable end-to-end, and a nucleic:// deep-link scheme already exists, so the groundwork is laid.


1. Why App Intents fits Nucleic

App Intents lets an app expose typed actions (AppIntent), typed data (AppEntity), and typed choices (AppEnum) to the system so they can be invoked from Siri / Apple Intelligence, Shortcuts, Spotlight, the Action button, Control Center, interactive Widgets, and Live Activity buttons — often without launching the app.

Nucleic's product thesis (see NUCLEIC_CONCEPT.md §25, docs/UX_IOS.md §1) is that with N agents running, the bottleneck is you, and the job is to answer "which session needs me, and let me act on it in one step." That is a near-perfect match for App Intents' strengths:

  • Glanceable, one-tap resolution. docs/UX_IOS.md §1 lists "Glanceable" and "Notification-first" as invariants; §8 already calls for widgets and Live Activities. App Intents is the connective tissue that makes those surfaces actionable, not just informational.
  • A small, well-typed action vocabulary. Approve, deny, allow-always, send follow-up, start, interrupt, integrate, discard — each already exists as a method on RemoteStore (iOS) or AppStore/SessionController (macOS), and each takes stable identifier types.
  • Rich queryable state. Sessions have a canonical SessionStatus and a needsAttention computation — ideal backing for an EntityQuery that answers "what needs me?" to Siri/Spotlight.

2. Current state — what exists, what doesn't

No App Intents today. A tree-wide search for import AppIntents, AppIntent, AppEntity, @Parameter, AppShortcut, IntentDescription, and EntityQuery returns no matches in either the macOS or iOS target.

Bridging surfaces that already exist (these lower the cost of adopting App Intents):

Surface Where Relevance
nucleic:// URL scheme ios/.../SessionActivityAttributes.swift (NucleicDeepLink ~94127); consumed in NucleicRemoteApp.swift:14-15 via .onOpenURLRemoteStore.handleDeepLink() (~112118) App Intents can openAppWhenRun and route via the same deep-link map; several intents can reuse this verbatim.
Live Activity + Dynamic Island ios/.../SessionLiveActivity.swift, LiveActivityManager.swift, Shared/SessionActivityAttributes.swift Live Activities can host interactive Button(intent:) — the fastest path to approve-from-lock-screen without a notification.
Interactive notifications ios/.../Notifications.swift (categories NUCLEIC_APPROVAL_ACTIONABLE, NUCLEIC_ALLOW, NUCLEIC_DENY, ~2452; handling ~127148) Already a working approve/deny-from-lock-screen path; App Intents would complement (widgets/Siri) rather than replace it.
Widgets target ios/NucleicRemote/NucleicRemoteWidgets/ An existing WidgetKit extension — the natural home for an AppIntentConfiguration widget and ControlWidget.
Stable, Codable/Sendable identifiers SessionID, ProjectID, ApprovalID, TodoID (Protocol/CoreIdentifiers.swift) Directly usable as AppEntity.id / intent parameter types.
macOS notification & Dock hooks Sources/NucleicApp/NucleicApp.swift:109-117 (status sound, alarm, dock bounce) Shows the app already models "needs you" attention events an intent could surface to Spotlight/Siri.

3. Architectural constraints (read before scoping)

These are the honest gotchas that shape what's feasible and how hard:

  1. The iPhone is a thin client with no local authority. Every action is an intent sent over an E2EE channel to the paired Mac (docs/UX_IOS.md §2, §9; RemoteStore methods at ios/.../Models/RemoteStore.swift). Two consequences for App Intents:

    • No optimistic execution. UX_IOS.md §6 mandates that actions require a live channel and must disable rather than silently fail. An App Intent that runs in an extension process (widget/Control Center/Siri) must therefore either (a) establish/verify the channel and report a clean failure via IntentResult + a spoken/dialog error when the Mac is unreachable, or (b) fall back to openAppWhenRun so the app process (which owns the connection) performs it. There is already a precedent: RemoteStore.respondFromNotification() (~124136) queues an approval for up to ~30 s on a dropped link — an intent can mirror that contract.
    • Scope gating. Device scope is .view / .approve / .control (WireMessages.swift, RemoteStore.canControl ~180). Intents must respect it: approve/sendInput need .approve; start/merge/discard need .control (which UX_IOS.md §2 notes is a later capability). Ship the approve-scope intents first; gate control-scope intents behind the same check the UI uses.
  2. The macOS app is a SwiftPM executable, not an Xcode app project. Package.swift:60 builds NucleicApp as an .executable product (later wrapped into a signed, notarized .app for Sparkle distribution — see BUILD.md). App Intents metadata is normally extracted by the Xcode build system's appintentsmetadataprocessor into the app bundle. With a SwiftPM executable you must ensure that step runs (e.g. a packaging step that invokes the extractor, or a thin Xcode wrapper target) so AppShortcutsProvider and entities register with the system. This is the single biggest feasibility risk on the macOS side and should be de-risked with a spike before committing to macOS Shortcuts/Spotlight. iOS (NucleicRemote.xcodeproj) is a normal Xcode project and has no such issue.

  3. Approvals are security-sensitive and risk-tiered. Risk (Protocol/Approval.swift:5-10: readOnly/write/execute/network/destructive/hostExec/unknown) already drives where an approval can be answered. UX_IOS.md §5.1 and UX_MACOS.md §4 forbid inline "Allow" on destructive/network/hostExec — those must open the app behind a biometric gate. Any approve-style App Intent must honor this: expose inline Allow only for low/medium risk; for high risk, the intent should openAppWhenRun to the guarded card. .allowAlways is likewise never offered on destructive requests (UX_MACOS.md §4).

  4. First-responder-wins. An approval may be resolved on another device between when a widget/Siri surface renders and when the intent fires (SYNC §5.4). Intents must handle "already resolved" as a normal, non-error outcome (return a friendly IntentResult dialog, not a throw).


4. Opportunity catalog — iPhone (NucleicRemote) — highest value

The phone is where App Intents pays off most: it is the "act from anywhere" surface, and its extensions (widgets, Live Activity, notifications) already exist to host intent buttons.

4.1 Approve / Deny an agent from anywhere (the flagship intent)

  • What: AnswerApprovalIntent(approval:, decision:) → wraps RemoteStore.respond(approval:decision:) (RemoteStore.swift ~818827) / respondFromNotification (~124136).
  • Where it shows up:
    • Interactive Live Activity / Dynamic Island button. Today the Live Activity is display-only with a Link to nucleic://session/<id> (SessionLiveActivity.swift ~135). Replace the link with Button(intent: AnswerApprovalIntent(...)) for low/medium-risk approvals → allow/deny without unlocking into the app. This is the highest-leverage single change in this document.
    • Home/Lock-screen widget (NucleicRemoteWidgets/): a "Top approval" widget whose Allow/Deny are Button(intent:).
    • Siri / Shortcuts: "Hey Siri, approve the agent" (disambiguates when >1 pending via the EntityQuery in §4.6).
    • Apple Watch / Control Center later — same intent, new surface.
  • Parameters: an ApprovalEntity (id = ApprovalID) + a Decision-backed AppEnum (.allow, .deny, and the three AllowAlwaysScope cases from Approval.swift:12-19, excluded for destructive).
  • Constraints: §3.1 (live channel), §3.3 (risk gating — high-risk falls back to openAppWhenRun + biometric), §3.4 (already-resolved is a friendly no-op).
  • Backing model: ApprovalRequest (Protocol/Approval.swift:21-52) — already Codable.

4.2 Unblock a session with a follow-up prompt

  • What: SendFollowUpIntent(session:, text:)RemoteStore.sendInput(text:sessionID:) (RemoteStore.swift ~829833). This is the "type the next step to unblock a paused agent" flow that UX_IOS.md §2 explicitly keeps at approve scope.
  • Where: Siri ("tell the auth-refactor agent to run the tests"), a widget "Reply" affordance, Shortcuts automation. The text parameter supports Siri dictation naturally.
  • Value: turns a blocked awaitingInput session green from a voice command or a Shortcut.

4.3 "What needs me?" — a query intent + Spotlight

  • What: SessionsNeedingMeIntent returning [SessionEntity] filtered by the existing attention rule (needsAttention: awaitingApproval, or awaitingInput && not completed — see the macOS SessionSummary.needsAttention at SessionDetailView.swift ~255265; the iOS store carries the same pendingApprovalCount / status on WireSessionSummary, RemoteStore.swift:46).
  • Where: Siri ("what needs me in Nucleic?"), Spotlight (index sessions so a search surfaces the waiting one), Shortcuts (feed the list into a notification/automation).
  • Backing: SessionStatus (CoreIdentifiers.swift:182-229) + TurnDisposition (~234239).
  • What: OpenSessionIntent(session:) with openAppWhenRun → reuse the existing nucleic://session/<id> route (RemoteStore.handleDeepLink ~112118). Nearly free given the deep-link infra already exists.
  • Where: Spotlight result tap, Siri ("open payment-flow"), a widget row, Shortcuts.

4.5 Answer an "Ask User Question"

  • What: AnswerQuestionIntent mapping to the AskUserQuestionCardView flow (AskUserQuestionCardView.swift ~48210). Because these are structured single/multi-select + free-text, they map cleanly onto App Intents parameter disambiguation and could even be voiced. Medium priority — richer UX than a plain approval, but the in-app card already handles it well.

4.6 Supporting entities & queries (shared infra for the above)

  • SessionEntity (id SessionID) with EntityQuery + a needsAttention EntityQuery variant.
  • ApprovalEntity (id ApprovalID) with a "pending approvals" EntityQuery.
  • ProjectEntity (id ProjectID) — enables "start a session in ProjectX" (control scope, §5).
  • All three ids are already Codable/Hashable/Identifiable (CoreIdentifiers.swift).

4.7 Control-scope intents (ship after scope=control lands)

UX_IOS.md §2 defers these; wire the intents but gate on RemoteStore.canControl:

  • StartSessionIntent(project:, prompt:, model?, effort?, useWorktree, auto?)RemoteStore.startChat(...) (~839846). Great "Hey Siri, start a Nucleic agent to …" entry.
  • IntegrateSessionIntent(session:, strategy:)RemoteStore.integrate(id:, mode:) (~870), with a IntegrationStrategy AppEnum (merge/rebase/squash).
  • InterruptSessionIntent, DiscardSessionIntent, ArchiveSessionIntentinterrupt/discard/ setArchived (~865873).

4.8 To-do capture & dispatch (a natural Siri/Shortcuts win)

  • CaptureTodoIntent(text:, project?)RemoteStore.captureTodo(...) (~849852). "Hey Siri, add a Nucleic to-do: fix the flaky payment test" — capture on the go, dispatch to an agent later.
  • DispatchTodoIntent(todo:)RemoteStore.dispatchTodo(...) (~855). TodoID/TodoStatus already exist (CoreIdentifiers.swift:99-119).

5. Opportunity catalog — macOS host (NucleicApp)

The Mac already has the richest UI, so App Intents here is about automation and voice, not filling a gap. Gate all of this behind the §3.2 packaging spike (SwiftPM executable → App Intents metadata extraction) — that's the prerequisite.

5.1 Start a session by voice / Shortcut (highest macOS value)

  • StartSessionIntent(project:, prompt:, model?, effort?, base?, useWorktree, auto?, autoShip?)AppStore.startChat() (AppStore.swift:2145) / SessionController.start(prompt:) (~614). Enables "Start a Claude agent in to " from Siri/Shortcuts/Action button, and lets power users script fleet launches. All parameters map to NewChatComposer fields (~509560).

5.2 Approve from the menu bar / Siri without foregrounding

  • AnswerApprovalIntentAppStore.respondToApproval(id:decision:by:) (AppStore.swift:7295). The Mac already has a menu-bar "needs you" surface (UX_MACOS.md §7); an intent lets Siri/Shortcuts answer low-risk approvals hands-free (respecting the §3.3 risk gate and the by responder label so multi-device attribution stays correct).

5.3 Integrate / ship a finished session

  • IntegrateSessionIntent(session:, strategy:)AppStore.integrateOpenSession(strategy) (AppStore.swift:4700) / SessionController.integrate(...) (~1226). IntegrationStrategy (WorktreeManager.swift:90-93) becomes an AppEnum. Note: interactive conflict handling (UX_MACOS.md §6) means the intent should return status and route to the app on conflict rather than resolve it headlessly.

5.4 Query intents & Spotlight for triage

  • DashboardStatsIntentAppStore.dashboardStats (AppStore.swift:5-28: projects, chats, activeChats, messages, tokens…). "How many agents are running?" / a Shortcut that posts a daily summary.
  • SessionsNeedingMeIntent (macOS twin of §4.3) over AppStore.summaries.
  • Spotlight indexing of sessions/projects via CoreSpotlight (not present today — agent confirmed no CSSearchableIndex usage) so a system search jumps to a session; pairs with an OpenSessionIntent.

5.5 Session control & config as Shortcut actions

Thin wrappers, useful for automations (e.g. "at 6pm, interrupt all running agents"): InterruptSessionIntent, ArchiveSessionIntent, SetModelIntent, SetEffortIntent, SetAutoIntent, SetAutoShipIntent → the corresponding SessionController setters (setModel ~989, setEffort ~1003, setAuto ~1029, setAutoShip ~1048, setArchived ~1134, interrupt ~974). Lower priority; batch behind AppShortcuts once the core intents ship.


6. Proposed App Intents surface (types)

A minimal, shared vocabulary that most intents above draw from:

Entities

  • SessionEntityid: SessionID, display title/project/status; EntityQuery + "needs me" query.
  • ApprovalEntityid: ApprovalID, title, risk; query over pending approvals.
  • ProjectEntityid: ProjectID, name, default branch.
  • TodoEntityid: TodoID, text, status (iOS-first).

Enums (AppEnum)

  • ApprovalDecisionOption — from Decision + AllowAlwaysScope (Approval.swift:12-61); destructive-safe subset.
  • RiskLevel — from Risk (Approval.swift:5-10) — drives inline-vs-open gating.
  • IntegrationStrategyOption — from IntegrationStrategy (WorktreeManager.swift:90-93).
  • BackendOption — from BackendID (CoreIdentifiers.swift:48-83).
  • EffortOption — low/medium/high/xhigh/max/orchestra (setEffort domain).

Intents (grouped by scope/priority)

  • approve-scope, iOS-first: AnswerApprovalIntent, SendFollowUpIntent, AnswerQuestionIntent, SessionsNeedingMeIntent, OpenSessionIntent, CaptureTodoIntent, DispatchTodoIntent.
  • control-scope / macOS: StartSessionIntent, IntegrateSessionIntent, InterruptSessionIntent, DiscardSessionIntent, ArchiveSessionIntent, config setters, DashboardStatsIntent.
  • NucleicShortcuts: AppShortcutsProvider — curates the few voice-worthy ones (approve, unblock, start, "what needs me").

All backing types are already Sendable/Codable (both agents confirmed), so AppEntity/AppEnum conformances are additive — no model refactor required.


7. Suggested phasing

  1. Phase 0 — de-risk (½–1 day each):
    • iOS: prototype AnswerApprovalIntent as a Button(intent:) inside the existing Live Activity (SessionLiveActivity.swift) for a low-risk approval. Proves the extension→host channel path under §3.1.
    • macOS: prove App Intents metadata extraction works for a SwiftPM .executable wrapped as an .app (§3.2). Do not scope macOS Shortcuts before this passes.
  2. Phase 1 — iOS approve-scope (highest ROI): AnswerApprovalIntent, SendFollowUpIntent, SessionsNeedingMeIntent, OpenSessionIntent, entities/enums, AppShortcuts. Wire into the interactive Live Activity + a new interactive widget. Reuse the notification approve/deny contract for the connectivity/queueing semantics.
  3. Phase 2 — iOS breadth: AnswerQuestionIntent, to-do intents, Control Center control, Spotlight indexing of sessions.
  4. Phase 3 — macOS (post-spike): StartSessionIntent, IntegrateSessionIntent, query intents, Spotlight, config-setter Shortcut actions.
  5. Phase 4 — control-scope on iOS: unlock start/integrate/discard/interrupt once scope=control ships (UX_IOS.md §2, open-Q). Intents already written in Phase 3 mostly port over.

8. Risks & open questions

  • macOS metadata extraction under SwiftPM (§3.2) — the gating unknown; spike first.
  • Extension-process connectivity (§3.1) — a widget/Siri intent may run when the host is unreachable. Decide per-intent: fail-clean-with-dialog vs. openAppWhenRun fallback vs. the existing ~30 s queue-on-reconnect used by respondFromNotification.
  • Risk gating parity (§3.3) — App Intents surfaces must reproduce the "no inline Allow on destructive/network/hostExec, biometric on open" rule exactly, or they'd become a softer approval path than the UI — a security regression. Encode it once in a shared helper both the UI and the intents call.
  • First-responder-wins (§3.4) — treat "already resolved elsewhere" as success, never an error.
  • Scope drift — keep every intent behind the same scope check the UI enforces (canControl / granted DeviceScope); never let an intent widen effective scope.
  • Attribution — thread a stable responder label (device name + identity fingerprint, as the UI does at AppStore.respondToApproval(... by:)) through intent-driven approvals so the audit trail stays truthful.
  • Discoverability vs. noise — curate AppShortcuts to the ~4 genuinely voice-worthy actions (approve, unblock, start, "what needs me"); expose the long tail through Shortcuts only.

Prepared from a code-level survey of Sources/NucleicApp, Sources/NucleicCore, Sources/NucleicProtocol, and ios/NucleicRemote. No App Intents code exists yet; all file:line anchors point at the actions and models an implementation would wrap.