#343

Let the agent hide the Live Activity during screenshot and screen recording steps

OpenZhehanZhangopened 8d ago

What you want to do

Users report that the Live Activity (Dynamic Island step updates, approval and permission cards) gets in the way of tasks whose result is an on-device screenshot or screen recording: the island content ends up in the capture. The agent should be able to ask Poki to take the Live Activity away right before such a step (after any approval on the island is answered), and bring it back after that step, or after a string of steps the owner already approved.

What happens today

Observed in the code (main at 296325d, Nov 14 2026):

  • There is one Live Activity per session, run by native/ios-app/Poki/Activity/ActivityController.swift. It is started with Activity.request(..., pushType: nil) (line 109), so there is no push token and no push-to-start.
  • Removing it is already supported and quick: end(dismissalPolicy:) (line 145) and the private hide() (line 165) call activity.end(nil, dismissalPolicy: .immediate). Nothing exposes this to the agent or the MCP.
  • Bringing it back only works while Poki is in front. start(deviceName:) documents this, and update(_:) tries to restart it whenever work resumes, but canStart (line 80) and hidesWhenIdle (line 78) are never assigned in AppModel, so from the background the request fails with ActivityKit's visibility error and only lastError records it.
  • With no activity, things that need the owner (hand-off, problem) fall back to a notification banner (AttentionNotice.post, line 194). Approvals have their own notification.
  • On iOS 26 and later, Poki also holds a BGContinuedProcessingTask (Runtime/ContinuedRun.swift) whose system progress UI is separate from the Live Activity. Whether that UI appears in the island during a run, and so in captures, is not yet checked on a phone.

What iOS allows (sources below):

  1. An app can end its own Live Activity at any time, from the background too, and .immediate removes it from the Lock Screen and the island at once (time not yet measured on a phone).
  2. A local Activity.request from the background fails with ActivityAuthorizationError.visibility. Apple's documented exceptions are a LiveActivityIntent (the intent must be run by the person or the system, for example from a widget button, a control, Siri or Shortcuts) and ActivityKit push-to-start (iOS 17.2 and later). Push needs the Push Notifications capability, which free Apple accounts cannot sign, and push-to-start has its own fixed on-device budget.
  3. iOS 26 adds a scheduled start (request(..., alertConfiguration:, start:)) that the system performs at a date even while the app is in the background, but it requires an alert configuration, which expands the island. Whether scheduling itself is allowed from the background is unverified.
  4. A capture of the screen shows the island only when it has content. With no Live Activity, a screenshot shows wallpaper there. A screen recording still shows the red recording indicator in the island; an app cannot remove it.

So hiding is easy and fast, and coming back mid-task from the background is the hard part.

Area

Poki iPhone app or Poki Setup

Proposal

Recommended design (not built, nothing tested on a phone):

  1. Agent request. A new MCP action, for example live_activity with mode: "hide" and until: "next_step" | "steps:<n>" | "show", or a hide_live_activity flag on the action that captures. The tool answers only after iOS reports the activity ended, plus a short settle delay, so the next capture is clean.
  2. Order with approvals. If the step needs approval, the approval card runs first on the island as today. Hide only after the owner's answer. For a string of steps, the owner approves the whole string once (one approval that names the N steps), then the activity hides for all of them, since a new approval during the hidden window cannot use the island.
  3. While hidden. The controller keeps the state current but sends nothing, keeps the activity wanted, and logs "hidden by the agent". Anything that needs the owner (a new approval, hand-off, problem) uses the existing notification path. Note that a banner also appears in captures, so the agent should be told the window was interrupted.
  4. Coming back.
    • If Poki comes to the front (the owner opens it, the run ends in Poki), restart at once, as today.
    • Otherwise, after the hidden window, the MCP answers live_activity: "hidden" with a reason, and the activity returns the next time Poki is in front. An opt-in "return hop" (open Poki for a moment to call Activity.request, then go back to the previous app) is possible but disruptive, so it should be off by default and never run during a capture.
    • On iOS 26 and later, test whether the scheduled start works from the background. If it does, it is a candidate for bringing the activity back without the hop, with the alert's island expansion timed so it never overlaps an input (per the rule that no island alert may coincide with an action).
  5. Fix the background restart attempt. Set canStart from the app's scene state so update(_:) stops calling Activity.request from the background and failing.

Alternatives considered:

  • Keep the activity but show an empty state. The island still shows a black pill around the camera, which captures still include. Not a real hide.
  • Pause updates. Stops changes but the island stays visible. Does not help.
  • Swiping the island away. iOS lets the person swipe horizontally on the island to minimize an activity, but automation must never touch the Live Activity, so the runner cannot use this.

To check on a phone before building: time from end(.immediate) to the island clearing; whether Poki's own screenshots and the system screenshot both stop showing the island after the end; whether the iOS 26 continued processing UI shows in the island; and whether the iOS 26 scheduled start can be requested from the background.

Sources:

Owner's direction (Nov 14 2026, 2:28 am ET)

The owner chose a simpler, one-way switch instead of hiding and restoring per step:

An in-task and a default switch, which can be started by the model or configured by the user in settings, to turn the Dynamic Island off temporarily until task completion and the Poki app reopens. In the meantime all permission requests are delivered in notifications; no step information is provided in notifications.

Feasibility check against the code (main at 296325d):

  • Turning off mid-task: feasible. end(dismissalPolicy: .immediate) works from the background. The switch needs a flag in ActivityController (for example offUntilReopen) that calls hide(), makes update(_:) skip its restart, and makes send drop updates while the state keeps being tracked.
  • Coming back on reopen: feasible, and it matches what iOS allows. The flag clears when the task has finished (the run reaches done, finishNow, or the session stops) and Poki's scene becomes active (PokiApp.swift already watches scenePhase). Then start(deviceName:) runs in the foreground as today. If the owner opens Poki before the task ends, the island stays off until the task finishes, and comes back on the first foreground after that.
  • Approvals in notifications: already in place. AppModel.askOwner always posts ApprovalNotice (time-sensitive, Allow with Face ID and Don't Allow on the notification) next to the Live Activity card. With the switch on, only the notification is used. Its text is the rule's question, not the agent's words, so no step information. The hand-off and problem banners (AttentionNotice) carry the hand-off title or the problem, not steps, and the finished-task notification stays as it is.
  • No step information anywhere while off: step updates only reach the Live Activity today, so dropping them in the controller is enough. Nothing new goes to notifications.

Proposed pieces:

  1. Setting (default switch). Settings › Live Activity: "Dynamic Island During Tasks" with Show (default) and Off Until Poki Opens. With Off, a task never starts or keeps a Live Activity. The pairing code and setup states keep using it, since Poki is in front then.
  2. Model switch (in-task). An MCP tool, for example live_activity_off, with an optional reason that is logged in the Activity Log. It answers once iOS reports the activity ended, so the next capture is clean, and it tells the agent that this is one-way for the task: the island comes back only after the task finishes and the owner opens Poki. Poki's own chat agent gets the same tool.
  3. Logging. "Live Activity off until Poki opens (model: )" and "Live Activity back after reopen" in the Activity Log.
  4. Fix the background restart attempt. Set canStart from the scene state so update(_:) stops failing Activity.request calls from the background (also needed so the switch can't be undone by mistake).

Limits to state in the docs:

  • A time-sensitive approval banner still shows on screen, so it can appear in a recording that is running when the approval comes. The runner sends no input while an approval waits.
  • On iOS 26 and later, the continued processing task's own system UI ("Poki is on", "Agents can reach this iPhone", no steps) is separate from the Live Activity. If it shows in the island, the switch does not remove it, because ending it would let iOS suspend Poki. Still to check on a phone.
  • Screen recordings always show iOS's red recording indicator.

Activity

4
ScottH11227d ago

Thanks for the detailed lifecycle analysis and the clear one-way-switch direction. During the open-issue audit I confirmed that canStart was still unwired and that direct start callers could also bypass it. I opened #374 for that independent part: creation now checks Poki's foreground state and UIApplication's active state, while keeping the deferred activity wanted and continuing to track work state.

All 18 selected Live Activity and language regressions pass on the iOS 27.3 Simulator, including the new background-deferral regression. The app and widget compile. Initial local setup failures (missing launcher archive, then unsigned Keychain access) are documented in the PR; the final run uses the repository's ad-hoc Simulator signing settings. All four PR checks also pass.

This does not implement the default/in-task hide switch, the model tool or its completion/reopen latch, and it does not remove iOS's separate continued-processing or recording indicators. Please keep #343 open for that feature and physical-device capture/foreground acceptance. Thank you for spelling out these boundaries; I am happy to keep working through review and the remaining integration work.

JosssphZhou7d ago

Thanks for the follow-up and for keeping the scope clear.

#374 (create the Live Activity only while Poki is in front) is approved on review. It is queued for a real phone check before merging: no island when a task starts in the background, one island when Poki returns to the front, and none after Stop.

This issue stays open for the rest: the setting and the tool the agent can call to turn the island off until the task ends and Poki is reopened, and the phone check of whether screenshots and recordings still show the island. The switch is the follow-up work.

JosssphZhou6d ago

Scott would like to take this one, so here is the scope we have in mind. Please push back on anything that looks wrong once you're in the code.

Now that #374 is in 0.4.2, what's left is a one-way switch:

  1. A tool the agent can call before a screenshot or recording step, for example hide_live_activity. It ends the Live Activity right away and keeps it off until the current task ends and the owner opens Poki again. There is no tool to bring it back, so the agent can't switch the island on and off during a task.
  2. It never hides a pending decision. If an approval or hand-off is waiting on the island, the call fails with a clear message and the agent has to wait for the answer first. While the island is off, anything that needs the owner falls back to the existing notification banner.
  3. A setting in Poki that lets the owner turn this off, so the agent can't hide the island at all. On by default.
  4. Tests: a regression for each rule above, including one that fails on today's main.
  5. Phone check: on a real iPhone, a screenshot and a short screen recording taken right after the call show no island, and the island comes back after Stop and reopening Poki.

Smaller PRs are welcome, for example the tool and its rules first, then the setting. Does this match what you had in mind?

ScottH11226d ago

Thanks for laying out the scope, Joseph. Yes, this matches what I had in mind, and I’d be happy to take it on now that #374 is in 0.4.2.

I’ll start by checking the task lifecycle and approval paths on current main. The hide action should be one-way for the current task: it should refuse while an approval or hand-off is pending, and any new request for the owner should still surface through the existing notification path. I’ll also check how the owner’s opt-out setting and the Stop/reopen behavior fit together, and flag any questions here before settling the implementation.

I’ll cover the lifecycle rules with regressions and verify screenshots, a short recording, and restoration after Stop and reopening on a physical iPhone. I’ll distinguish Poki’s Live Activity from iOS’s own capture indicators in the test notes.