#513

feat(ios): show connecting and failed voice states on the island and play a ready cue

OpenJosssphZhouwants to merge intofrom

Summary of changes

  • Extend PokiState.Voice with .connecting and .failed so the Live Activity and Dynamic Island reflect the real lifecycle of a voice session instead of showing Listening (聆听中) before the Realtime session is established.
  • While a voice session is exchanging its ephemeral token, negotiating WebRTC, and waiting for the Realtime API's session.created event, display Connecting (连接中 / 連線中) with the waiting bear pose (.thinking) and a quiet compact waveform on the Dynamic Island, Lock Screen, and in-app TalkBar (including background starts from TalkHereIntent).
  • Transition to Listening (聆听中) and trigger VoiceReadyCue at the exact moment VoiceModel receives session.created from the Realtime API (wasConnecting && holdingMicrophone), never by a fixed delay:
    • Synthesize a short, soft two-note PCM chime (180 ms, 587 Hz to 880 Hz with gentle attack and exponential decay) and play it as a UI system sound (kAudioServicesPropertyIsUISound = 1), which respects the iPhone hardware Silent Switch.
    • When the completion callback reports that iOS muted the sound in silent mode (elapsed < 0.05 s), trigger a single light impact vibration (UIImpactFeedbackGenerator(style: .light)) instead.
  • When connecting fails or the connection drops, keep the Live Activity visible in the .failed voice state (Connection failed / 连接失败 / 連線失敗, subtitle Check the network, then try again. / 请检查网络后重试。, bear pose .tripped) with a Try Again (重试 / 重試) button (RetryTalkIntent over Darwin notification TalkSignal.retry) that retries voice.start() directly from the Dynamic Island or Lock Screen.
  • Treat state.voice != old.voice as critical in ActivityController.isCritical so voice phase transitions update the Dynamic Island immediately without waiting for the 700 ms progress coalescing window.
  • Coordination with PR #508 (fix/island-listening-clipped): PR #508 adjusts the expanded island's bottom clip and ActivityMetrics.sizes for face.endsTalk voice states, while this PR adds .connecting, .failed, RetryTalkButton, and VoiceReadyCue without duplicating layout sizing changes. .connecting sets face.endsTalk == true (so it automatically inherits PR #508's 142 pt expanded-island height), .failed uses .tall with .retryTalk, and both branches merge cleanly in either order (git merge-tree verified with zero conflicts).

Motivation

Starting a live voice session requires an ephemeral token exchange, WebRTC ICE/DTLS negotiation, and waiting for OpenAI's session.created event. Previously, TalkActivity.state(of:) mapped .connecting to .listening (聆听中) and TalkActivity.startHere() displayed .listening before voice.start() began. As a result, the Dynamic Island immediately read Listening during cold start while early speech was still dropped, and a failed connection disappeared from the Dynamic Island instead of offering a retry action.

Test plan

  • Built and ran PokiTests on the iPhone 16 Pro simulator (0FFFE228-FF90-4E6E-B6A2-06D8C5B84A3A):
    • VoiceModelTests: 26 tests executed, 0 failures (verifying that TalkActivity shows .connecting before session.created, switches to .listening and fires readyCue strictly on session.created, does not fire readyCue on failure or silent test runs, transitions to .failed on connection failure, and retries via TalkSignal.retry).
    • LiveActivityTests: 17 tests executed, 0 failures (verifying ActivityFace and localized strings for .connecting and .failed across en, zh-Hans, and zh-Hant).
    • LanguageTests: 12 tests executed, 0 failures.
  • Verified on the iPhone 16 Pro simulator (nz.zheha.poki -uiLanguage zh-Hans -pokiTalkRehearsal 1,offline) with screen recording and poki-log.txt inspection:
    • Tapping the voice button shows 连接中 with the waiting bear on both TalkBar and the Dynamic Island, then switches to 聆听中 and logs [voice] session created, session 1, [voice] ready cue triggered, and [voice] ready cue played at the same timestamp when session.created arrives.
    • Under offline failure (URLError.notConnectedToInternet), the Dynamic Island transitions from 连接中 to 连接失败 with the 重试 button.

Screenshots or screen recording

Recorded on the iPhone 16 Pro simulator showing the transition from 连接中 (waiting bear) to 聆听中 upon session.created and from 连接中 to 连接失败 with 重试 when offline.

AI assistance: Other. I have reviewed and understand these changes.

  • I removed passcodes, tokens, and private screen contents from this PR.
  • I release this contribution under the AGPL-3.0-only license.