#40
Helper handoffs: typed handoff and hand-back, never a dead end, answer cards, safe chips, owner assigns
Helper handoffs are now real: the API shows who Mimi asked, a helper that can't answer hands the message back, and nothing ends in a dead end.
Two problems from the live demo recording led to this:
- "Who is the highest debtor?" got a dense Collector paragraph with no sign Mimi had handed it over.
- "Any orders available now" went to Brand, which replied "For orders, the orders helper or your rep will help you with that." That's a dead end. The cause was the fast classifier: no phrase rule matched, so
fast_non_moneysent the owner's question tobrand/promo-info.
The contract is in docs/HANDOFFS.md: field tables plus an example JSON for each of the five features. The OpenAPI file docs/api/openapi.json is regenerated. Field names are stable under contract: handoffs-2026-10-30.1.
What changed
- Handoff in the API. Every turn (
POST /v1/turns,GET /v1/turns/{id}) now carries a typedhandoff. It hasfrom_helper: mimi,to_helper, a one-linereasonwritten by code ("Asking Collector"),via,hop,stepswith names and timings, andstatus. The status isworkingwhile the turn is queued or running. The thread also gets ahandoffevent before the first reply and ahandoff_resultevent after it, so Mimi posts the result back on the thread.GET /v1/threads/{id}listshandoffs. - Never a dead end.
- Owner messages like "any orders / any new orders / show me the orders" now go to a new rule-only skill,
order/open-orders. Stock questions like "what's in stock / stock levels" go tostock/low-stock-report. Both are picked by rule inrouter.OWNER_RULES, never by the model. Both are answered by code from the book with a list card, at no cost. - Shop messages like "is X available / what's in stock / can I order X / any noodles available / una get X for stock" are new
order/order-intakephrase rules inlexicon.py. "Any promo available" still goes to Brand. - The claims gate now holds deflections (
claims.deflections, problem kinddeflection): "the X helper will help", "ask/contact your rep", "your rep can help you with that". "Ask your rep to add it to the order" and "your rep will confirm X with you" still pass. - Hand-back: if a helper still can't answer after its one repair, Mimi gets a typed
handbackand re-routes the message once. The target comes from an owner rule, then the phrase rules without that helper, then the words of the deflection itself. Money skills need the sender's own phrase evidence. The limit is one hop. After that, Mimi answers from the book when the skill has a card, or asks one clear question.
- Owner messages like "any orders / any new orders / show me the orders" now go to a new rule-only skill,
- Answer cards. Two kinds: a
shop_row(shop id, name, owed, overdue, days late, credit limit, withremindandopen_shopactions) and alist(items, qty, status, amount). Cards are owner-only. Every figure is summed in code from receivables and approvals. What code read is kept as acard_readthread event, so every figure can be traced. When a card is present the reply is one sentence. For a pure read that sentence is written by code. When the helper actually did work, its own first sentence is kept after the gate passed it. - Chips. Each owner answer gets 2 or 3 chips with a label and an intent (
turnoropen). Chips are alwaystier: safe.chip_is_saferefuses approve, pay, transfer, refund and similar words. Money moves stay approval-only. - The owner assigns a helper. This works with
@Collector/@Orders ...at the start of a message, or withassign_toonPOST /v1/turns. It only reaches that helper's owner skills and every policy tier still applies.assign_tofrom anyone but the owner gets a 422. A newGET /v1/tasksendpoint (owner only) returns per-helper task status (working,needs_you,done,handed_back,failed), withby_statuscounts.
Evals (offline, ₦0, no re-record)
| Eval | Before (main) | After |
|---|---|---|
| Router eval, 300 messages (cascade) | 98.67%, clarify 4.33%, 0 money misroutes | unchanged: 98.67%, 4.33%, 0 |
| Router eval, rules and phrases only | 90.67%, clarify 12.67% | 91.0%, clarify 12.33% |
New router handoff cases (eval/router/handoffs.jsonl, 32 cases) | 14/32 correct, 6 sent to Brand, 3 would need the model | 32/32, 0 to Brand, 0 model calls |
| Claims-gate eval | bad held 27/27, controls held 0/24 | bad held 30/30, controls held 0/26 (3 deflection cases + 2 controls added) |
| e2e replay (SQLite, 11 SIGKILLs) | 27/27 scenes, routing 100% over 43 turns | 29/29 scenes, routing 100% over 44 turns; recorded cost unchanged |
| Order held-out (and skills mode) | 46/47 | 46/47 |
| Skills catalogue / skills eval / per-skill evals | pass | pass (open-orders folder is a draft; 20 active skills unchanged) |
Two scenes were added to the e2e:
- "'Any orders available now' to Order, answered from the book (never Brand)". It is answered by code, so there's nothing to record.
- "every finished turn carries a typed handoff from Mimi". This one passes on 44/44 turns.
What is not changed (and why)
- Helper prompts, tool lists, the router prompt, the catalogue text and
CATALOGUE_VERSIONare untouched.order/open-ordersis rule-only, so the model never sees it. Every recorded answer still replays. BRAND_SKILLandORDER_SKILLstill tell the model to point off-topic questions at "the right helper or the rep". The gate and the hand-back now catch that in code. Rewriting those lines needs a paid re-record of the Brand and Order evals, so it's not in this PR.- An e2e scene for "Who is the highest debtor?" (a Collector model run plus its card) also needs a paid re-record, so it's not added. The card itself is covered by the unit tests.
Linear: RAI-284.
Activity
2This pull request is a draft
Merging is unavailable until it is marked ready for review.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
❤️ Share
Comment
@coderabbitai helpto get the list of available commands.