#40

Helper handoffs: typed handoff and hand-back, never a dead end, answer cards, safe chips, owner assigns

Drafttobi-techywants to merge intofrom

Helper handoffs are now real: the API shows who Mimi asked, a helper that can't answer hands the message back, and nothing ends in a dead end.

Two problems from the live demo recording led to this:

  • "Who is the highest debtor?" got a dense Collector paragraph with no sign Mimi had handed it over.
  • "Any orders available now" went to Brand, which replied "For orders, the orders helper or your rep will help you with that." That's a dead end. The cause was the fast classifier: no phrase rule matched, so fast_non_money sent the owner's question to brand/promo-info.

The contract is in docs/HANDOFFS.md: field tables plus an example JSON for each of the five features. The OpenAPI file docs/api/openapi.json is regenerated. Field names are stable under contract: handoffs-2026-10-30.1.

What changed

  1. Handoff in the API. Every turn (POST /v1/turns, GET /v1/turns/{id}) now carries a typed handoff. It has from_helper: mimi, to_helper, a one-line reason written by code ("Asking Collector"), via, hop, steps with names and timings, and status. The status is working while the turn is queued or running. The thread also gets a handoff event before the first reply and a handoff_result event after it, so Mimi posts the result back on the thread. GET /v1/threads/{id} lists handoffs.
  2. Never a dead end.
    • Owner messages like "any orders / any new orders / show me the orders" now go to a new rule-only skill, order/open-orders. Stock questions like "what's in stock / stock levels" go to stock/low-stock-report. Both are picked by rule in router.OWNER_RULES, never by the model. Both are answered by code from the book with a list card, at no cost.
    • Shop messages like "is X available / what's in stock / can I order X / any noodles available / una get X for stock" are new order/order-intake phrase rules in lexicon.py. "Any promo available" still goes to Brand.
    • The claims gate now holds deflections (claims.deflections, problem kind deflection): "the X helper will help", "ask/contact your rep", "your rep can help you with that". "Ask your rep to add it to the order" and "your rep will confirm X with you" still pass.
    • Hand-back: if a helper still can't answer after its one repair, Mimi gets a typed handback and re-routes the message once. The target comes from an owner rule, then the phrase rules without that helper, then the words of the deflection itself. Money skills need the sender's own phrase evidence. The limit is one hop. After that, Mimi answers from the book when the skill has a card, or asks one clear question.
  3. Answer cards. Two kinds: a shop_row (shop id, name, owed, overdue, days late, credit limit, with remind and open_shop actions) and a list (items, qty, status, amount). Cards are owner-only. Every figure is summed in code from receivables and approvals. What code read is kept as a card_read thread event, so every figure can be traced. When a card is present the reply is one sentence. For a pure read that sentence is written by code. When the helper actually did work, its own first sentence is kept after the gate passed it.
  4. Chips. Each owner answer gets 2 or 3 chips with a label and an intent (turn or open). Chips are always tier: safe. chip_is_safe refuses approve, pay, transfer, refund and similar words. Money moves stay approval-only.
  5. The owner assigns a helper. This works with @Collector / @Orders ... at the start of a message, or with assign_to on POST /v1/turns. It only reaches that helper's owner skills and every policy tier still applies. assign_to from anyone but the owner gets a 422. A new GET /v1/tasks endpoint (owner only) returns per-helper task status (working, needs_you, done, handed_back, failed), with by_status counts.

Evals (offline, ₦0, no re-record)

EvalBefore (main)After
Router eval, 300 messages (cascade)98.67%, clarify 4.33%, 0 money misroutesunchanged: 98.67%, 4.33%, 0
Router eval, rules and phrases only90.67%, clarify 12.67%91.0%, clarify 12.33%
New router handoff cases (eval/router/handoffs.jsonl, 32 cases)14/32 correct, 6 sent to Brand, 3 would need the model32/32, 0 to Brand, 0 model calls
Claims-gate evalbad held 27/27, controls held 0/24bad held 30/30, controls held 0/26 (3 deflection cases + 2 controls added)
e2e replay (SQLite, 11 SIGKILLs)27/27 scenes, routing 100% over 43 turns29/29 scenes, routing 100% over 44 turns; recorded cost unchanged
Order held-out (and skills mode)46/4746/47
Skills catalogue / skills eval / per-skill evalspasspass (open-orders folder is a draft; 20 active skills unchanged)

Two scenes were added to the e2e:

  • "'Any orders available now' to Order, answered from the book (never Brand)". It is answered by code, so there's nothing to record.
  • "every finished turn carries a typed handoff from Mimi". This one passes on 44/44 turns.

What is not changed (and why)

  • Helper prompts, tool lists, the router prompt, the catalogue text and CATALOGUE_VERSION are untouched. order/open-orders is rule-only, so the model never sees it. Every recorded answer still replays.
  • BRAND_SKILL and ORDER_SKILL still tell the model to point off-topic questions at "the right helper or the rep". The gate and the hand-back now catch that in code. Rewriting those lines needs a paid re-record of the Brand and Order evals, so it's not in this PR.
  • An e2e scene for "Who is the highest debtor?" (a Collector model run plus its card) also needs a paid re-record, so it's not added. The card itself is covered by the unit tests.

Linear: RAI-284.

Activity

2
coderabbitai[bot]1d ago

IMPORTANT

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

This pull request is a draft

Merging is unavailable until it is marked ready for review.