#11831

durable: steerRestart option so a steer can restart an in-flight generation

OpenSoju06opened 3d ago

What do you want to change?

Add HarnessSettings.steerRestart: "never" | "beforeOutput" | "beforeTools" (default never) to @earendil-works/pi-durable. When a steer arrives while the generation is still in prepare or request (no tool round running), the running request is aborted and the run is handed to a successor generation that sees both user messages in one request.

Why?

Today a steer (whenBusy: "steer") joins only at a postTools or final boundary. A turn that answers with text alone has no tool boundary, so a second user message sent 400 ms after the first waits for the whole answer and is then answered in a run of its own. For a chat product this is a queue, and the model never sees the correction while it still matters ("book it" -> "no, don't").

Trace on main with the faux model (first token 1.5 s): A at t0, B at t0+0.4 s -> A answered at 2.0 s, B answered at 4.0 s in a second request. With beforeOutput: one request, both settle done at 2.4 s.

How? (optional)

Reuse the existing abort mark: at steer admission write the mark when the generation task is running in prepare|request; in the generation's abort handler, when the run carries the steer, convert the partial to an aborted entry (outside the context), place the steer at a postTools boundary and hand the run to a fresh generation. Tool rounds, retry backoff and compaction waits keep today's behaviour. beforeOutput never discards visible text; beforeTools also restarts while answer text or tool arguments are streaming.

I have this implemented with 7 faux-provider tests (durable harness + conversation suites 949/949, npm run check clean) on a fork branch and would open the PR after lgtm.

Drafted with an agent, reviewed and submitted by me.

Activity

0

No activity yet.