#1342

libfx 0.0.14: with local() durability, a steer stored just before a tool returns misses the running turn

Openjagenaujagenauopened 26d ago

libfx: 0.0.14 and 0.0.14-dev.1561.g1a070af32cdf, native backend, Node 25 (macOS arm64). Works on 0.0.13-dev.1432.g1df91f0b4ebf.

What happens

With local() durability, session.steer() sent while a tool is running doesn't reach the running turn. The model request after the tool goes out without it, the turn ends with end_turn, and about a second later the steer runs as a turn of its own.

With memory() durability, the same code works: the turn emits user_message and the request after the tool carries the steer. 0.0.13-dev.1432 also works with local().

The steer() docs say it "waits for a running tool", so we expected the steer to join the turn.

On 0.0.14 the local() run below misses the turn every time (6 of 6 runs). On dev.1432 it joined the turn in 3 of 3.

Repro without a key

node steer.mjs local (or node steer.mjs for memory())

import { createFxAgent, memory } from "libfx"
import { local } from "libfx/durable-local"
import { mkdtempSync } from "node:fs"
import { tmpdir } from "node:os"
import { join } from "node:path"

const sse = (parts) => new Response(new ReadableStream({ start(c) {
  for (const p of parts) c.enqueue(new TextEncoder().encode(`data: ${JSON.stringify(p)}\n\n`)); c.close() } }),
  { headers: { "content-type": "text/event-stream" } })
const usage = { inputTokens: { total: 1, noCache: 1, cacheRead: 0, cacheWrite: 0 }, outputTokens: { total: 1, text: 1, reasoning: 0 } }
const text = (t) => sse([{ type: "stream-start", warnings: [] }, { type: "text-start", id: "t" }, { type: "text-delta", id: "t", delta: t },
  { type: "text-end", id: "t" }, { type: "finish", finishReason: { unified: "stop", raw: "end_turn" }, usage }])
let calls = 0
const requests = []
const fetch = async (url, init) => {
  if (!String(url).endsWith("/ai/language-model")) return new Response(JSON.stringify({ data: [] }))
  const body = JSON.parse(init.body)
  const steered = JSON.stringify(body.prompt ?? body).includes("Keep it short.")
  requests.push(steered ? "request with steer" : "request without steer")
  if (requests.length === 1)
    return sse([{ type: "stream-start", warnings: [] },
      { type: "tool-call", toolCallId: `call_${++calls}`, toolName: "wait", input: "{}" },
      { type: "finish", finishReason: { unified: "tool-calls", raw: "tool_use" }, usage }])
  return text(steered ? "saw the steer" : "no steer")
}

let release, started
const running = new Promise((r) => (started = r))
const agent = createFxAgent({
  apiKey: "k", model: "anthropic/m", fetch,
  durability: process.argv[2] === "local" ? local({ dir: mkdtempSync(join(tmpdir(), "steer-")) }) : memory(),
  tools: [{ name: "wait", description: "w", idempotent: true, inputSchema: { type: "object", properties: {} },
    execute: async () => { started(); await new Promise((r) => (release = r)); return "waited" } }]
})
const session = agent.session()
const turn = session.prompt("work")
const events = []; let answer = ""
const reading = (async () => { for await (const e of turn) { events.push(e.type); if (e.type === "text_delta") answer += e.delta } })()
await running
await session.steer("Keep it short.") // the tool is still running
release()
await reading
const result = await turn.result
console.log({ durability: process.argv[2] ?? "memory", events: events.join(" "), answer, stopReason: result.stopReason, requests })
await new Promise((r) => setTimeout(r, 1500))
console.log("1.5s later:", requests)
await agent.close()

0.0.14, local:

{
  durability: 'local',
  events: 'tool_start tool_end text_delta',
  answer: 'no steer',
  stopReason: 'end_turn',
  requests: [ 'request without steer', 'request without steer' ]
}
1.5s later: [ 'request without steer', 'request without steer', 'request with steer' ]

0.0.14, memory, and 0.0.13-dev.1432, local:

{
  events: 'tool_start tool_end user_message text_delta',
  answer: 'saw the steer',
  stopReason: 'end_turn',
  requests: [ 'request without steer', 'request with steer' ]
}

Expected

A steer sent while a tool runs is in the next model request of that turn, whatever the durability.

Activity

1
jagenaujagenau26d ago

A correction after narrowing it down: the steer is missed only when the tool ends right after steer() resolves. In the repro, release() runs immediately after await session.steer(...).

Delay between steer() resolving and the tool returning, with local() on 0.0.14:

delayresult
0 ms, 1 mssteer missed, runs as its own turn
5, 10, 20, 50 mssteer joins the turn
100–2200 mssteer joins the turn

So it's a small race: the turn picks up the steer slightly after steer() resolves, and a tool result that lands in between sends the next model request without it. memory() and 0.0.13-dev.1432 (local()) join the turn even at 0 ms.

In practice a person rarely steers within a few milliseconds of a tool finishing, so the impact is smaller than the title suggests. A steer that steer() has stored should still reach the turn's next request.