What happened?
The strict-tools fallback from #10409 works, but it isn't remembered. Every request first goes out with strict tools, gets the 400, then retries without strict:
400 invalid_request_error: The compiled grammar is too large, which would cause performance issues. Simplify your tool schemas or reduce the number of strict tools.
The tool set doesn't change between turns, so after the first rejection every later turn hits the same 400. I ran a workflow with about a dozen subagents on claude-opus-5-5. Each one had read/bash/edit/write plus a StructuredOutput tool (an array of objects with two enums) as its strict tools. Nearly every assistant turn in those sessions has an anthropic_strict_tools_fallback diagnostic, up to 65 per session. That's two requests per turn and a few hundred 400s in my proxy logs in ten minutes.
The cause is that disablePreferredStrictTools in anthropic-messages.ts is local to a single stream() call.
Steps to reproduce
- Register enough tools with
constrainedSampling: { type: "json_schema", strict: "prefer" }that Anthropic rejects the grammar (the setup from #10409 works). - Run a few turns.
- Every turn's assistant message has an
anthropic_strict_tools_fallbackdiagnostic, and every turn makes two requests.
Expected behavior
Once a strict tool set has been rejected for a model, later requests with the same strict schemas should go out without strict right away. A small bounded cache keyed by model plus a hash of the strict tool schemas would do. The Bedrock fallback (842c7317) has the same per-request behavior.
I can open a PR for this.
Version
1.4.0
AI disclosure: this text was drafted by an AI assistant from my session logs and the pi-ai source.
Activity
0No activity yet.