Feature Request: Add quality as a Routing Strategy in AI Gateway

This proposal adds a quality sort mode to AI Gateway's router, alongside the existing cost/ttft/tps options, so quality-sensitive workloads can use automatic routing without giving up output quality.

Summary

AI Gateway already supports routing decisions based on:

• cost
• ttft (time-to-first-token)
• tps (tokens per second)

There's currently no way to route based on output quality.

For quality-sensitive applications, this often forces developers to bypass the router and hardcode a specific provider/model. That means giving up gateway benefits that are already real and documented: automatic failover, provider order/fallback (providerOptions.gateway.order, providerTimeouts), and cost-based routing.

Adding a fourth routing option:

providerOptions: { gateway: { sort: "quality" } }

would let developers optimize for response quality the same way they already optimize for cost, latency, or throughput.

The Problem

All three are operational metrics. None answers: which provider is most likely to produce the best response for this request?

For many workloads — coding assistants, agentic workflows, legal review, contract analysis, requirements engineering, technical documentation, research — response quality matters more than speed or cost. Today these workloads are frequently locked to a single hardcoded model because the router has no quality-based mode, which reduces the value of the gateway abstraction for exactly the use cases where correctness matters most.

Proposed Solution

Add sort: "quality" to the existing routing options:

providerOptions: { gateway: { sort: "quality" } }

When selected, the gateway ranks candidate providers by a quality score and routes to the top-ranked one. Failover behavior is unchanged: if the top-ranked provider is unavailable, the router moves to the next-ranked one, same as it already does for cost/ttft/tps.

This is a non-breaking, additive change — no new endpoints, no change to existing sort values. (It does add one new accepted value to the sort field, which is the extension point this proposal relies on.)

Possible Quality Scoring Approaches

Option 1 — Static capability rankings. Benchmark-driven rankings maintained by the gateway across categories like coding, reasoning, summarization, data extraction, long-context analysis, and creative generation. Requests get matched to a category by explicit configuration or inference. This is the lighter implementation path.

Option 2 — Dynamic, outcome-based scoring. Build rankings from production telemetry: regeneration/retry rate, acceptance rate, evaluator/LLM-judge scoring, or explicit user feedback. The router improves from real traffic rather than a static benchmark.

These aren't mutually exclusive — a practical rollout could start with Option 1 and layer in Option 2 as telemetry accumulates. (Workspace-specific tuning is conceivable as a further step, but isn't part of this proposal — see Scope.)

Why This Fits AI Gateway

The routing framework already exists. sort: "cost" | "ttft" | "tps" is already a working mechanism; sort: "quality" extends the same abstraction rather than introducing a new one.

The telemetry and provider-order/failover pipeline already exists. AI Gateway already tracks per-request latency and throughput, and already supports provider order preference and timeout-triggered failover (providerOptions.gateway.order, providerTimeouts). A quality signal is a new field riding the same pipeline, not a new collection or failover mechanism.

Customer Impact

A coding assistant might prefer the provider that produces more consistently correct completions over the cheapest one. A legal-review agent might accept somewhat higher latency for stronger analysis. A requirements-analysis system might prefer the strongest reasoning model over the fastest response. Today these workloads often hardcode a provider to get that outcome — and lose automatic failover and multi-provider resilience in exchange. Quality-aware routing would let them keep both.

Precedent

Quality-aware routing is already standard among comparable platforms: GitHub Copilot Auto balances intelligence, quality, and efficiency across model tiers; Azure AI Foundry Model Router exposes explicit Cost / Quality / Balanced modes; AWS Bedrock Intelligent Prompt Routing uses a trained routing model to cut cost while holding quality steady; and OpenRouter's Auto Router routes by task-type capability rankings with its own Cost/Quality tradeoff setting. AI Gateway is the one comparable router without a quality dimension. Looking forwards to a future preferred state an AI Exchange would benefit most on a quality reassurance. Allowing the AI prompt request being routed through an Exchange and see AI as a commodity and tradeable assets.

Scope

This proposal is intentionally limited. It does not propose model marketplaces, spot pricing, auctions, futures markets, or capacity trading. It proposes one thing: adding a quality dimension to the existing AI Gateway router.

Expected Outcome

Adding sort: "quality" would let AI Gateway optimize for the metric many developers care about most — the quality of the answer — and close a feature gap relative to other model-routing platforms, without adding new endpoints or changing existing routing behavior.

A fourth routing mode, quality, would let developers use AI Gateway for quality-sensitive workloads without giving up the platform's multi-provider routing, failover, and cost/latency optimization. The existing routing architecture is already the right extension point for this capability.