Problem / Use Case
Providers ship new models every week, but OmniRoute's model list only updates when a new release comes out, every few weeks. Until then, stable users can't see models that already exist upstream, even when the fix is already merged.
Example from #17179: deepseek-v4.1-flash was missing on Ollama Cloud. The fix was merged (#16856), but users still had to wait for the next release to get it.
Most of these fixes are plain data: a new model name, a context window, a "supports reasoning" flag, a retired model. They don't need new code, yet today they wait for the full release cycle.
Proposed Solution
Publish the model list as a small signed data file that OmniRoute instances pick up on their own, separately from releases.
- For maintainers, nothing changes. Models are still edited in the existing registry through normal PRs. On merge, CI exports the model data, signs it, and publishes it on GitHub Releases.
- For users, new models show up within hours instead of at the next release, with no update and no restart.
- Safe by default. The model list shipped with the release stays the fallback. If the file is turned off, unreachable or invalid, OmniRoute behaves exactly as it does today. Users' own settings always win over the file.
- Locked down. The file can only describe models: names, limits and capabilities. It can never change provider URLs or authentication, so even a compromised file could not redirect anyone's API keys.
- No surprises on older versions. If a new model needs code that an installation doesn't have yet, that installation ignores it and shows "update required" instead of offering a model that would fail.
Most of the building blocks already exist. Radar already downloads, verifies and merges a signed free-model feed, and the model registry can already be refreshed in memory. This proposal applies the same approach to the whole catalog. It would stay separate from Radar: the catalog says what exists, Radar says what's free.
Alternatives Considered
- More frequent patch releases. These help, but every model fix would still need a full release.
- Third-party catalogs (models.dev, LiteLLM). These are already used for pricing and capabilities, but they don't carry OmniRoute's own fixes, such as retired models or provider-specific flags.
- Live discovery only. Not every provider has a model-listing endpoint, and when a parser breaks, discovery silently falls back to the outdated list.
Acceptance Criteria
- With the feature off or the file unreachable, the model list is identical to today's.
- A model added to the registry and merged appears on a running instance without an update or a restart.
- Invalid, tampered or older files are rejected.
- The file cannot carry provider URLs, authentication or headers.
- User overrides take precedence over the file.
- The dashboard shows where each model comes from (release / feed / local) and when the feed last synced.
- There is a setting to turn it off, and an environment variable as a kill switch.
Area
Provider Support, Proxy / Routing
Related Provider(s)
All (example: Ollama Cloud, #16823)
Additional Context
- Discussion: #17179
- Related: #8926, #16687, #16714
- I'm happy to help implement this if the direction is accepted.
Expected Test Plan
Unit tests cover:
- fallback to the bundled list when the file is missing;
- rejection of bad signatures, older revisions and forbidden fields;
- user overrides winning over the file;
- skipping models that need newer code.
An integration test publishes a new feed revision and checks that /v1/models shows the new model without a restart.
Activity
0No activity yet.