#52
Add support for custom providers and local models
Resolves #8 Alternative implementation to #36
By providing options to specify custom endpoint, we can cover both uses of using local models and using different providers. See updated README.md for details.
AI disclamer: Claude Code used
Activity
9I would prefer the custom apiBase approach in this PR to #36. IMO, it is more future-proof.
lars20070reviewed6mo ago
lars20070reviewed6mo ago
dundalekreviewed6mo ago
dundalekreviewed6mo ago
As a note since creating the PR I discovered draft of llm uri scheme spec.
That would make setting it more convenient, instead of needing 4 env vars like:
LAT_LLM_BASE=https://openrouter.ai/api/v1
LAT_LLM_MODEL=qwen/qwen3-embedding-8b
LAT_LLM_DIMENSIONS=4096
LAT_LLM_KEY=sk-or-v1-xyzWe could use a single var like:
LAT_LLM_KEY=llm://:sk-or-v1-xyz@https://openrouter.ai/api/v1/qwen%2Fqwen3-embedding-8b?dimensions=4096
But I will wait on maintainer feedback before spending more time on this.
@dundalek is attempting to deploy a commit to the Yury Selivanov's projects Team on Vercel.
A member of the Team first needs to authorize it.