#52

Add support for custom providers and local models

Opendundalekwants to merge intofrom

Resolves #8 Alternative implementation to #36

By providing options to specify custom endpoint, we can cover both uses of using local models and using different providers. See updated README.md for details.

AI disclamer: Claude Code used

Activity

9
vercel[bot]7mo ago

@dundalek is attempting to deploy a commit to the Yury Selivanov's projects Team on Vercel.

A member of the Team first needs to authorize it.

lars200706mo ago

I would prefer the custom apiBase approach in this PR to #36. IMO, it is more future-proof.

lars20070reviewed6mo ago
lars20070reviewed6mo ago
dundalekreviewed6mo ago
dundalekreviewed6mo ago
dundalek6mo ago

As a note since creating the PR I discovered draft of llm uri scheme spec.

That would make setting it more convenient, instead of needing 4 env vars like:

LAT_LLM_BASE=https://openrouter.ai/api/v1
LAT_LLM_MODEL=qwen/qwen3-embedding-8b
LAT_LLM_DIMENSIONS=4096
LAT_LLM_KEY=sk-or-v1-xyz

We could use a single var like:

LAT_LLM_KEY=llm://:sk-or-v1-xyz@https://openrouter.ai/api/v1/qwen%2Fqwen3-embedding-8b?dimensions=4096

But I will wait on maintainer feedback before spending more time on this.