#41

feat: Gemini embedding provider + LAT_LLM_ENDPOINT for custom servers

Openkgray-wasteologywants to merge intofrom

Closes #40

Summary

  • Gemini provider: Auto-detects AIza key prefix → routes to Google's OpenAI-compatible embedding endpoint (gemini-embedding-001, free tier)
  • LAT_LLM_ENDPOINT: Generic escape hatch for any OpenAI-compatible embedding server (Ollama, LM Studio, vLLM). Takes highest priority when set. Pair with LAT_LLM_MODEL to override model name.
  • dimensions in request body: Ensures providers that default to higher dimensions (Gemini → 3072) truncate to the expected 1536. Also correct for OpenAI.

Changes

FileChange
src/search/provider.tsAdd gemini provider, customProvider() function, LAT_LLM_ENDPOINT/LAT_LLM_MODEL env var support in detectProvider()
src/search/embeddings.tsPass dimensions in embedding request body
tests/search.test.ts6 new tests: Gemini detection, endpoint override, trailing slash handling, custom provider

Design notes

  • Complements #36 — this handles the "custom API endpoint" tier, #36 handles "bundled local inference"
  • Follows @jeffmm's proposed priority chain from #36 discussion
  • Gemini uses Google's OpenAI-compatible endpoint, so no format changes needed
  • 91 additions, 1 deletion. All existing tests pass.

Test plan

  • npm run typecheck passes
  • npx vitest run tests/search.test.ts — all 14 tests pass (8 existing + 6 new)
  • Full test suite: 158 passed, 2 pre-existing pnpm not found failures unchanged
  • Manually verified Gemini semantic search works with LAT_LLM_KEY=$GEMINI_API_KEY lat search "query"

Activity

3
vercel[bot]7mo ago

@kgray-wasteology is attempting to deploy a commit to the Yury Selivanov's projects Team on Vercel.

A member of the Team first needs to authorize it.

grumdreviewed6mo ago