#98
feat: local prompt caching (opt-in disk cache + ai cache commands)
Repeated prompts hit the Gateway every time. Running git diff | ai text "summarize" or ai image "a logo" twice pays twice, even though the result is the same. There is no local cache today.
Propose an opt-in disk cache.
- Off by default. Enable per-run with `--cache` or globally with `AI_CLI_CACHE=1`. Disable with `--no-cache` when fresh output matters.
- Cache key is model + prompt + system prompt + image references + relevant options (temperature, max tokens, etc.). Different models or options do not collide.
- Stored under `~/.cache/ai-cli` (override with `AI_CLI_CACHE_DIR` / `XDG_CACHE_HOME`). Include `AI_CLI_CACHE_TTL` to control expiry, default 7 days.
- Add `ai cache` helpers: `ai cache status`, `ai cache clear`, `ai cache prune --max-size 500M`.
Example:
```bash git diff | ai text "summarize these changes" --cache
first run: calls Gateway
git diff | ai text "summarize these changes" --cache
second run: Cache hit for openai/gpt-5.5 (0.02s)
```
Applies to `text`, `image`, `video`, `audio`, and `evaluate`. Failures are not cached. Works with `count` and multi-model runs.
I have a fork ready (`moah0911/ai-cli` branch `feat/prompt-cache`) and can open a PR if you approve the approach.
Activity
0No activity yet.