#573
Issues/phaseo/Phaseo/#573

Support explicit cache pricing separately from implicit cache pricing

OpenDanielButler1opened 5mo ago

Context

We are intentionally shipping Alibaba Cloud implicit cache pricing first and deferring explicit cache pricing support.

The current catalog now models implicit cache pricing for qwen/qwen3.7-plus, but does not attempt to represent explicit cache creation/read prices with first-class semantics.

Follow-up

Introduce explicit cache pricing support in a way that is unambiguous across catalog, API, and SDK surfaces.

Proposed scope

  • Define first-class explicit cache pricing meters instead of overloading generic cache fields.
  • Decide how explicit cache read vs explicit cache write should surface in control-model pricing responses.
  • Audit existing providers using cache pricing to classify which entries are implicit-only vs explicit cache pricing.
  • Backfill Alibaba Cloud explicit cache pricing for qwen/qwen3.7-plus once the meter model is settled.
  • Revisit existing ambiguous cache-priced cards such as qwen/qwen3.7-max and qwen/qwen3.6-max-preview.

Acceptance criteria

  • Explicit cache pricing can be represented distinctly from implicit cached-input pricing.
  • Control/API/OpenAPI/SDK surfaces are consistent.
  • Alibaba explicit cache prices can be added without ambiguity.

Activity

2
linear-code[bot]5mo ago

AI-105

opencode-agent[bot]5mo ago

We're super sorry about this issue you're facing! We'll have someone look at this as soon as possible.

Issue understanding

The catalog currently uses generic cached_read_text_tokens / cached_write_text_tokens meters for any cache pricing. This conflates two fundamentally different patterns:

  • Implicit cache (Google, Groq, CrofAI): cache read price bundles all cache costs (write+storage+read); no separate write meter; cached tokens are a subset of input tokens.
  • Explicit cache (Alibaba Qwen 3.7 Max, Anthropic): cache creation and cache reads are separately billable operations with distinct prices; write is a separate meter; cached tokens are not a subset.

There is no type-level distinction between these semantics, so consumers (API responses, SDKs, billing) cannot tell whether a cached_read_text_tokens entry is an implicit discount or a first-class explicit cache product. The request is to introduce first-class semantics so explicit cache creation/read prices are unambiguous in the catalog, API (CompatibilityPricing, PricingSummary), OpenAPI spec, and both SDKs, then backfill qwen/qwen3.7-plus explicit cache pricing and re-audit ambiguous entries like qwen/qwen3.7-max and qwen/qwen3.6-max-preview.

Key files: packages/data/catalog/src/data/pricing/, apps/api/src/routes/v1/control/models.ts (CompatibilityPricing), apps/api/src/routes/v1/control/models.catalogue.ts (PricingSummary/PRICING_METERS), apps/api/src/pipeline/pricing/types.ts (PricingDimensionKey), apps/api/src/pipeline/pricing/engine.ts (subset logic), apps/docs/openapi/v1/openapi.yaml (pricing schema), packages/sdk/sdk-ts/src/oapi-gen/models/Model.ts, packages/sdk/sdk-py/src/gen/models.py.

AI Agent Prompt (Copy/Paste)

Implement first-class explicit cache pricing semantics across the AI Stats catalog, API, OpenAPI spec, and SDK surfaces.

**Bug:** The catalog currently uses generic `cached_read_text_tokens` / `cached_write_text_tokens` meters for all cache pricing, with no way to distinguish implicit cache (discounted reads that are a subset of input tokens, with cache write bundled) from explicit cache (separately billable cache create + read operations). This makes it impossible to correctly model providers like Alibaba Cloud that offer explicit cache as a first-class product (e.g., `qwen/qwen3.7-plus`).

**Likely area:** `packages/data/catalog/`, `apps/api/src/routes/v1/control/models.ts` (CompatibilityPricing), `apps/api/src/routes/v1/control/models.catalogue.ts` (PricingSummary), `apps/api/src/pipeline/pricing/types.ts` (PricingDimensionKey), `apps/api/src/pipeline/pricing/engine.ts` (subset logic), `apps/docs/openapi/v1/openapi.yaml`, `packages/sdk/sdk-ts/src/oapi-gen/models/Model.ts`, `packages/sdk/sdk-py/src/gen/models.py`.

**Suggested fix direction:**
1. Introduce new `PricingDimensionKey` values for explicit cache (e.g., `explicit_cache_write_text_tokens`, `explicit_cache_read_text_tokens`) alongside existing implicit cache meters.
2. Add an `explicit` boolean or enum field (`cache_mode: "implicit" | "explicit" | null`) to the catalog `pricing.json` rules so providers can tag cache entries.
3. Extend `CompatibilityPricing` in `models.ts` to include `explicit_cache_write` / `explicit_cache_read` fields (or add a `cache_mode` discriminator) and expose them in the control API response and OpenAPI spec.
4. Update `PRICING_METERS` in `models.catalogue.ts` and `CORE_PRICING_METER_VALUES` in `apps/web/src/lib/pricing/meters.ts` to include the new meters.
5. Update the TypeScript SDK generated types and Python SDK `gen/models.py`.
6. Regenerate OpenAPI client code (`pnpm openapi:gen`).
7. Audit all existing cache-priced catalog entries to classify them as implicit vs explicit (based on provider docs — Alibaba Qwen 3.7 Max / 3.6 Max Preview have write+read separately = explicit; Google/Groq/CrofAI have only cached_read with bundled costs = implicit).
8. Add explicit cache pricing for `qwen/qwen3.7-plus` under `alibaba-cloud` (currently missing cache pricing entirely).
9. Revisit ambiguous entries: `qwen/qwen3.7-max`, `qwen/qwen3.6-max-preview`.
10. Update the pricing engine subset logic (`engine.ts`) to only apply cached-read-is-subset-of-input deduplication for implicit cache.
11. Add/update tests in `packages/data/catalog/src/data/__tests__/` and the API pricing pipeline tests.

**Important:** Reproduce the issue first by inspecting the catalog pricing JSON files, the API control model response shape, the OpenAPI spec, and the SDK generated types. Do not rely solely on this bug report — verify the current state of each layer and run `pnpm validate:data`, `pnpm lint`, `pnpm typecheck`, and `pnpm test` after changes.

Clarifying Questions (Optional)

  1. Should the distinction live as a new field on each pricing rule (e.g., "cache_type": "explicit"), or should new dedicated dimension keys be created (e.g., explicit_cache_write_text_tokens)? Both approaches have trade-offs — a decision is needed before implementation.
  2. How should explicit cache write vs explicit cache read surface in CompatibilityPricing — two new fields, or a single nested object?

This is an automated response - a team member will follow up shortly.

opencode session  |  github run