#17305

feat(usage): retain reported cache-write duration tiers in diagnostics

Openxiaoyaner0201opened 2d ago

Problem

OmniRoute already tracks aggregate cache-read/cache-write usage and exposes a bounded prompt-cache health summary. It does not retain the provider-reported 5-minute and 1-hour cache-write buckets throughout its translation and persisted diagnostic paths.

Anthropic can report both usage.cache_creation.ephemeral_5m_input_tokens and usage.cache_creation.ephemeral_1h_input_tokens. Collapsing them into one write counter loses the distinction needed for tier-aware API-equivalent diagnostics. A client cache TTL hint is not a substitute for the provider's reported usage, and a subscription estimate is not a cash invoice or a precise account-quota deduction.

The existing native Claude OAuth TTL normalization in #10221 is a separate serving-policy feature. This request does not propose changing that policy.

Implementation: #17307 (open for review).

Proposed bounded slice

  • Preserve reported 5m/1h counters through native and translated stream/nonstream paths, including cumulative and EOF receipts.
  • Retain nullable per-call diagnostic metadata; legacy aggregate-only rows remain unknown.
  • Extend the existing cache-health management projection and dashboard with reported, unknown and unpriced portions.
  • Snapshot a diagnostic-only API-equivalent price for the reported and priced tier portion. Do not reprice history or change quota deductions, existing provider-exact costs or subscription accounting.

Acceptance

  • A receipt with total write 100, 5m 30 and 1h 70 remains total 100, not 200.
  • Later empty or partial cumulative fragments do not erase earlier valid buckets; repeated snapshots are not added as deltas.
  • Missing, partial, invalid and inconsistent data remain visible and are never inferred from a requested TTL.
  • Existing routing, account selection, TTL defaults, quota/budgets and default cost consumers remain unchanged.
  • Migration/restore retains old installations and nullable legacy rows.
  • No prompt contents or credentials are collected, and summaries reuse the existing bounded retention/query path.

Follow-up boundary

Requested-versus-effective outgoing TTL, account-selection/conversation-level explanations and change-linked before/after optimization evidence are distinct follow-ups. This first slice reports provider receipt tiers; it does not claim to measure outgoing TTL or prove that a particular account change caused waste.

Infographic

Illustrative aggregate, not a benchmark or invoice. Unknown is a data-availability category, not a third TTL setting.

中文说明

问题

OmniRoute 已有缓存读取、写入总量记账和有界的缓存健康汇总,但尚未在协议转换与持久化诊断链路中完整保留提供者报告的5分钟、1小时两档缓存写入量。

Anthropic 可以同时报告 usage.cache_creation.ephemeral_5m_input_tokens 和 usage.cache_creation.ephemeral_1h_input_tokens。只保存写入总数,会丢失按档位进行 API 等价估算所需的信息。客户端声明的 TTL 不能替代提供者的实际用量回执;订阅账号的 API 等价估算也不是现金账单或账号额度的精确扣减。

#10221 已实现的原生 Claude OAuth TTL 归一化属于另一项服务策略,本需求不修改它。

实现 PR:#17307,已开放审查。

首个独立切片

  • 在原生、转换后的流式和非流式响应中保留两档回执,覆盖累计快照与 EOF 收尾。
  • 增加可为空的逐调用诊断信息,历史只有总数的记录保持未知。
  • 扩展已有缓存健康管理接口及页面,区分已报告、未知和未定价部分。
  • 记录已报告且可定价部分的诊断性 API 等价价格快照,不重算历史、不修改额度扣减、提供者精确成本和订阅记账。

验收要求

  • 写入总数100、5m30、1h70的回执,仍为总数100,而不是200。
  • 后续空对象或部分累计片段不擦除已有有效桶;重复快照不按增量叠加。
  • 缺失、部分、无效及不一致数据如实显示,不从请求 TTL 推断。
  • 路由、选号、TTL 默认值、额度/预算及默认成本使用方保持原有行为。
  • 迁移与恢复兼容既有安装和可为空的旧记录。
  • 不采集提示词正文或凭据,汇总沿用已有的有界保留与查询链路。

后续边界

客户端声明与实际发出 TTL 的对照、选号/会话级原因解释、优化变更前后效果验证,分别作为后续工作。本切片只报告提供者回执档位,不声称已测到实际发出 TTL,也不宣称某次换号已经被证明造成浪费。

说明图

这是跨调用汇总示例,不是性能基准或现金账单;未知档位表示缺少数据,不是第三种 TTL 设置。

Activity

0

No activity yet.