Serve Jeff with adaptive shared-GPU batching

Interactive and bulk decisions share one pinned Jeff Fast model through an authenticated HTTP service. It encodes each request once, groups similar lengths into bounded batches, and applies deadlines, admission limits and bulk fairness. The portable image includes verified weights and upstream notices; the Salad recipe supports moving the service to a new GPU.

Measured over two minutes on RTX PRO 6000 Blackwell, localhost HTTP, four logical CPU cores and a deterministic 50/50 traffic mix. A separate 6,000-request, 90%-bulk run also had zero errors. Saturation screening reached 181–186 requests/sec versus 60–63 for an unbatched control on another identical GPU; those rates have higher queueing latency.

A latency-first configuration (batch 1, zero intentional waits, interactive-only) completed 1,000 requests at 50/sec with zero errors: inference p50/p95 12.06/19.86 ms; localhost HTTP 14.58/28.01 ms. Two same-GPU 25-rps repeats measured 26.19/26.20 ms HTTP p95. Its full 5,943 predictions also passed retention gates. This is a warm workload result, not a universal 20-ms guarantee; the full general panel had 21.77 ms p95 inference and a 581 ms HTTP outlier. Direct regional routing remains unprovisioned. Configuration and evidence are in gpu/jeff/evidence/latency.json.

All 5,943 development predictions passed retention gates against native Jeff and the published fixed-batch recipe; native macro deltas were −0.0925 pp broad/+0.2608 pp general. Input collation matched the immutable preparation exactly across 1,712 batch comparisons. Weights, BF16 and context handling remain fixed. These are development point gates, not a new JevBench score or proof of statistical noninferiority.

Remote HTTPS completed 6,000/6,000 requests at 100/sec: interactive p50/p95/p99 224/391/640 ms, bulk 236/457/761 ms. A 192-request heavy burst passed all 24 API/deadline/overload/recovery checks (159 successes, 33 admission rejections).

Limitations: new shapes still cause compiler stalls. HTTPS used a temporary Cloudflare tunnel with no uptime guarantee and a 200-request concurrency cap; a 256-request heavy burst hit a transport error. The SSH 100-rps test failed substantially. Salad is not provisioned; its account scope/API access remains unresolved. Image build/import/hash verification and native GPU inference were verified separately. The evidence records overload failures and a rejected tensor-count optimization whose apparent win disappeared under a same-GPU ABBA comparison.

Visual evidence is not applicable: this adds a standalone backend and deployment recipe without a visible interface change. Results and reproduction commands are in gpu/jeff/README.md and gpu/jeff/evidence/measurements.json and https.json.