vercel-optimize: build-cost audit foundation (build_machine_oversized gate, build signals, fixes)

Summary

This PR makes vercel-optimize useful for build-cost questions. It has three parts:

1. A new gate, build_machine_oversized, and a build signals collector that later build gates can share. The gate flags projects pinned to Enhanced or Turbo and recommends Elastic.
2. A build-focused audit path. Build checks use no route metrics, so an Observability Plus blocker no longer stops a build-cost audit.
3. Five fixes to existing code, found by running the skill end to end, including a security fix: Deploy Hook URLs were being written to signals.json.

It's the first of several build gates. The others are listed under "Next" and aren't in this PR.

The gate and collector

• lib/build-signals.mjs collects 14 days of deployments for the linked project (GET /v7/deployments, up to 5 pages). For up to 40 recent READY/ERROR builds it reads GET /v13/deployments/:id:

  • duration.timeForBilling and duration.cpuTimeForBilling
  • resourceConfig.buildMachine (purchaseType, cores, memory, machineSelectionType)
  • buildSkipped
  • whether the build came from a Deploy Hook (meta.deployHookName is present; source is git for both pushes and hook builds)

  It uses platform data only. It never reads or stores meta.deployHookId, which is the secret end of the hook URL.

• lib/gates/build-machine-oversized.mjs fires when a project is pinned to Enhanced or Turbo, with at least 5 successful builds on that fixed machine and an estimated 300+ CPU minutes in the window. It's account-scoped, so it lands in the platform bucket. It recommends Elastic and reports the break-even billed duration on the next smaller machine as context.

Why

Build CPU Minutes are billed duration × vCPUs. A Turbo build (30 vCPUs) that bills 2 minutes uses 60 CPU minutes. Many projects stay pinned to a large machine after a one-time need, and the extra cores often don't speed the build up.

The skill's rules ban savings figures, so the impact label states a break-even instead:

High impact — builds on the pinned Turbo machine bill a median 2 minutes; switching to Elastic lets each build use a smaller machine when it fits (Enhanced alone costs less unless builds take longer than 7 billed minutes).

The fix is Elastic, not a fixed smaller machine, because build peak memory isn't collected. Elastic moves a project up after an out-of-memory failure (oom-failure) and down after consistently short builds (short-build-duration).

Gate rules

• Median billed minutes come from successful builds only; failed builds count toward volume but would otherwise drag the break-even down.
• Skipped builds, builds Elastic sized, and builds on another machine are excluded, so a project that switched setups partway through the window isn't mixed.
• CPU minutes come from cpuTimeForBilling when every sampled build has it, otherwise from billed minutes × vCPUs.
• A memory-related Elastic resize reason (oom-failure, high-peak-memory) on the project suppresses the gate.
• Turbo → Enhanced confidence is 0.8; Enhanced → Standard is 0.55 because Standard halves memory to 8 GB.

Build-focused audit path

SKILL.md and references/observability-plus.md: for build-focused requests (Build CPU Minutes, build machines, slow or expensive builds), an Observability Plus blocker no longer pauses the audit. The agent tells the user that route metrics are unavailable, using a new template, and continues with --continue-without-observability. Scope, auth, and link blockers still stop the run.

Fixes to existing code

Running the skill as an agent against a fixture monorepo surfaced these:

• Security: Deploy Hook URLs were written to signals.json. collect-signals.mjs stored the full /v9/projects response, including link.deployHooks[] with each hook's id and url. Anyone holding one can trigger builds, and the test agent printed one from the signals. The stored config now keeps only each hook's name, ref, and createdAt. This affects every run, not just build audits, and can be split into its own PR.
• The rerun after an Observability Plus blocker used stale data. SKILL.md said to "scan and merge again" into the same run directory, but merge-signals.mjs refuses to overwrite without --force. The merge failed and the gates ran on the blocked collection, which has no usage or build data. SKILL.md now passes --force.
• Account-scoped recommendations without a platform_ prefix were graded as code fixes. build_machine_oversized was held back at quality 0.17 for lacking file:line evidence. Candidates with an <account> ref now also get the account rubric (the higher score wins), and builds, vCPUs, and billed/CPU minutes count as evidence.
• Impact labels fell back to placeholders. When a written recommendation omits the gate's o11ySignal, the label now re-derives it from signals.json. A build_minutes_fanout fired only by the repo scanner no longer claims a billing threshold was exceeded.
• The repo scan included installed skills. scan-codebase.mjs now skips .claude and .agents, where project-installed skills live. Before, it scanned this skill's own source and reported it as findings.

Validation

Live fixture project

A fixture monorepo on the vercel-internal-playground team, with real builds:

The fixture also confirmed the API shapes the collector relies on: duration.cpuTimeForBilling equals billed minutes × cores, and resourceConfig.buildMachine is an object with purchaseType and machineSelectionType.

End-to-end agent run

The patched skill was installed in the fixture repo and run by Claude Code from a plain prompt ("Our Vercel build costs are way too high on the web project…"):

• The skill was selected from its description, and the build-focused path continued without pausing.
• build_machine_oversized was kept: quality 0.84 (Good), 7 of 7 claims verified. The Turborepo caching recommendation scored 0.88 (Excellent), 9 of 9.
• No savings figures, no fixed-Standard recommendation, no hook IDs in the output.
• Across three runs the agent tended to write its own summary instead of printing final-message.json verbatim, and once skipped the verify and render steps. That's existing workflow behavior, but worth knowing.

Tests

36 new fixture tests, synthetic data only, including one that checks a fake deployHookId never reaches the signals output.

node --test packages/vercel-optimize-tests/test/*.test.mjs   # 805/805
node skills/vercel-optimize/scripts/check-docs-fresh.mjs     # OK
node skills/vercel-optimize/scripts/check-citations.mjs      # OK

File changes

Known limitations and open questions

• Peak memory. The gate can't tell that a project needs its machine's memory, so it steers to Elastic rather than staying silent. Reading peak memory from the build system report would close this; how customers access that report isn't documented yet.
• One project per run. The audit covers the linked project only. A monorepo with five Vercel projects needs five runs; a team-wide view would be a separate change.
• Rate limits. The collector makes up to 45 API calls per run. Should it share lib/throttle.mjs with the metric queries?
• Thresholds (5 successful builds, 300 CPU minutes, priority at 5,000) are first-pass defaults.
• Applying fixes. vercel-optimize is report-only today. Is changing the machine setting with approval in scope for this skill, or a companion?

Related

• Metric queries on current CLIs: 16 of 32 metric queries, including the Observability Plus canary, fail with invalid_request on CLI 62.5.0+ because metric names, dimension casing, and allowed aggregations changed. Fixed separately in link to the count-aggregation PR.

Next

The upcoming "Optimize build costs on Vercel" KB guide (link once published) lists about 30 build optimizations; this PR covers machine right-sizing. The next gates reuse the same builds signal and project config, and the fixture already exercises each one:

• Deploy Hook rebuilds (triggeredByDeployHook)
• Skip-unaffected projects off in a monorepo (enableAffectedProjectsDeployments)
• Preview build volume per branch (target, branch)
• Commit bursts that one-build-per-branch would collapse (buildQueue.configuration, deployment timestamps)
• Failed builds on large machines (error rate × CPU minutes)
• Repo-scanner rules: Sentry widenClientFileUpload, .next/cache in Turborepo outputs, --webpack builds, missing turbo.json env declarations