#1134

fix: prevent stale WAP audit approval with candidate-bound checks

Openiamgpwants to merge intofrom

Summary

Add opt-in wap.require_candidate_bound_checks for branch WAP. A passing audit must identify the same branch and immutable hash that promotion publishes. Existing custom-check and no-check behaviour remains unchanged unless the policy is enabled.

The reproduced failure was a check passing on revenue of 41, followed by another candidate write changing revenue to 52 before the promotion sensor pinned the branch. The old sensor published 52 using the earlier passing check.

Changes

  • Automatically bind standalone Phlo checks to a branch hash and reject changes during their execution.
  • Provide phlo_dagster.wap_launch.bind_wap_audit for custom and in-band read-only checks.
  • Preserve candidate identity in durable Dagster check evidence. Reject missing or stale bindings before writing promotion intent.
  • Bind the policy into immutable launch context, preserve legacy context serialisation when disabled, and reject snapshot WAP at configuration validation.
  • Document configuration, recovery actions, supported checks, and limitations.
  • Validate tagged WAP runs against their immutable launch context before allowing legacy audit behaviour. Disabling strict mode after launch now fails before the check body executes.
  • Run Observatory, GitHub writer and plugin checks in independently selected CI jobs. Require each selected job in the aggregate and full-queue evidence. Use the runner's preinstalled Chrome with pinned agent-browser 0.38.1 instead of installing OS dependencies during CI.

This reuses the existing publication, receipt and conditional-cleanup path. It does not add another journal or publication engine. The broader SSP research notes are not included.

Verification

  • Regression cases failed before the gate was added because stale and unbound checks still published.
  • uv run --locked pytest packages/phlo-dagster/tests -m 'not integration' -q: 545 passed, 13 deselected. The new policy-disable case failed before the follow-up fix and now confirms the check body never executes.
  • Public export, plugin discovery, provider compatibility and snapshot-helper suites: 26 passed, 2 deselected.
  • make lint-python format-python typecheck-python: passed, including all six import contracts.
  • make docs-reference-check docs-generate: passed, 1,044 pages generated. Codespell and git diff --check: passed.
  • Disposable real Dagster/Trino 477/Nessie 0.108.3/MinIO/Iceberg probe: all six scenarios passed. The stale candidate containing 52 was retained, the target remained 41, and zero publication submissions occurred. Unchanged publication, target drift, failed audit, lost acknowledgement and post-pin mutation controls remained correct.
  • Workflow aggregation, evidence, cache-scope and selector contracts: 268 passed. make actionlint zizmor: passed, no findings. Writer typecheck, 11 tests and build passed; plugin tests: 8 passed, no skips.
  • Observatory production build and full node scripts/browser-smoke.mjs passed with agent-browser 0.38.1 and AGENT_BROWSER_EXECUTABLE_PATH pointing to existing orb Chrome. No browser or OS dependency installation was needed. This checks startup, navigation, queries, permission failure, staging and caller identity; it does not establish hosted runner duration.

The real-stack probe was rerun successfully after the follow-up changes. It used in-process Dagster with injected local code-location and catalogue resolution, not an authenticated remote launcher or full process-level provider discovery. The fresh repository-wide make check passed every lane except its initial Ruff lint invocation, which read an assertion while workflow tests were still being edited. The reported SIM300 assertion is corrected in the pushed code; final full Ruff lint and formatting reruns pass. The repository-wide non-integration pytest lane passed. A production documentation build attempted during investigation was killed with exit 137; documentation generation and the current head's hosted documentation build passed, but a complete local site build is not claimed. The current head's Observatory, writer and plugin GitHub jobs have all passed. Remaining exact-head CI is running.

Compatibility and limits

Enable with wap.enabled: true and wap.require_candidate_bound_checks: true in phlo.yaml. Deploy launchers, workers and sensors together. Policy changes require a new launch rather than downgrading an existing run.

Strict mode requires every executed check, including advisory checks, to carry a binding and does not permit runs without checks to publish. In-band checks must bind their own read-only evaluation, not the asset's write operation. Branch stability does not prove arbitrary SQL read the intended ref or bind external comparison data. Keep candidate branches under trusted writer control without concurrent resets. Any later candidate commit invalidates prior audits, even if it changes an unrelated table.


    

Activity

300
coldtea-pr-lens[bot]4d ago
  • Redraw · drawn at fde90b7, the branch has moved on

Architecture


Data flow


Review

1 finding on the diff · reviewed fde90b7

TIP

Run npx skills add coldteadotai/pr-lens, then tell your coding agent: "Diagram the change you just made with PR Lens and attach it to the pull request."

🪧 More tips
  • Run npx @coldtea/pr-lens-cli analyze --base origin/main on a branch, then npx @coldtea/pr-lens-cli render .pr-lens/graph.json. Same lenses, your own model key, before the pull request exists
  • Click the link under each diagram to open it on a canvas you can zoom, pan and step through
  • The diagrams are links. Click one to open it on the canvas, then press W or click play to walk through the change
  • Open a diagram on the canvas, then press W or click play to walk through the change one step at a time
  • The CLI's render reads .github/pr-lens.yml and applies your renames, exclusions and lane pins at draw time
  • Set github.comment.collapsed: true in .github/pr-lens.yml to fold the comment behind one View architecture and data flow row. Drawing still runs as before
  • Set github.draw: on-demand in .github/pr-lens.yml and PR Lens stops drawing pull requests on its own. Comment @pr-lens draw on a pull request when you want that one drawn
  • Add .github/workflows/pr-lens.yml with coldteadotai/pr-lens/packages/action@v0 and your model provider's key as its api-key to run PR Lens from your own CI. Any /chat/completions endpoint works
  • After a push, tick Redraw to draw the latest commit
  • Switch GitHub to dark mode and the diagrams follow. The moving dots are this pull request's data in motion
  • Set github.description: false in .github/pr-lens.yml to stop PR Lens adding buttons to your PR descriptions

Thanks for using PR Lens! It's free for OSS. If it helped, a star helps other teams find it. 

coldtea-pr-lens[bot]reviewed4d ago
coldtea-pr-lens[bot]reviewed4d ago