#1136

fix(ci): wait for the dependency producer before triage refusal

Openiamgpwants to merge intofrom

Goal and scope

Fixes #1203. The scheduled Security triage job must not reject its own completed dependency scan because GitHub had not listed the producer job yet.

Scope: the allowlist guard and producer check in scripts/report_dependency_risk.py, plus its tests. No workflow, permission, token, or intake change.

Non-goal: retrying the artifact download, or retrying unrelated GitHub API calls.

Change map and risk

  • handoff() now separates immutable allowlist facts from the mutable producer job list. A refusal reports every failed predicate with its observed and expected value, for example event='workflow_dispatch' (expected 'schedule'), instead of one opaque conjunction.
  • The producer job is polled every 5 s for up to 120 s, inside the triage job's 10-minute timeout. Behaviour stays fail-closed: a completed job whose run id, attempt or SHA does not match this scan is refused on the first read, as is more than one completed producer. A producer that never becomes visible is refused at the deadline with the states observed.
  • PRODUCER and SECURITY_WORKFLOW constants replace inline duplication, and a test pins PRODUCER to the job name in security.yml so a rename cannot silently break the guard.

Risk: real mismatches now surface after the read that proves them rather than instantly, and a genuinely missing producer surfaces after the bounded wait. Nothing that writes to GitHub changes: publish() still runs only once the guard passes.

Verification

  • make check: 15/15 checks passed.
  • uv run --locked pytest tests/scripts: 644 passed.
  • uv run --locked pytest tests/scripts/test_report_dependency_risk.py: 33 passed.
  • test_producer_appearing_after_first_poll_is_accepted reproduces the incident: it fails with the retry loop removed, and passes now. test_missing_producer_rejected_at_deadline bounds the wait and asserts the observed-state message.
  • Read-only replay of run 40458746490's API payloads: every allowlist predicate passes ([]), so the job list was the only possible rejection, and the wait accepts that run's producer after a simulated lagged first poll. handoff() itself was not executed, because a passing guard publishes issues.

Recovery and maintainer decisions

  • Revert this commit to restore single-snapshot behaviour; there is no data, migration, or published artifact involved.

  • Acceptance criterion 4 of the issue — a green security / dependency triage — needs a re-run of 40458746490 or the next scheduled run after merge. The triage job checks out main for this script, so a re-run exercises the fix without waiting for 01:00 UTC.

  • No independent reviewer yet.

  • For a fix, name the regression test that fails before the fix and passes after it, or justify why no regression test applies. tests/scripts/test_report_dependency_risk.py::test_producer_appearing_after_first_poll_is_accepted


    

Activity

295
coldtea-pr-lens[bot]3d ago
  • Redraw · drawn at 23f7d42, the branch has moved on

Architecture


Data flow


Review

✨ Nothing flagged · reviewed 23f7d42


TIP

After a push, tick Redraw to draw the latest commit

🪧 More tips
  • Run npx skills add coldteadotai/pr-lens, then tell your coding agent: "Diagram the change you just made with PR Lens and attach it to the pull request."
  • Run npx @coldtea/pr-lens-cli analyze --base origin/main on a branch, then npx @coldtea/pr-lens-cli render .pr-lens/graph.json. Same lenses, your own model key, before the pull request exists
  • Click the link under each diagram to open it on a canvas you can zoom, pan and step through
  • The diagrams are links. Click one to open it on the canvas, then press W or click play to walk through the change
  • Open a diagram on the canvas, then press W or click play to walk through the change one step at a time
  • The CLI's render reads .github/pr-lens.yml and applies your renames, exclusions and lane pins at draw time
  • Set github.comment.collapsed: true in .github/pr-lens.yml to fold the comment behind one View architecture… row. Drawing still runs as before
  • Set github.draw: on-demand in .github/pr-lens.yml and PR Lens stops drawing pull requests on its own. Comment @pr-lens draw on a pull request when you want that one drawn
  • Add .github/workflows/pr-lens.yml with coldteadotai/pr-lens/packages/action@v0 and your model provider's key as its api-key to run PR Lens from your own CI. Any /chat/completions endpoint works
  • Switch GitHub to dark mode and the diagrams follow. The moving dots are this pull request's data in motion
  • Set github.description: false in .github/pr-lens.yml to stop PR Lens adding buttons to your PR descriptions

Thanks for using PR Lens! It's free for OSS. If it helped, a star helps other teams find it. 

iamgp3d ago

@phlo-agent review the change at head 23f7d429fdf1231c8d2b442d125d6552776a38e4. Focus on the new bounded producer wait and the refusal messages in scripts/report_dependency_risk.py, and on whether the added tests would catch the reported bug.

phlo-agent[bot]3d ago

Reviewed head 23f7d42 against base 304877b3: the new bounded wait and refusal paths, tests/scripts/test_report_dependency_risk.py, security.yml / dependency-triage.yml, issue #1203, and CI at this head (GitHub Actions check suite success, pr / required green). Static inspection only; I did not execute the pull request code.

No blocking findings. Every predicate of the old conjunction is still enforced, the wait is bounded, and nothing that writes to GitHub moves ahead of the guard. One test-strength gap and one diagnostic nit.

Minor — the regression test pins only one shape of the lag

test_producer_appearing_after_first_poll_is_accepted makes the producer absent from the first poll (jobs.pop()), then appends it completed. The other shape consistent with the incident — the job listed but still in_progress a few seconds after it finished — is never exercised with a budget: the context fixture pins PRODUCER_WAIT_SECONDS = 0, so the in-progress case in test_in_progress_run_requires_one_completed_dependency_producer only asserts a refusal.

So a retry that polls only while no same-named job is listed (while not producers and monotonic() < deadline), or that refuses as soon as it sees a non-completed producer, reproduces #1203 for that variant with the whole suite green — including the test the description names as the regression proof. The shipped loop is correct, because acceptance is gated on completed rather than on the job merely existing (scripts/report_dependency_risk.py:220-239). Parametrizing the lag test over "absent" and "present as in_progress" → completed would keep it that way.

Note — wait refusals reuse the allowlist sentence, and diagnostics can duplicate

_refusal prefixes the timeout and identity-mismatch refusals with the same "Only an allowlisted daily scan with one completed dependency producer…" sentence (:170-175). Since the issue's complaint was that a lag failure looks identical to a real allowlist rejection, anything keyed on that sentence still conflates them; only the appended producer states disambiguate.

_allowlist_failures also reports one cause twice: head_sha when the value is malformed and differs from the run's (:188 alongside :208-209), and run_attempt when attempt < 1 (:187 alongside :204-205). Cosmetic, but it works against the "one line per failed predicate" intent.

What I checked and found sound

  • Guard equivalence: _allowlist_failures covers event, head branch, default branch, run id, attempt, full-SHA format, head SHA, both repository names, workflow id and path, run path, status and attempt >= 1 (:178-210); _await_producer still requires exactly one completed producer whose run_id, run_attempt and head_sha match, refusing immediately on a mismatch or a duplicate completed producer. The only acceptance widened is "the completed producer becomes visible on a later read".
  • Bound: 120 s deadline at a 5 s poll, checked after each read so a producer completing at the deadline is still accepted; comfortably inside timeout-minutes: 10 in dependency-triage.yml#L19.
  • The added test does catch the reported bug: the base handoff() took one snapshot and refused when len(completed_producers) != 1 (base :139-176), so test_producer_appearing_after_first_poll_is_accepted fails before this change and passes now. test_completed_dependency_job_must_match_exact_attempt usefully pins that a completed job's identity mismatch is refused without waiting.
  • PRODUCER is pinned to security.yml's dependency job name, which matches security.yml#L40.

Validation remaining

Acceptance criterion 4 is still unproven: no run has exercised the wait. The re-run plan in the description does work — the triage step checks out ref: main, and the reusable workflow file it calls is unchanged by this pull request — so a re-run of 40458746490 executes the fixed script.

Separately, github.download_artifact's artifact listing is still a single read. If that read lags instead of the job list, the guard passes and the run converts to a "dependency assessment unavailable" escalation issue rather than refusing. Out of the stated scope, same lag class, worth a follow-up only if it recurs.

iamgp3d ago

Addressed the test-strength gap in 0f2eaa1: test_producer_appearing_after_first_poll_is_accepted now parametrises over an absent producer and one listed as in_progress, so acceptance stays gated on completed rather than on the job merely existing. I checked that the in_progress case fails against a loop that refuses as soon as a non-completed producer appears, which is the wrong implementation you described.

On the diagnostics: the opening sentence stays on wait refusals because it names which guard refused, and the appended producer state is what separates lag from an allowlist mismatch; nothing keys on the sentence. The duplicated head_sha and run_attempt lines only arise from a malformed argument, where the second line names the actual defect, so I left them rather than add dedupe machinery for a case the attempts API rejects first.

The artifact-listing lag is the same class but outside this change; a follow-up only if it recurs.