#2568

subagent-driven-development: a finding the reviewer can't verify because it depends on the production environment gets graded Minor and deferred "to check at deploy"

Openjcampeloopened 10d ago
  • I searched existing issues and this has not been proposed before

Related but different: #2471 asks for deferred Minors to carry recoverability as well as likelihood. This issue is about findings nobody could verify. A reviewer who can't check a finding has no likelihood to grade, and that gets read as low risk.

What problem does this solve?

In one SDD run, the implementer flagged that a login redirect built with new URL(target, request.url) "only works if the proxy forwards the right Host". The task reviewer graded it Minor because it couldn't be checked without the production build behind the reverse proxy. The controller ledgered it as minor (deferred): check at deploy. All three treated "can't verify here" as "low risk".

The final whole-branch review read the framework's code (Next.js 16, next start) and showed the defect would happen every time. request.url is built from the server's own listen address, the proxy's redirect rewriting wouldn't fix that Location, and login would have broken for every user in production.

SDD already has "⚠️ Cannot verify from diff", but only for requirements that live in unchanged code or span tasks, and the controller has to resolve those before marking a task complete. A finding whose severity depends on an environment the reviewer doesn't run falls outside that rule, so it goes to the Minor route, where "Minor findings never enter the loop".

Proposed solution

  • In task-reviewer-prompt.md, if a finding's severity depends on an environment the reviewer can't run (proxy, host headers, production build, environment variables), the reviewer reports it as a ⚠️ Cannot verify item. It isn't graded Minor by default.
  • In "The fix loop", the controller resolves such an item before marking the task complete, on the same footing as the existing ⚠️ items. Either it verifies the behaviour for real (runs the production build, reads the framework code), or it applies the cheap fix that removes the dependency on the environment (here, building the URL from configuration). "Check at deploy" is not an allowed outcome for a finding on a main user flow.

What alternatives did you consider?

Relying on the final reviewer, which caught it here. But a deferred Minor reaches the final review as a one-line ledger entry with nothing that says it was never verified, and in this run the implementer flagged it, the reviewer graded it Minor and the controller deferred it before it got there.

Is this appropriate for core Superpowers?

Yes. Every deployment target has an environment the reviewer doesn't run, so this isn't specific to one stack.

Environment (required)

FieldValue
Superpowers version7.0.0 (skills/subagent-driven-development/SKILL.md and task-reviewer-prompt.md identical to main as of 2026-10-20)
Harness (Claude Code, Cursor, etc.)Claude Code
Harness version2.1.304
Your model + versionClaude Opus 5.5 (claude-opus-5-5)
All plugins installedsuperpowers 7.0.0, playwright, frontend-design, code-review, skill-creator, vercel 0.50.0, supabase 0.1.16, pyright-lsp 1.0.0, andrej-karpathy-skills 1.0.0, claude-code-setup 1.0.0, diagram-design 2.6.115
OS + shellWindows 11 Pro, Git Bash

Written by the agent at its human partner's request, from what it recorded during the session (2026-10-19). The session's model at the time of the run isn't stated in that record.

Principle: when a reviewer can't verify something, that is an unknown, not a low risk. An unverified finding on a main flow gets resolved by verifying it or by removing the dependency, never deferred to the environment where the failure would first show up.

Activity

0

No activity yet.