#2587

writing-plans / subagent-driven-development: plans fix the type model task by task, so structural problems surface only at the broad review (Claude Code)

Opensergeykadopened 8d ago
  • I searched existing issues and this is not a duplicate (searched: "writing-plans", "pre-flight", "type model", "interfaces block", "architecture section plan", "midpoint review"; closest: #1803 Checkpoints - Adversarial Review, #2309 writing-plans: self-review misses task-state/DAG closure in executable plans)

Environment

FieldValueProvenance
Superpowers version7.0.0 (plugin cache, not a checkout)current observation: ~/.claude/plugins/cache/claude-plugins-official/superpowers/7.0.0
HarnessClaude Codecurrent observation
Harness version2.1.306current observation: claude --version
Modelscontroller claude-opus-5-5; implementers sonnet; reviewers opuscurrent observation and dispatch records
Other pluginspr-review-toolkit, code-review, frontend-design, claude-md-management, mattpocock-skills, skill-creator, elements-of-style, episodic-memory, code-modernization, chrome-devtools-mcp, claude-hud, headroom, show-mecurrent observation: plugin cache
OS + shellLinux 7.0 (Ubuntu), bashcurrent observation

Is this a Superpowers issue or a platform issue?

  • I confirmed this issue does not occur without Superpowers installed

This is about the plan format and the review gates, so it has no platform part. It is not a runtime bug and was not reproduced without Superpowers.

What happened?

Two consecutive plans for one project (a Python analytics runtime), called the earlier plan and the later plan below, were written with writing-plans and run with subagent-driven-development. Both needed a large structural fix round after the tasks, and in both the problems came from the plan text, not from implementer choices.

The later plan (15 tasks, 1,722 lines). A broad review after Task 8 (review-pr toolkit plus /simplify, 9 opus reviewers) found 9 structural changes, 7 other Important findings and 31 Minor findings. Every task before it had passed its per-task review. Over the whole plan, the 15 tasks took about 7 hours and the three broad review passes with their fixes about 8.5 hours (halfway about 3, final about 3.5, a second final pass about 2). The structural findings fell into recurring patterns. Each one was written into the plan's task text, so the implementers built what they were told:

  • One fact, many optional fields. A fact that some objects always have was added as optional fields on several types, one task at a time. The code then re-checked it with about ten runtime guards.
  • One rule, several homes. A rule about one kind of thing was defined separately in four places, because two tasks each needed it and each defined its own version.
  • Copied behaviour. Two sibling classes, each from a different task, repeated the same multi-step method with small differences.
  • Data needed before it exists. An object built late in the flow was required by an earlier step, so that step worked around it by reading a lower-level table directly.
  • Lost error types. A structured list of typed errors was flattened into one string at a module boundary, because the plan's interface for that function raised a plain string error.
  • Import cycle. A class was placed in a different package from its only user, so an extra interface was needed to break the import cycle.

The earlier plan (15 tasks, 790 lines) showed the same pattern: after the tasks, a whole-branch review, then review-pr and /simplify passes, led to a structural refactor that moved per-kind special cases into one record per kind and replaced an untyped config with typed models. The time after the last task until merge was about as long as the tasks.

Where it comes from, as observed:

  1. writing-plans gives each task its own Interfaces block (Consumes and Produces). No section shows the whole type model: which type owns each domain fact, where each kind's traits live, which module may import which, and which error type crosses each boundary. When two tasks need one fact, each task defines it.
  2. The subagent-driven-development pre-flight scan checks consistency. On the later plan it found 6 blocking defects, all of the form "a signature cannot reach its data" or "a test contradicts the spec", and 38 minor ones. It does not judge design shape, and it found none of the 9 structural problems.
  3. Per-task reviewers see one task's diff, so a fact defined twice across tasks is invisible to them. The first review that sees the whole design is the final one, after later tasks have copied the core abstractions.
  4. The independent pre-flight runs at the start of execution. The plan's author only self-reviews, so defects found by the scan come back as plan amendments in the executing session (the later plan: one amending round before Task 1).

Results so far

We applied the suggestions below to the later plan from Task 9 on (a type model section, a design-shape check of Tasks 9 to 15 before they ran, reviewers checking diffs against the type model):

Tasks 1 to 8 (before the changes)Tasks 9 to 15 (with the changes)
Problems caught in the plan text before execution0 (the pre-flight checks consistency only)7 blocking, found by the design-shape check
Structural findings in the code, at the broad reviews9 (halfway review)11 (5 at the final review, 6 more at a second pass after those fixes)

The shape check is the clear gain: it found 7 problems in plan text before any code existed. The type model did not reduce structural findings. The 11 in Tasks 9 to 15 were about runtime flow (which handler owns an event, where state lives over time), which a table of fact owners does not cover. Tasks 9 to 15 also built on a core the halfway fixes had already cleaned up, so even this comparison favours the second half. Some of the second pass findings may come from the first round's fixes rather than from the plan.

Where in the skills (v7.0.0)

Example (made up, same shape as the real findings)

A shop supports three payment methods: card, bank transfer and voucher. Each has a fee rate. Three tasks of the plan need the fee:

Task 2  Produces: Order.payment_method: str
                  Order.fee_rate: float | None      # set at checkout
Task 4  Produces: FEES: dict[str, float] = {"card": 0.02, "transfer": 0.0, "voucher": 0.0}
                  def checkout(order: Order) -> Receipt    # reads FEES[order.payment_method]
Task 6  Produces: Refund.fee_rate: float | None
                  def refund(order: Order) -> Refund       # if order.payment_method == "voucher": no refund

Every row of the pre-flight passes: Task 4 consumes what Task 2 produces, and Task 6 consumes both. The per-task reviews pass too, since each diff matches its task. The result is the fee in three places, two optional fields that later code must check for None, and a name check on "voucher" in shared code. Adding a fourth method means finding all of them.

A type model section before Task 1 would have stated it once:

Fact                  Owner
fee rate, refundable  PaymentMethod (one record per method, in PAYMENT_METHODS)
an order's method     Order.method: PaymentMethod   (never None)

Tasks 2, 4 and 6 then cite these rows. A design-shape check can flag the plan text above before execution: an optional field for a fact every order has, a rule held in a dict apart from its kind, and a name check on a kind.

Steps to reproduce

  1. Write a plan of 10 or more tasks with writing-plans for a feature that adds several kinds of one concept, where early tasks build the core abstractions and later tasks consume them.
  2. Run it with subagent-driven-development, with per-task reviews.
  3. Run a whole-branch review that looks at structure across files (for example review-pr plus /simplify) after the core tasks.
  4. Count structural findings and check how many are mandated by the plan text.

Expected behavior

The plan defines one owner per domain fact and one home per kind's traits before the tasks, and a design-level check catches duplication or optional fields for always-present facts before execution starts, or at the latest right after the core-abstraction tasks.

Actual behavior

The type model is fixed piece by piece across tasks. Design problems are caught only by a broad review, after several tasks depend on them, and the review passes and their fixes cost more time than the tasks (8.5 hours against 7 in the later plan).

Suggestions

These are what we now do in the project's own instructions, ordered by how well the results above support them. They may or may not fit the skills.

  • Measured to help: a design-shape checklist in the pre-flight (one owner per fact; no optional field for a fact its kind always has; no trait split across modules; no code that branches on a kind's name; a typed error at each boundary; an import direction for each new module), run by a fresh reviewer at the end of writing-plans, before handoff. It found 7 problems in plan text before any code existed.
  • Did not help on its own: an optional "Type model" section before Task 1 (each domain fact and its one owner type, each kind of thing and the one record that holds its traits, which module may import which, the error type at each boundary), with each task's Interfaces citing its rows. It gives the shape check something to check against, but structural findings did not drop.
  • Not yet tried: skeleton first (Task 1 builds the type model as code and gets a broad review before other tasks build on it), and a way to settle runtime flow in the plan (which component handles each event, where state lives over time), the kind of problem that made up the later findings.

Related: #1803 (adversarial review before execution) covers checking the plan before it runs, and #2309 covers task-boundary state in the self-review. Neither covers the plan's type model or a design-shape check.

Debug log or conversation transcript

Local session data and the run's progress log are available but not attached, because they contain private project code. The counts above come from the plan's results summary, written from the run's progress log and review files, and from the merged findings file of the last review pass.


Filed with the diagnosing-superpowers skill.

Activity

0

No activity yet.