#2612

subagent-driven-development: 15-task port of a ~2,000-line R script hit 5 session limits; coordinator context grew to 830k tokens (Claude Code)

Openmattallshopsopened 7d ago
  • I searched existing issues and this is not a duplicate (searched: token, tokens, context window, controller, expensive, rate limit, long session, session limit, cost, test suite slow; closest: #1152 "Codex: Implementation of relatively simple plan with subagent-driven development consumes full 5h token budget in a single run (PLUS plan)", which is on Codex with 5.0.7)

Environment (required)

FieldValueProvenance / supporting evidence
Superpowers version7.0.0 (not a checkout)historical evidence; installed_plugins.json, installed before the first session
Harness (Claude Code, Cursor, etc.)Claude Codehistorical evidence; transcript records
Harness version2.1.306historical evidence; version field on assistant records
Your model + versionclaude-opus-5-5 (main loop, 8 subagents); claude-sonnet-5-5 (35 subagents); claude-haiku-5-5 (small)historical evidence; message.model and cost-state records
All plugins installedsuperpowers@superpowers-marketplace onlycurrent observation; installed_plugins.json
OS + shellWindows 11 Pro 10.0.26300, Git Bash and PowerShell 5.1current observation

Is this a Superpowers issue or a platform issue?

  • I confirmed this issue does not occur without Superpowers installed

The reporter has not tried reproducing without superpowers. Evidence for involvement is below; it does not establish cause.

What happened?

A first-time Superpowers user ported a legacy R reporting script (about 1,850 lines, plus a 190-line helper) with 8–10 Config workbooks to a parquet pipeline, using brainstorming → writing-plans → subagent-driven-development. The user expected token use and waits comparable to earlier refactors done piecemeal in small chat sessions. Instead, the build hit the session limit five times (transcript lines 1497, 2522, 2869, 2880, 3255 of the plan-and-build session) and the monthly spend limit twice at the start of the next session (lines 35, 49), after a one-time limit reset and a $20 credit purchase. The user felt every test run took 10 minutes or more, and corrected the agents' decisions repeatedly.

What the transcripts show:

  • Cost. The harness's cumulative cost-state for the plan-and-build session, subagents included: $154.83, 323.8M Opus cache-read tokens, 1.09M Opus output tokens, 18.1 h duration (line 3901). The next two sessions: $14.64 and $7.93.
  • Coordinator context. 67k tokens at the first request of the plan session (line 38), 487k at plan approval (line 868), 552k by Task 2 (line 1249), 830k at the end (line 3898). No compaction occurred. During Tasks 1–15, the coordinator's 183 requests read 110.1M cached tokens; all 39 subagents of the session together read 110.5M. In the largest turn ("resume", line 1500), the coordinator read 51.5M against 25.4M for the 18 subagents it dispatched.
  • What filled the context before Task 1. writing-plans re-read sources the brainstorm session had already read (the legacy script, the hub decisions file, two sister projects' code). It wrote the plan as one 89,656-character Write (line 534) and edited it 37 times before approval.
  • Dispatches. 15 tasks plus the final review became 39 subagents: implement, review, and fix and re-review rounds for five tasks, then a final review, a fix wave and its re-review.
  • Test time. The plan defined green as the whole suite and listed a full-suite run in each task's steps. The suite (one Rscript per test file) went from about 6 s through Task 7, to 2.0 min at Task 8, 4 min at Tasks 14–15, 6–9 min at Task 13, 9–10 min at Task 17, a timed 9m41.9s, and one 63.8-minute call. Implementers ran the full suite two or three times per task in Tasks 10, 12, 13, 14, 15 and 17; the implementer prompt says "run the full suite once before committing". In total, 53 suite calls took 165 minutes. Five test runs exceeded the tool timeout and were moved to the background. Polling followed; three sleeps were blocked and four Monitor calls failed validation. One implementer killed three leftover R processes before re-running. Measured afterwards, one test file accounts for 73% of suite time, and nine of fifteen files take under 3 s.
  • Late reversal. During brainstorming, the assistant proposed rewriting the published table in full on every run, guessed "under a minute", and said "the build will time it". The user agreed. No benchmark is recorded until the final whole-branch review, after all 15 tasks, which estimated 10–12 minutes and 8–10 GB per run. The user then rejected the whole-table rebuild, which added a fix wave and two new tasks.
  • Exit. After the subagent loop, the user asked for the remaining task to be written directly. The main session did it in 55 requests and 11.2M tokens over 33 minutes.

Steps to reproduce

  1. brainstorming over two sessions for a port of an existing ~2,000-line R script into an incremental parquet pipeline; the design is approved.
  2. New session: "The next step is the implementation plan, using superpowers:writing-plans." The plan is written and approved in the same session (15 tasks, full suite per task).
  3. "the plan is approved … start executing the build" → subagent-driven-development in a worktree, in that same session.
  4. Observable: coordinator context above 500k tokens from Task 2 on; five session-limit stops; suite runs of 2–10 minutes repeated within tasks.

Expected behavior

Token use and wall-clock comparable to doing the same port in small chat sessions; test runs well under 10 minutes.

Actual behavior

Five session-limit stops, then a monthly spend-limit stop. A coordinator context of 487k–830k tokens re-read on each of 290 requests. 53 full-suite runs totalling 165 minutes. A run-cost estimate from brainstorming that was first measured after the build, followed by a redesign.

The four steps that cost the most, beside the skills' own text

Added at the reporter's request. Each item quotes the installed 7.0.0 skill text and cites the transcripts. Line numbers without a session id are in the plan-and-build session. This section proposes no change.

  1. The plan and the build ran in one session.
    • Skill text: requesting-code-review/SKILL.md:79 dispatches reviewers because "reviewing the diff inline burns the context window you need to keep driving the work". brainstorming/SKILL.md:191–192 ends with "invoke superpowers:writing-plans and no other skill", and subagent-driven-development runs "in the current session". No 7.0.0 skill mentions starting a new session between phases (searched: new, fresh and separate session; context window; /clear).
    • Transcripts: the plan session began at 67k tokens (line 38). The coordinator then entered Task 1 carrying the plan session's 487k (line 868). It re-read that context on each of its 183 requests in Tasks 1–15, a total of 110.1M cache-read tokens.
  2. The choice of execution method, and its cost comparison, was skipped.
    • Skill text: writing-plans/SKILL.md:179–205 offers two methods with a recommendation. Subagent-driven: "Most thorough; costs a fresh context per task and per review". Native: "Cheapest and fastest … Runs well with a mid-tier session model". When a method "has already been supplied", the skill asks only whether the plan captures what the user wants.
    • Transcripts: a project brief written before the first Superpowers session had pre-selected subagent-driven, and the plan session reported the choice as "already settled" (line 625). The reporter, a first-time user, never saw the comparison. The coordinator ran on the most capable model for all of its requests.
  3. Full-suite runs repeated within one task.
    • Skill text: subagent-driven-development/implementer-prompt.md:47–48: "run the focused test for what you're changing; run the full suite once before committing, not after every edit". subagent-driven-development/SKILL.md:392–394: "Name the covering test files in the fix message — a one-line fix does not need the whole suite."
    • Transcripts:
      • The Task 13 implementer re-ran the full suite after small edits ("Full suite once more, then commit"; agent-aaaa0979d6bf4fff8 lines 477, 507, 686).
      • A suite run in the Task 8 implementer outlasted the tool timeout and was moved to the background. The implementer then started it three more times while the first run was still going (agent-a3809bbdd41b58c4c lines 110, 144, 188).
      • The coordinator's fix-round brief asked for "the whole suite" (line 2548).
      • In total, 53 full-suite calls took 165 minutes.
  4. A run-time assumption was left for the build to measure, with no spike first.
    • Skill text: brainstorming/SKILL.md:104–106: "A quick throwaway build is often the cheapest way to learn what they want or whether something works. Offer one when it would help."
    • Transcripts: the brainstorm proposed rebuilding the whole output on every run, estimated "under a minute", and said "the build will time it" (brainstorm session 1e9a81a0, line 899). No spike was offered. The first measurement came in the final review, after all 15 tasks: 10–12 minutes and 8–10 GB per run (lines 3324, 3354). Reversing the design added a fix wave (125 requests, 27.2M cache-read tokens) and two more tasks across two more sessions.

Reporter's notes

Context from the reporter after reading the diagnosis:

  • This was their first Superpowers project, and they chose to follow its defaults as designed. The project brief set aside two of their own conventions to do that: one unit of work per session, and pausing before each commit to read the diff. Those conventions kept earlier refactors of similar code, done piecemeal in small sessions, well within usage limits.
  • Nothing in the brainstorm → plan → build flow told a new user that chat history does not carry into the next session. Nor did it say how to hand off between phases so the next phase does not re-read the same sources. The plan session re-read four sources the brainstorm had already read.
  • They could not tell where the full-suite run on every task came from. The implementer prompt's own wording reads as once before committing.
  • They had to correct the agents' decisions, or suggest better code or workflows, 13 times (cited in the diagnosis). Asking the process to be more efficient did not prevent more limit stops the next day.
  • The harness's cost records for the four sessions that have one total about $221 at API rates. The reporter would not have paid that for a port of this complexity. They plan to keep using Superpowers.
  • In the reporter's view, the four steps above are the ones that cost the most, and each one runs against the skill text quoted beside it.

Debug log or conversation transcript

Session id(s): 7ff973d7-4040-4d5a-ac38-4db7469c7949 (plan and build), 684e8588-5262-45b1-9db5-ed58f069d9f5, c37b9c24-46fb-4c11-b05c-1cc92d6a7db4, and brainstorm sessions 378c934b-7c5d-4949-8b23-02510a85fd6f and 1e9a81a0-625e-4931-8d04-608c30eb3312. Delivered local archive: none built. Attached bundle: no claim; attach only after approval. Superpowers involvement per the diagnosis report: likely, with evidence at transcript lines 38, 885, 1037–3552, 868–3898 (plan-and-build session) and line 113 (next session). This report does not propose a fix.


Filed with the diagnosing-superpowers skill. Model, harness, harness version, and installed plugins are listed above.

Activity

0

No activity yet.