Problem
task-done PLAN N BASE -- CMD appends a completion line to the ledger only if CMD exits 0. For a
task whose real verification does not run in the shell (a SQL test script sent through an MCP tool, a
manual check in a browser), the only way to get a ledger line is to pass a stand-in command such as
echo "tests passed". The ledger then records, in the same format as a real test run
(tests: echo … → tests passed), an assertion the script never checked. A later reader — or a session
resumed after compaction — cannot tell the two apart.
To reproduce
- Take a plan with a database-migration task whose tests run through an external SQL tool.
- Close the task with
task-done PLAN 3 BASE -- echo "SQL tests passed". - The ledger shows
Task 3: complete (…, tests: echo 'SQL tests passed' → SQL tests passed), indistinguishable from a real test run.
Proposal
Add a second form, for example task-done --verified-by "<tool> — <evidence summary>" PLAN N BASE,
that records verified-by: <…> in a format visibly different from tests: <command> → <result>.
Have the skill's completion contract name it as the only allowed route for non-shell verification.
Principle
A ledger that proves completion must distinguish evidence the script observed from evidence the agent reports. When both look identical, the weaker one silently inherits the stronger one's credibility.
Related (not duplicated here)
The ledger also records the last log line as the "result", which for some runners is a duration line rather than a pass count — already tracked in #2342 and PR #2348.
Activity
0No activity yet.