#798

Recipe: turn a bad prompt into a great one — automatic prompt scoring + repair with Claude

OpenAmritha902opened 3mo ago

The gap

Every new Claude user writes weak prompts at first — but the hard part isn't fixing them, it's knowing what's wrong. The cookbook has great evaluation recipes, but none that take a messy real-world prompt, diagnose it, and repair it. I'd love to contribute that recipe.

What it does (before → after, in one notebook)

Input: "fix my resume idk make it good" Output: a scored diagnosis + a rewritten, structured prompt that measurably scores higher.

Concrete outline

  1. Score — Claude grades a draft against 5 best-practice dimensions (role, context, specificity, format, structure) and returns a structured JSON rubric with a per-dimension reason for any low score.
  2. Repair — a single "prompt compiler" call rewrites the draft into a well-structured prompt, preserving the user's intent.
  3. Prove it worked — re-score the rewritten prompt and show the lift; run it over a small batch to report average improvement (a real eval loop, not a vibe check).
  4. Make it reusable — wrap it in a drop-in score() / improve() helper readers can paste into their own apps.

Why it's worth merging

  • Fills a genuine gap (diagnose-and-repair, not just evaluate).
  • Immediately practical — anyone building on Claude wants "make my users' prompts better."
  • Self-contained, matches your existing notebook style, easy to review.

Why I can deliver it cleanly

I already built these exact ideas into a working, open-source browser extension — with 100+ passing tests, reproducible benchmarks, and design docs: https://github.com/Amritha902/promptcomplete I'll adapt the concepts into a polished, standalone notebook.

Try it live (no install): https://claude.ai/code/artifact/4d15fab9-3403-4cba-b916-a4442b0768b7

Does this fit, and is anything similar already planned? Happy to tune the scope to whatever you'd find most useful. I can have a draft ready quickly.

Activity

0

No activity yet.