`misc/building_evals.ipynb`: model-based grader uses the ineffective “reason first, verdict second” ordering, with a stray `=` also present in the prompt.

Two things in the model-based grading section, one substantive and one trivial. Happy to PR either or both.

1. The grader ordering. build_grader_prompt puts the answer and the rubric into a single user message, then asks the model to reason inside <thinking> tags before emitting <correctness>. Recent work measures that ordering specifically: a judge that sees the candidate and then reasons toward a verdict accepts wrong answers at a 0.719 false-positive rate; the same judge that commits its own answer first, before the candidate enters context, drops to 0.012 (Zhou, arXiv:2607.05904).

I surveyed the default judge templates shipped across eval frameworks and found none implement the commit-first ordering, with nine sharing one ancestor prompt — arXiv:2609.00088, corpus and scorer at https://github.com/idilgozel/evaluator-integrity. Since this notebook is where a lot of people first learn the pattern, the ordering here probably propagates further than most.

Worth saying: cell 17 already gives the right instinct — "the only way to know if a model-based grader can do a good job grading your task is to try... read some samples." My finding is that this is cheaply measurable rather than a matter of eyeballing. Measure the grader's own accuracy on the task before trusting it to grade the task; in my runs a judge whose committed answer was wrong made gaming worse under the standard fix, and that was predictable in advance for pennies.

2. A stray character. The prompt string contains:

An answer is correct if it entirely meets the rubric criteria, and is otherwise incorrect. =

The trailing = looks unintentional and ships into the model's context.

I'd propose a PR adding a commit-first variant alongside the existing grader (not replacing it — the current one is fine for rubric-matching against a golden answer, and the two-call version costs more), plus the typo fix. Say the word and I'll open it.

Disclosure: I'm building a company in evaluator conformance, so I have a commercial interest. The paper and corpus are public and CC-BY regardless.