Same score.
Different failure.
How the benchmark works, step by step—and why recorded money errors with the same strict penalty can mean different things.
English narration and visible captions. No login or subscription is needed. The original 84-answer study and the separate 72-answer targeted follow-up remain distinct. Neither establishes general model superiority.
Read the English transcript
Two models got the same score, but their mistakes told completely different stories.
Twenty-eight fictional cases tested field normalization. Three model IDs produced eighty-four answers. Gemini scored twenty-eight; Claude and Gemma, twenty-six.
Explicit rules in. Raw responses out. Code checks JSON, status, and canonical value. Native traces preserve each prompt and answer.
Take the text four dot seven zero zero. The declared Brazilian convention makes that dot a thousands separator. Claude called it ambiguous. Gemma returned the right amount, but omitted the required decimal places. Both lost a strict point.
After those results, we registered twelve new money cases before collecting answers. Same three model IDs, two repetitions: seventy-two responses, separate from the original eighty-four.
For three dot six zero zero, the source declares a decimal point. Claude abstained again. Gemma omitted a required zero in repetition one, then passed this case in repetition two.
Gemini scored twelve twice. Claude scored seven, then six. Gemma, eleven, then twelve. Claude had seven unnecessary abstentions, three JSON-container violations, and one wrong amount. All twenty-four control responses passed.
These targeted cases do not establish general model superiority. Values and wording changed together. Inspect the raw answers and grading code. Same score. Different failure.
This is an AI-assisted editorial explanation of fictional benchmark cases, not a new measurement, human audit or blinded review.