LLM as a Judge

Using one model to evaluate another sounded like an elaborate way to automate confirmation bias. But especially for teams built around human data work in AI training, that idea was hard to trust.

The pressure came from volume. A programme with 10,000 tasks does not end when the last annotation is submitted. Those tasks still need review, error analysis and calibration. At scale, quality assurance can consume nearly as much effort as production itself. Add multi-turn conversations, code execution or agent trajectories, and the review burden grows even faster.

The first response was selective automation. Environment checks, format validation, deterministic labels and obvious policy violations could be handled without asking a reviewer to inspect every item. The harder question was whether a model could assess the parts that depended on judgement.

The original MT-Bench paper was a turning point. It reported that GPT-4 agreed with human preferences more than 80 per cent of the time, roughly matching the agreement observed between human evaluators in its experiments.

The result was enough for teams to begin testing the approach seriously, even if few were ready to trust it unattended.

Once teams began using LLM judges more widely, they also began finding the defects. Judges could favour longer answers, change their verdict when candidate positions were swapped and prefer outputs that resembled their own model family. Research on position bias, self-preference and broader judge reliability has since mapped many of these failures.

Teams kept building on LLM-as-a-judge anyway, for the same reason they'd adopted it in the first place. The volume problem hadn't gone anywhere. What changed was the question in front of them, from whether the approach worked at all to how to keep it honest at scale, without letting a model's confidence pass for authority.

The best judge changes with the task

I have used frontier models across annotation programmes, preference evaluation and agentic task review. Claude, GPT and Gemini models have all been useful in different settings. None of them is the obvious answer for every workflow. Model selection begins with the rubric, evidence format, risk level and budget of the workflow, not a leaderboard position.

Model family	Where it tends to work well	What needs watching
Claude	Long trajectories, detailed rubrics and qualitative review	Over-interpretation, self-preference and whether the reasoning supports the score
GPT	Structured grading, schema-constrained outputs and cross-checking	Score compression, framing sensitivity and preference for familiar output styles
Gemini Flash tier	High-volume screening and lower-cost repeated passes	Nuanced criteria and longer dependency chains
Open-weight models	Private deployments and controlled environments	Calibration drift and weaker generalisation outside the trained domain
Specialised judge models	Stable, repeated evaluation workloads	Performance beyond the data used to train them

These are working observations rather than fixed rankings. Model versions change. Prompt behaviour changes with them. A judge that performs well on response quality may behave very differently on code or a fifty-step trajectory. The sharpest split I've actually seen wasn't between model families so much as between task types. A model that enforced a rubric strictly on chat-style quality checks needed a rewritten prompt before it would apply the same rigour to a code diff.

In complex evaluations, I usually avoid treating one verdict as final. A stronger model can handle the primary review, while another model checks targeted slices or borderline cases. Humans then resolve disagreement and high-risk decisions.

Agreement rates become useful only when teams examine where the judges diverge and which verdict human reviewers consistently uphold. Two judges can agree because the answer is genuinely good, or because they share the same blind spot, and only the disagreement cases tell you which.

The rubric decides what the model can see

When the rubric is vague, the judge fills in the missing definition itself, usually without telling you which assumptions it made.

Before choosing the model, the task itself needs to be scoped properly. That means mapping the directions the task can take, its common failure modes, and the evidence needed to tell partial success from genuine completion.

A criterion such as "rate the quality from one to five" pushes all of that work onto the judge. It must decide what quality means, how much each dimension matters and what separates one score from the next. The number may look precise while hiding several assumptions. The fix is to make the prompt carry every rubric dimension the task actually needs, not just the headline instruction. If quality is really several criteria bundled together, the prompt should state each one, assign it a weight, and give the judge explicit instructions for computing the weighted score itself, rather than leaving that arithmetic to inference.

For an agentic coding task, the evaluation can be separated like this.

Dimension	Evidence	What the judge decides
Functional result	Test output and final artefact	Did it work?
Requirement coverage	Task specification and artefact	Was the request completed?
Trajectory quality	Tool calls and file changes	Did the agent make defensible progress?
Constraint adherence	Environment policy and trajectory	Did it stay within the rules?
Code quality	Diff or repository state	Is the solution maintainable?
Side effects	Dependencies and environment changes	Did completion introduce avoidable risk?

This structure matters because agentic tasks can fail in several ways at once. A solution may pass the tests but violate a constraint. A trajectory may be sound while the final implementation misses one requirement. A single score compresses those differences too early.

The prompt should therefore behave like an executable rubric. Each criterion needs a clear definition, boundary examples and instructions for ambiguous cases. The judge also needs the right evidence. Asking whether an application works without supplying execution results turns evaluation into guesswork with good formatting.

Different domains need different levels of trust

Once the rubric is settled, how much of the judgment can move to a model still depends on the domain, and specifically on what a wrong verdict costs.

In SWE-bench-style evaluation, a large part of the workflow can be automated. Environment setup, dependency checks, test execution and repository state can be verified through the harness. An LLM judge becomes useful for the parts that deterministic checks handle poorly, such as unnecessary complexity, suspicious workarounds or whether the trajectory reflects a credible attempt to solve the task.

For more on how this plays out in agentic systems specifically, see my earlier post on SWE-bench annotation pipelines.

For preference data, the requirement is different. The evaluator needs to understand the prompt, compare the intent of two responses and rank them against a defined set of criteria. Here, semantic judgement matters more than environment verification. A judge may be assessing helpfulness, relevance, factuality or style, and those dimensions need to be kept separate if the resulting preference signal will feed an RLHF pipeline.

Medical annotation sits at another point on the spectrum. AI can help with routing, consistency checks and obvious omissions, but the sensitivity of the domain makes full automation difficult to justify. Human review remains central, especially where the decision depends on clinical context or could affect downstream care.

The design should follow the risk. Some tasks can be resolved automatically. Some can be scored automatically and audited by humans. Others should only use the model to flag likely issues for expert review.

The hidden cost decides the workflow

Accuracy alone doesn't decide whether a judging setup is viable. Teams also have to account for time, reviewer capacity, model cost and the expense of handling failures.

A larger model may score better on a complex rubric but become impractical across hundreds of thousands of items. A cheaper model may handle the first pass well enough if uncertain cases are routed elsewhere.

This is where cascades become useful. A faster judge clears the obvious cases and a stronger one reviews the uncertain band, with humans left for what's ambiguous, sensitive or high-stakes. The practical value comes from removing routine cases from the reviewer queue while preserving human attention for ambiguity, risk and disagreement.

In practice, I route an item to review when the automated verdict stops looking stable.

• Two judge passes disagree materially.
• The reasoning conflicts with the final score.
• The evidence is missing or contradictory.
• The task belongs to a category with weak historical agreement.

Those signals also reveal problems in the system itself. If the escalation rate is consistently high, the issue may be the rubric, the evidence or the judge selection. More automation will not repair an unclear task definition.

Human judgement moves higher in the stack

That escalation logic is really about where human effort moves, not whether it disappears.

Instead of scoring every item, reviewers focus on calibration, disagreement and risk. This raises the skill requirement. Auditing a judge means understanding the task well enough to identify a pattern rather than simply overturn one score.

Human decisions also need to flow back into the pipeline. Repeated overrides can show that a criterion is unclear, a model is weak on a particular slice or the evidence being supplied is incomplete.

The dashboard should track more than headline accuracy. Disagreement between judges, disagreement with humans, escalation rates, score distributions and reviewer overturns all matter. A rising overturn rate on one category usually means the rubric needs a rewrite before the model needs replacing.

Versioning matters as well. Every judgement should remain traceable to the model, prompt, rubric and evidence bundle that produced it. Cloud models change. Without that record, a shift in scores can be mistaken for data drift or reviewer inconsistency.

This becomes especially important in RLHF and preference-data programmes. A judge that consistently rewards verbosity, excessive caution or a familiar style does more than distort an evaluation report. It can shape the next training signal.

Systems such as Constitutional AI show why AI-generated feedback is attractive. It makes critique, revision and preference generation easier to scale. The operational responsibility is making sure those preferences were chosen deliberately, not inherited quietly from a judge that happened to prefer longer, more hedged answers and was never audited for it.

The system matters more than the model

There is no lasting answer to which LLM makes the best judge, and I've stopped expecting one. What holds up across every programme I've run is deciding upfront what a wrong verdict actually costs. A routine formatting check costs little. A trajectory that gets promoted into training data costs a lot. That decision, more than any benchmark, determines how much of the review queue can move to a machine and how much stays with a person.

For complex agentic work, I still prefer a strong primary judge, deterministic checks, targeted cross-model review and human escalation. For high-volume, well-bounded work, a faster model can take the first pass while a stronger model handles the uncertain band. For sensitive domains, the model stays in a supporting role.

Scale arrives with the model. Reliability comes from everything built around it. That's the rubric, the evidence bundle, the escalation rules, and the audit trail that lets you trace a bad score back to the prompt that produced it.