Agent Evaluation: How Do You Measure a Multi-Step System?
Evaluating an agent is not the same as scoring one answer. Task success, step quality, tool choice, cost and robustness testing in a practical agent evaluation framework.
Evaluating a single-turn model is comparatively easy: you compare the answer against a reference. An agent takes fifteen steps, calls three tools, hits an error, backtracks, and eventually produces something. Here “is it correct?” is not enough on its own — you also need to know what happened at which step.
Four layers of measurement
- Task success: did the user get what they asked for? This should be binary and uncontroversial.
- Trajectory quality: was the result reached by a sensible route, or by luck?
- Efficiency: how many steps, how many tokens, how long?
- Safety: did it exceed its authority or do something irreversible?
Outcome or trajectory
Looking only at the outcome is tempting and misleading. An agent that reaches the right answer by the wrong route collapses as soon as conditions shift slightly. The converse also holds: an agent whose route was sound and which erred on the final step is far easier to fix.
The practical approach records both: score the outcome as binary and log the trajectory so it can be inspected step by step.
Measuring tool choice
| Error type | Symptom |
|---|---|
| Wrong tool | Another tool is called when the right one exists |
| Missing call | The model invents instead of using a tool |
| Unnecessary call | An expensive tool is called when the answer is known |
| Bad parameters | Right tool, wrong arguments |
These four have different fixes: the first needs clearer tool definitions, the second a tighter instruction, the third routing, the fourth schema validation. You cannot improve what you have not separated.
Robustness tests
In the real world tools fail, APIs time out and documents come back incomplete. A good evaluation set includes these deliberately: a malformed tool response, an empty result set, two contradictory sources. What the agent does in those situations tells you more than its performance under ideal conditions.
The single most valuable test is this: what does it do when it does not know? An agent that invents is the most expensive agent in production.
Cost and latency are metrics
Accuracy alone is meaningless in agent evaluation. A version that is two percent more accurate and three times more expensive counts as a regression for most products. Every evaluation run should report average tokens and duration per task.
Frequently asked questions
Does LLM-as-judge work for agents?
For subjective dimensions such as trajectory quality, yes — but task success should be checked programmatically where possible. A vague judge produces a vague metric.
How many tasks are enough?
Fifty well-chosen tasks beat five hundred random ones. Diversify the coverage: easy, hard, adversarial, and tasks that should fail.
Why do agent results differ between runs?
Sampling is stochastic. Run each task several times and report a success rate; a single run is not a measurement.
Agent evaluation is the infrastructure that sets your development speed: a system you cannot measure can only be improved by guesswork. For more, see LLM evaluation and observability.
