Reasoning Models and Test-Time Compute: Buying Time to Think
Reasoning models generate thinking steps before answering. What test-time compute buys you, when it is wasted, and how it changes latency and cost.
For a long time a model’s capability was set during training: how much data, how many parameters. A new axis has been added — how much the model thinks while answering. Reasoning models generate internal steps before replying, and the compute spent on those steps can raise accuracy directly.
What test-time compute is
A classical model takes the question and starts producing the answer immediately. A reasoning model first writes out a solution process internally: it decomposes the problem, produces intermediate results, checks itself, and only then answers. Those intermediate steps may be hidden from the user, but they are generated as tokens and they are billed.
Why it works
An answer produced in one shot is the model’s first instinct. Intermediate steps create a chance to catch errors: once a wrong assumption is written down, the model can notice and correct it. The gain is clearest on verifiable problems — mathematics, code and tightly constrained planning.
Where it pays and where it is waste
| Task | Reasoning |
|---|---|
| Multi-step maths, code | Clear gain |
| Constraint solving, planning | Gain |
| Document summarisation | Small or none |
| Classification, extraction | Waste |
| Creative writing | Variable |
The general rule: reasoning pays when the answer can be checked step by step. For one-shot outputs judged subjectively, the extra cost buys little.
Latency and cost
Thinking steps produce tokens, and tokens mean both money and time. A request can move from a few seconds to tens of seconds. That may be unacceptable in an interactive interface and entirely reasonable in a background analysis.
Reasoning should therefore be designed as a switch: off for easy requests, on for hard ones.
Consequences for product design
Long response times must be handled honestly in the interface. Showing users what is happening and why they are waiting lands far better than a silent progress bar. Streaming and a cancel option are close to mandatory for long operations.
Frequently asked questions
Should I show the thinking steps to users?
Showing a summary builds trust. Showing the raw chain can be both confusing and misleading; that text is not the final answer.
Is this the same as chain-of-thought prompting?
A similar idea in a different place. One is encouraged through the prompt; the other is built into the model's training and tends to be more consistent.
Does it eliminate hallucination?
No. It reduces it without removing it; a model can still defend a wrong step convincingly. Grounding in sources remains necessary.
Test-time compute is a new lever that lets you buy quality with money — but only on the right problems. For more, see reducing hallucinations and model routing.
