What is Test-Time Compute?

Test-time compute is the computing a trained AI model spends while answering a question. Computing means the arithmetic the chips do, and more of it costs electricity and time. Engineers call running a finished model a test of it, so test time is answer time, as opposed to training, the one-off run that built the model. The same model gets hard questions right more often when it is allowed more arithmetic per answer. There are three ways to allow it: reasoning at more length before answering, producing a batch of answers and going with the most common, or having a separate checker rank its attempts. OpenAI showed the last two on a 15-question maths competition exam in 2024. One model answering each question once scored 74%. Allowed 64 answers per question, with the most common taken, it scored 83%. Allowed 1,000 answers, with a checker picking the best, it scored 93%. The model never changed. The bill did: every extra attempt is arithmetic, paid in seconds on the clock and in the price of the answer.

Test-time compute is the arithmetic an AI model does at the moment it answers, as opposed to the arithmetic that trained it. Spending more of it per question gets better answers out of a model that is already built. Computing is measured in operations, because operations are what cost chip hours and electricity, and the wait follows from them. Engineers call running a finished model a test of it, hence the name. Training set the model once, and nothing here changes what training built. Spending more at answer time takes one of three forms. It can write out longer reasoning before it answers, which is chain-of-thought: its answer comes out word by word, so more words before the final one is more room to work, check and retry. It can produce several answers to the same question and go with the most common, so a chance slip in one attempt is outvoted by the rest. Or a separate checker, trained to tell good attempts from bad, can rank its attempts and pick one. Looping, where a model runs its own layers again before any word appears, is a fourth. A 2024 study put a budget on it. Compare a modest model with one 14 times larger. The large one does roughly 14 times the arithmetic for every word, so for the same total operations the small one can produce roughly 14 times as many words or attempts. Spent on more attempts and revisions, with a checker choosing among them, the modest model's budget beat the large model's, but only on questions the modest model could manage some of the time. On questions beyond it, more attempts failed the same way, only slower. Nobody looks at the answer key in any of this: the vote and the checker work from the attempts alone. The cost lands per question, at inference: each extra word and attempt is arithmetic, which is why thinking answers take longer and are priced higher. That makes two dials. Scaling laws describe the first, set before the model exists: its size and how much it was trained on. Test-time compute is the second, set per question, after the model exists and without changing it.

Test-time compute is the computing a trained AI model spends per question, and giving the same model more of it, more attempts or more reasoning, raises its score up to a point. Parallel parking works the same way. An experienced driver slots into a tight space in one go. A newer driver, given three attempts, usually gets in too, with no more skill than they had on go one. The extra attempts pay on a space the newer driver can nearly manage. A space that is simply too short stays empty however many goes they take, and each go costs time and holds up the street. The driver's skill is the model's size and training, the attempts are the test-time compute, the nearly-manageable space is the question within reach, and the too-short space is the question beyond it. The match ends here: a driver can see which attempt worked, while a model has to be told by a vote or a checker.