Test-Time Compute

Test-time compute is computation spent while a model is answering a specific query, rather than while it is being trained. Instead of only asking “how big should this model be, and how much data should it see during training,” test-time compute asks a second question for every individual request: “how much extra thinking, sampling, or searching should this one query get before it produces a final answer.” A model can sample the same prompt eight times and vote on the most common answer, generate a long internal chain of reasoning before committing to a response, or search over a tree of partial solutions and prune the ones a verifier scores poorly, all without changing a single weight. This matters because it decouples quality from training-time scale: a smaller, cheaper-to-train model given enough test-time compute can match or beat a much larger model on the same problem, and unlike training compute, test-time compute can be spent selectively, more on a hard competition-math problem, none at all on “what’s the capital of France.” Chain-of-thought prompting was the first widely used form of this idea; the current generation of reasoning models (OpenAI’s o1/o3, DeepSeek-R1, and others) turned it into a first-class, trainable capability rather than a prompting trick.

Why Spend Compute at Inference Instead of Training

For years, the dominant scaling story in language models was “bigger model, more data, more training FLOPs.” Test-time compute is a distinct, complementary axis: for a fixed model, spending more compute per query, not per training run, can buy accuracy that would otherwise require a much larger model. The 2024 paper “Scaling LLM Test-Time Compute Optimally” by Snell, Lee, Xu & Kumar made this precise: on a FLOPs-matched comparison, a compute-optimal test-time strategy let a smaller base model outperform a model roughly 14x larger on problems where the smaller model already had some non-trivial chance of getting the right answer. The catch in that same paper is important: test-time compute works best on problems the model is not hopelessly bad at to begin with; sampling harder isn’t a substitute for a model that has no idea where to start, and the paper’s compute-optimal strategy adapts how much extra compute to spend based on how hard the specific prompt appears to be, rather than spending the same fixed budget on every query.

Sampling-Based Methods: Best-of-N and Self-Consistency

The simplest family of test-time compute methods just samples more than once. Best-of-N generates N independent candidate answers (usually with some sampling temperature so they actually differ) and picks the best one according to some scoring signal, a reward model, a verifier, or, in the absence of either, majority vote. Self-consistency, introduced by Wang et al. (2022), is the majority-vote special case applied specifically to chain-of-thought reasoning: sample N different reasoning paths, extract the final answer from each, and return whichever answer appears most often, on the intuition that a genuinely correct answer tends to be reachable via more distinct lines of reasoning than an incorrect one.

# Self-consistency for chain-of-thought reasoning.
# Scenario: a grade-school math tutoring bot where a single greedy
# chain-of-thought sometimes makes an arithmetic slip; sampling several
# independent reasoning paths and voting catches most of those slips.
from collections import Counter

def self_consistency(model, question, n_samples=10):
    votes = []
    for _ in range(n_samples):
        reasoning = model.generate(
            f"Question: {question}\nLet's think step by step.",
            temperature=0.7,
        )
        votes.append(parse_final_numeric_answer(reasoning))
    return Counter(votes).most_common(1)[0][0]

Both approaches share the same cost profile: quality improves as N grows, but with sharply diminishing returns, doubling N rarely doubles accuracy, and past some point the marginal sample almost never flips the majority vote. That diminishing-returns curve is exactly what the widget below plots.

Search-Based Methods: Beam Search, Tree-of-Thought, and Verifier-Guided Search

Sampling independently and voting throws away a lot of information: every candidate is generated from scratch, so a promising partial reasoning path that goes wrong halfway through gets no credit for the good half. Search-based methods instead build a solution incrementally and use a scoring signal to decide which partial paths are worth continuing. Beam search keeps the top-k partial sequences at each step, scored by the model’s own log-probabilities. Tree-of-Thought generalizes this to explicit branching and backtracking over intermediate “thoughts” rather than raw tokens, allowing the search to abandon an entire line of reasoning and try another. The most powerful variant, verifier-guided search (including Monte Carlo Tree Search, MCTS, applied to reasoning steps), uses a separate model, a process reward model (PRM) that scores individual reasoning steps rather than only final answers, to decide which partial solutions to expand and which to prune early.

# Verifier-guided beam search over reasoning steps.
# Scenario: an automated proof assistant for a coding-competition judge,
# where wrong intermediate steps should be pruned before wasting compute
# extending them into a full, still-wrong solution.
def verifier_guided_search(model, prm, prompt, beam_width=4, max_steps=6):
    beams = [(prompt, 0.0)]  # (partial_text, cumulative_score)
    for _ in range(max_steps):
        candidates = []
        for partial_text, score in beams:
            next_step = model.generate_next_step(partial_text, temperature=0.9)
            step_score = prm.score_step(partial_text, next_step)  # 0..1 quality estimate
            candidates.append((partial_text + next_step, score + step_score))
        # Keep only the top `beam_width` partial solutions, drop the rest
        beams = sorted(candidates, key=lambda c: c[1], reverse=True)[:beam_width]
    return max(beams, key=lambda c: c[1])[0]

Verifier-guided search is more expensive per candidate than plain best-of-N (it needs a trained PRM, and it evaluates every intermediate step, not just the final answer) but is dramatically more compute-efficient because it stops extending bad paths early instead of running them to completion before discovering they were wrong.

graph TB
    classDef default fill:#ffffff,stroke:#4338CA,stroke-width:2px,color:#0F172A,rx:8px,ry:8px;
    classDef data    fill:#EEF0F7,stroke:#0D9488,stroke-width:2px,color:#0F172A,rx:8px,ry:8px;
    classDef process fill:#F7F8FC,stroke:#6366F1,stroke-width:2px,color:#0F172A,rx:8px,ry:8px;
    classDef output  fill:#4338CA,stroke:#4338CA,stroke-width:2px,color:#ffffff,rx:8px,ry:8px;

    P([Prompt]):::data --> S["Sample N candidate reasoning paths"]:::process
    S --> C1[("Candidate 1")]:::data
    S --> C2[("Candidate 2")]:::data
    S --> C3[("Candidate N")]:::data
    C1 --> V["Verifier / reward model scores each candidate"]:::process
    C2 --> V
    C3 --> V
    V --> SEL["Select highest-scoring (or majority) answer"]:::process
    SEL --> O([Final answer]):::output

Long-Chain Reasoning Models: Test-Time Compute as a Trained Behavior

The most recent shift is folding test-time compute directly into how a model is trained, rather than bolting it on at inference time via an external sampling loop. OpenAI’s o1 and o3 models, and DeepSeek’s open-weight R1, are trained (via reinforcement learning, in DeepSeek-R1’s case using verifiable rewards from math and code problems whose correctness can be checked automatically) to generate a long internal chain of reasoning before producing a final answer, and to allocate more of that reasoning to harder problems on their own, without an external search loop deciding N or beam width for them. OpenAI reported that o1’s performance improves both with more reinforcement learning at training time and with more thinking time at test time, treating the two as separate, additive scaling axes. DeepSeek-R1 showed that this reasoning behavior, including self-reflection and revisiting earlier steps mid-answer, can emerge from reinforcement learning with a verifiable reward signal alone, without a human-labeled dataset of reasoning traces.

# Illustrative sketch of what a long-chain reasoning model does
# differently from a plain chat model at inference time.
# Scenario: a coding-interview practice tool where a hard algorithm
# question should get much more internal reasoning than "what does
# this one-line function do."
def generate_with_adaptive_reasoning(model, prompt):
    # The model itself decides how many internal reasoning tokens to
    # spend, conditioned on how hard the prompt appears, rather than
    # a fixed external loop choosing N or beam width.
    response = model.generate(
        prompt,
        reasoning_effort="auto",  # model allocates thinking tokens per-query
    )
    return response.final_answer

The Compute Trade-off

Every method above spends inference compute to buy accuracy, and every one of them shows the same diminishing-returns shape: the first few extra samples, search steps, or reasoning tokens help a lot, and each additional unit of compute after that helps less. The practical question for a production system is almost never “should we use test-time compute” but “how much, on which queries.” Drag the slider below to see how accuracy on a hard-question benchmark (illustrative curve, shaped after the diminishing-returns pattern reported in the Snell et al. paper) responds as the sample count N in a best-of-N setup grows.

Illustrative curve shaped after the diminishing-returns pattern reported for best-of-N sampling; exact numbers vary by model and task. Relative inference cost scales roughly linearly with N, while accuracy gains flatten out well before N reaches its upper range.

The shape of that curve is the whole argument for a compute-optimal, per-query strategy: spending N=64 samples on an easy query wastes 64x the cost for almost no accuracy gain over N=1, while spending only N=1 on a genuinely hard query leaves real accuracy on the table. This is exactly what the Snell et al. compute-optimal method tries to solve, estimating how hard a given prompt is and allocating more or less test-time compute accordingly, rather than using one fixed N for every request.

Verifiers and Reward Models

Every search-based and best-of-N method needs some way to judge candidate quality, and the choice of verifier matters as much as the search strategy itself. An outcome reward model (ORM) scores only the final answer; a process reward model (PRM) scores each intermediate reasoning step, which is more expensive to train (it needs step-level labels or a way to generate them) but catches errors before a search wastes compute extending a broken partial solution. In domains with automatically checkable answers, math with a known numeric result, code with unit tests, the verifier can simply be the checker itself: run the candidate’s code against the test suite, or compare the final number against ground truth, no learned reward model required. DeepSeek-R1’s use of automatically verifiable rewards (correct/incorrect on math and code problems, checkable by a program) rather than a learned reward model is a big part of why it could be trained with pure reinforcement learning at meaningfully lower cost than a pipeline dependent on human-labeled reasoning traces.

Comparing the Approaches

What’s New (2025-2026)

• Long-chain reasoning has become the default framing for test-time compute, rather than one technique among several. OpenAI’s o1/o3 and DeepSeek-R1 folded sampling-and-search behavior into the model itself via reinforcement learning, so a single forward pass with a longer internal reasoning trace replaces what used to require an external best-of-N or search loop.
• Verifiable rewards have displaced learned reward models in several open efforts, following DeepSeek-R1’s demonstration that reasoning behavior (including spontaneous self-correction) can emerge from RL against automatically checkable math/code answers alone, without a curated reasoning-trace dataset.
• Adaptive, per-query compute allocation is now a design goal, not just a research finding. Production reasoning models increasingly expose a controllable “reasoning effort” setting so a caller can trade latency and cost for accuracy on a per-request basis, echoing the compute-optimal, difficulty-aware allocation strategy from Snell et al.
• Process reward models are being used more for search-time pruning than for standalone scoring, since catching a wrong intermediate step early is far cheaper than generating and later discarding a full wrong solution.
• Test-time compute is increasingly discussed alongside agent evaluation, since multi-step agentic tasks (tool calls, multi-turn planning) are a natural setting for the same sample-and-verify or search-and-prune ideas, just applied across an entire trajectory rather than a single answer.

Summary

Test-time compute reframes model quality as something a system can spend more or less of per query, not just something fixed once at training time. Sampling-based methods (best-of-N, self-consistency) trade inference cost for accuracy with diminishing returns as N grows; search-based methods (beam search, tree-of-thought, verifier-guided search with a process reward model) spend that compute more efficiently by pruning bad partial solutions early instead of running them to completion; and the newest reasoning models (o1/o3, DeepSeek-R1) fold the decision of how much to think directly into the model via reinforcement learning, often against automatically verifiable rewards. The practical takeaway echoed across the Snell et al. results, OpenAI’s o1 reporting, and DeepSeek-R1’s training recipe is the same: test-time compute is most valuable when it is spent adaptively, more on the queries that need it, none on the ones that don’t, rather than applied uniformly regardless of difficulty.

# Scenario: a homework-help API where a subset of queries are flagged
# as "hard" (competition math, multi-step proofs) and the product is
# willing to pay for extra latency/cost on just those queries.
from collections import Counter
import re

def best_of_n_with_verifier(model, prompt, n=8):
    candidates = [model.generate(prompt, temperature=0.8) for _ in range(n)]
    answers = [extract_final_answer(c) for c in candidates]
    # Majority vote acts as a cheap "verifier" when no reward model exists
    winner, _ = Counter(answers).most_common(1)[0]
    return winner

def extract_final_answer(text):
    match = re.search(r"final answer:\s*(.+)", text, re.IGNORECASE)
    return match.group(1).strip() if match else text.strip()

is_hard = classify_difficulty(user_query)  # cheap upstream classifier
n = 8 if is_hard else 1
answer = best_of_n_with_verifier(model, user_query, n=n)