Malhar Ujawane

Blog

Post·

What are reasoning models? GPT-5.1 and the cost of thinking

Reasoning models write hidden thinking tokens before answering, billed as output. How GPT-5.1's reasoning effort works, with a runnable cost script.

What are reasoning models? GPT-5.1 and the cost of thinking

A reasoning model writes a hidden chain of thought before it answers, and you pay for those hidden tokens at the output price. GPT-5.1, which reached OpenAI’s API on November 13, 2025, makes that trade easier to control: it thinks less on easy questions and can skip thinking entirely when you set reasoning_effort to none.

What changed with GPT-5.1?

OpenAI shipped GPT-5.1 to ChatGPT on November 12 and to the API a day later. For builders, the API post has four changes worth knowing:

  • Adaptive reasoning. The model varies how long it thinks based on how hard the task looks. OpenAI’s example: asked for an npm command to list global packages, GPT-5 at medium effort used about 250 tokens and 10 seconds, and GPT-5.1 used about 50 tokens and 2 seconds.
  • A none effort level, which is the default. GPT-5 added minimal in August; GPT-5.1 goes one step lower, and OpenAI suggests low or medium for harder tasks and high when reliability matters more than speed.
  • 24-hour prompt caching with prompt_cache_retention='24h'. Cached input stays 90% cheaper than uncached input.
  • Same price as GPT-5: $1.25 per million input tokens and $10 per million output tokens.

It also adds apply_patch and shell tools for coding agents, which deserve their own post.

Why should a builder care about hidden tokens?

Because they decide your bill and your latency, and you can’t read them. OpenAI’s reasoning guide says reasoning tokens “are not visible via the API” but “still occupy space in the model’s context window and are billed as output tokens.” Output tokens cost eight times as much as input tokens on GPT-5.1, so a model that thinks for 1,000 tokens to write a 150-token answer is mostly charging you for work you never see.

Where do the thinking tokens go?

The usage object tells you. Every Responses API reply includes output_tokens_details.reasoning_tokens, and the guide shows a sample. This script prices that sample at GPT-5.1 rates, then prices OpenAI’s npm example, then simulates the one failure I’d want every team to know about. It needs no API key.

# reasoning_cost.py: where the money goes when a model thinks.
# Python 3.13, no dependencies, no API key. Prices: GPT-5.1 in the API,
# USD per 1M tokens, same as GPT-5 (OpenAI, 13 Nov 2025).
IN, CACHED, OUT = 1.25, 0.125, 10.00

def bill(usage):
    inp = usage["input_tokens"]
    cached = usage["input_tokens_details"]["cached_tokens"]
    out = usage["output_tokens"]
    hidden = usage["output_tokens_details"]["reasoning_tokens"]
    usd = (inp - cached) * IN / 1e6 + cached * CACHED / 1e6 + out * OUT / 1e6
    return usd, hidden, out - hidden

# 1. The sample usage object from OpenAI's reasoning guide.
usage = {"input_tokens": 75, "input_tokens_details": {"cached_tokens": 0},
         "output_tokens": 1186, "output_tokens_details": {"reasoning_tokens": 1024}}
usd, hidden, visible = bill(usage)
print(f"visible output {visible:>5} tokens")
print(f"hidden reasoning {hidden:>3} tokens ({hidden / usage['output_tokens']:.0%} of output)")
print(f"cost ${usd:.6f}, of which reasoning ${hidden * OUT / 1e6:.6f}")

# 2. OpenAI's launch example: one npm question, approximate output tokens.
print()
for label, out_tokens in [("GPT-5 medium", 250), ("GPT-5.1 medium", 50)]:
    per_call = out_tokens * OUT / 1e6
    print(f"{label:<15} ~{out_tokens:>3} tokens  ${per_call:.4f}/call  "
          f"${per_call * 100_000:,.0f} per 100k calls")

# 3. The max_output_tokens trap (a simulation of the documented behaviour):
# the cap counts reasoning too, so thinking can use it all up.
print()
cap, wants_to_think = 300, 1024
spent_thinking = min(cap, wants_to_think)
answer_tokens = cap - spent_thinking
print(f"max_output_tokens={cap}: thinking {spent_thinking}, answer {answer_tokens}, "
      f"status={'incomplete' if answer_tokens == 0 else 'completed'}, "
      f"billed ${cap * OUT / 1e6:.4f} for no answer")

Output:

visible output   162 tokens
hidden reasoning 1024 tokens (86% of output)
cost $0.011954, of which reasoning $0.010240

GPT-5 medium    ~250 tokens  $0.0025/call  $250 per 100k calls
GPT-5.1 medium  ~ 50 tokens  $0.0005/call  $50 per 100k calls

max_output_tokens=300: thinking 300, answer 0, status=incomplete, billed $0.0030 for no answer

In the sample, 86% of the output is thinking and 86% of the cost is thinking. The npm comparison uses OpenAI’s own rounded token counts, so treat the $250 versus $50 as an illustration of the shape, since your prompts will think differently.

Setting the effort is one field on the request. This is the request shape for the Python SDK; it needs a key, so there’s no output to paste:

# openai==2.8.0
from openai import OpenAI

client = OpenAI()
resp = client.responses.create(
    model="gpt-5.1",
    reasoning={"effort": "none"},   # or "low", "medium", "high"
    input="Classify this ticket as billing, bug or other: 'I was charged twice.'",
)
print(resp.usage.output_tokens_details.reasoning_tokens)

How do reasoning models work?

Start with a plain language model: it predicts one token at a time, and every token it writes becomes input for the next one. Writing intermediate steps before the answer gives the model more computation on the way to that answer, which is why “think step by step” prompts helped older models.

A reasoning model is trained to do this by itself. OpenAI described its first one, o1, in September 2024 as “trained with reinforcement learning to perform complex reasoning,” producing “a long internal chain of thought before responding.” The same post reports that performance improved with more time spent thinking. The API hides that chain and returns either nothing or a summary of it.

So a request has three kinds of tokens: the input you send, the reasoning the model writes and hides, and the answer you see. The effort setting is a hint about how long the middle part should be. It’s a hint, so the model still decides, and GPT-5.1’s change is that it decides more aggressively in both directions.

Where does reasoning break, and what should you do?

The worst surprise is max_output_tokens. That cap counts reasoning plus the answer, so a low cap can be used up entirely by thinking. The guide says you then get status: "incomplete" with reason max_output_tokens, possibly “before any visible output tokens are produced,” and you still pay for the input and reasoning. OpenAI recommends reserving at least 25,000 tokens for reasoning and output while you experiment.

Two bars. Top bar, max_output_tokens 300: an orange reasoning segment fills the entire 300-token cap, status incomplete, 300 output tokens billed, empty answer. Bottom bar, max_output_tokens 25,000: 1,024 reasoning tokens in orange, 162 answer tokens in black, the rest of the budget dashed as unused and not billed.

The second problem is latency you can’t see coming. A user waiting on a reasoning model watches a spinner while it thinks. For chat-style interfaces, none or low keeps responses fast, and preambles (short visible updates the model writes before tool calls) give the user something to read.

The third is that more thinking is sometimes wasted. Classification, extraction and formatting rarely improve with reasoning, and OpenAI’s own GPT-5 post noted that reasoning above low added little on a simple long-context retrieval benchmark. Agents that plan, call several tools and recover from errors are where reasoning earns its cost.

My starting point, as an opinion: none for classification, extraction and formatting, low for agents with tools, medium for coding agents, and high only on tasks where a wrong answer costs more than the wait. Then log reasoning_tokens per request and move the setting based on what you see. If you run several attempts in parallel, like the best-of-N worktree setup I wrote about recently, the reasoning cost multiplies by N. Shrinking what goes into the context helps too, which is the point of code execution with MCP.

FAQ

Can I see a reasoning model’s thinking?

No, not the raw tokens. OpenAI returns the count in the usage object and can return a model-written summary if you ask for one with the summary reasoning parameter.

Do I pay for reasoning tokens?

Yes. They’re billed as output tokens, at $10 per million for GPT-5.1 as of November 13, 2025.

Is reasoning effort none the same as a non-reasoning model?

OpenAI says none makes GPT-5.1 “behave like a non-reasoning model for latency-sensitive use cases,” and reports it’s better at parallel tool calling than GPT-5 at minimal. Whichever level you pick, reasoning_tokens in the usage object shows what you actually paid for.

Is GPT-5.1 cheaper than GPT-5?

Per token, no: prices are identical. Per task it can be cheaper when adaptive reasoning spends fewer tokens on easy requests, and more expensive on hard ones where it thinks longer.

Sources

Get new posts in your inbox

© 2026 Malhar Ujawane. Built with Astro.