Malhar Ujawane

Blog

Review·(updated October 29, 2026)

Claude 3.5 Sonnet: a coding model at Sonnet's price

Claude 3.5 Sonnet launched 21 June 2024 at $3 and $15 per million tokens, with Artifacts and a 64% internal coding score.

Claude 3.5 Sonnet: a coding model at Sonnet's price

Claude 3.5 Sonnet came out on 21 June 2024. It is the first model in the 3.5 family. Anthropic’s price is $3 per million input tokens and $15 per million output tokens, with a 200K context. The coding number in the same post is an internal agentic evaluation: Claude 3.5 Sonnet solved 64% of the problems, and Claude 3 Opus solved 38%. The model is free on Claude.ai and the iOS app.

I read the launch from a delivery seat that also sits on data platforms. The procurement change is a mid-tier price on the model I would actually hand a diff.

What did Anthropic ship on 21 June 2024?

Anthropic’s positioning is the sentence to keep next to the dollars. Claude 3.5 Sonnet outperforms competitor models and Claude 3 Opus on a wide range of evals, at the speed and cost of the mid-tier Claude 3 Sonnet, and at twice the speed of Claude 3 Opus. The card they printed is $3 and $15 per million. Opus’s dollar price is not a figure I have for this note, so the Opus comparison I will repeat is the speed claim and, below, the coding score.

It is free on Claude.ai and the iOS app. Pro and Team get higher rate limits. It is also in the Claude API, on Amazon Bedrock, and on Google Cloud Vertex AI. A team that already buys through one of those clouds can put the model on paper it already has. This announcement gives one pair of rates for the model.

The context window is 200K tokens, sold next to a coding model. Last month’s GPT-4o API window is 128k, at $5 and $15 per million. The March 2023 GPT-4 public window was 8,192 tokens, with limited access to a 32,768-token model beside it. A pile of source files fits under 200K and overflows 8,192. The ceiling still leaves the choice of files to the pilot. Twice the speed of Opus has no milliseconds attached. I would time the edit I care about. Their factor is the claim. A trace is the measurement.

They also say it is stronger on GPQA, MMLU, and HumanEval. The paragraphs this note uses give the direction and the three names, and no percentages beside them. The coding percentage below is the one they defined a task for.

What does the 64 percent score measure?

Anthropic’s figure for agentic coding is their own harness. Claude 3.5 Sonnet solved 64% of the problems. Claude 3 Opus solved 38%. The task is fixing a bug or adding a feature in an open source codebase, starting from a natural-language description. With tools, the model can write code, edit code, and execute code. Anthropic says it is good at code translation and legacy migration.

That harness is theirs. It is not SWE-bench. The comparison the post supports is inside the harness: 64 against 38, Sonnet against Opus, on this task. I would keep 64% in a sentence that names the harness, and I would keep it out of a public board.

Two scores from Anthropic’s internal agentic coding evaluation: Claude 3 Opus at 38 percent, Claude 3.5 Sonnet at 64 percent. A footnote says this is Anthropic’s own harness, not SWE-bench.

The task shape is why the number still matters. A bug or a feature, in words, in a repository, with the model allowed to write, edit, and run code, is a backlog item. “With tools” means the score includes the model acting. What I can take from the post is that definition, the two scores, and their judgment that it is good at code translation and legacy migration. I do not have a repo list, a sandbox, or a timeout to attach to the 64%. I would score a sample of the bugs I would actually hand over, on the tests that already gate them, before I treated 64% as a forecast. Their result is the mid-tier model ahead of their own Opus, 64 to 38.

Legacy migration is the phrase I would circle from a data-platform seat. Moving code between stacks is a real queue, and “good at” is the hypothesis. The check is paired and small: the same file, the same instruction, the diff read by someone who owns the destination stack. The 64% is a reason to run that sample.

What are Artifacts for?

Artifacts shipped the same day, as a preview on Claude.ai. Code, documents, and website designs open in a side window you can edit. Anthropic describes this as the move from a conversational AI to a collaborative workspace.

The result they are selling is an object in the workspace. A person edits the document, the page, or the code in the window. The deliverable is that object, rather than a chat bubble someone copies out and reformats. For a short spec, a one-page interface, or a draft a colleague will mark up, the window is the feature. I would try a document and a small page before I decided the sentence had landed.

The 64% harness is a different claim in the same post: an agent that writes, edits, and executes code in a codebase. Artifacts is the side window. I would grade them apart. An editable page is evidence about the window. The coding score is evidence about the harness.

The preview lives on Claude.ai, including the free app, and Pro and Team raise rate limits for people who stay there. A website design in that window and a private repository are different payloads. The API, Bedrock, and Vertex are the paths when the payload is a repository I have to account for. The free session is the path for an Artifacts trial on material I am already willing to put there.

How does the list price compare with GPT-4o?

The script prices one made-up agent turn against GPT-4o’s May list price. Thirty thousand tokens in, two thousand tokens out. GPT-4o is $5 per million input tokens and $15 per million output tokens. Claude 3.5 Sonnet is $3 and $15, with the 200K context on Sonnet. The comparison is list price. It is not a quality ranking, and it is not the 64% harness rewritten as dollars. No install, no network, no API key.

# sonnet_price.py — Python 3, no dependencies.
# GPT-4o, 13 May 2024: $5 / $15 per million.
# Claude 3.5 Sonnet, 21 June 2024: $3 / $15 per million. 200k context on Sonnet.
PRICE = {"GPT-4o": (5, 15), "Claude 3.5 Sonnet": (3, 15)}
inp, out = 30_000, 2_000  # one made-up agent turn
for name, (pin, pout) in PRICE.items():
    usd = inp * pin / 1e6 + out * pout / 1e6
    print(f"{name:<20} ${usd:.3f} per call   ${usd * 1000:,.0f} per 1,000")

Output (Python 3):

GPT-4o               $0.180 per call   $180 per 1,000
Claude 3.5 Sonnet    $0.120 per call   $120 per 1,000

On this basket GPT-4o prints at $0.180 a call, $180 for a thousand. Claude 3.5 Sonnet prints at $0.120 a call, $120 for a thousand. Output is $15 per million on both cards, so a turn with this much output costs the same on the output side. The gap is the input rate, $5 against $3, across 30,000 tokens of prompt. An agent loop that resends a long history feels that gap on every turn. A turn that is a short prompt and a long answer barely feels it, because the output rates match.

Thirty thousand tokens fits GPT-4o’s 128k context and Sonnet’s 200K context. It does not fit the March 2023 GPT-4 window of 8,192 tokens. It does fit that generation’s 32,768-token model, at the higher rate in the GPT-4 note. On these two cards the basket is an ordinary prompt, which is why a list-price script can ignore the window.

Replace inp and out with a trace before quoting $0.120. The price table came from the announcements. The basket is made up so the table has something to multiply. The script compares list price. The 64% harness is the quality claim, and it is not SWE-bench. A failed diff costs another attempt, which only shows up when you run the tasks and count them.

What I would do

I would hand Claude 3.5 Sonnet a diff when the job matches the harness and the channel matches the data. The job is a bug or a feature, in words, in a repository, with tools that can write, edit, and execute. The model is $3 and $15 per million, with a 200K context, at twice the speed of Claude 3 Opus, and at the speed and cost of the mid-tier Sonnet line. The channel is the API, Bedrock, or Vertex.

I would hand Artifacts a document or a short page on Claude.ai, on material that is fine to put in the app, and I would see whether the edit happens in the side window. I would price the pilot from the script, then from a real trace, and report it as list price against GPT-4o’s May card: output tied at $15 per million, input at $3 against $5. The quality column stays 64% and 38% on their harness, with a line that this is not SWE-bench. Legacy migration gets a migration sample. Bug fixing gets bugs I would actually assign, checked on the tests that gate them.

Opus’s dollar price stays off the page. Opus appears as the 38% line and as the speed baseline. GPQA, MMLU, and HumanEval stay a direction, names only. The limit I would set before a pilot is the phrase “execute code”: which directory those tools may touch, on the channel allowed to see the repo.

Update, 29 October 2026

The June 64% was Anthropic’s internal agentic coding harness, a different eval from SWE-bench. An October 2024 upgrade later published SWE-bench Verified numbers. On 30 October 2024, Anthropic reported the upgraded Claude 3.5 Sonnet at 49%, the previous state of the art at 45%, the original Claude 3.5 Sonnet at 33%, and Claude 3 Opus at 22%, all on that agent scaffold.

Computer use is a later beta about screenshots and a mouse, a different release from this June model. Claude Opus 4.5’s effort parameter is where I look when the question is how many tokens a task spends. Git worktrees for parallel agents is how I would run more than one attempt once a model is editing a repository, which is the capability this launch described.

Get new posts in your inbox

© 2026 Malhar Ujawane. Built with Astro.