Malhar Ujawane

Inspiration

Inspiration·

A 10-year-old CNN beat two frontier LLMs at skin cancer diagnosis

A 2015 Inception v3 CNN beat GPT-6 Astra and Claude Fable 5.1 at diagnosing 200 skin lesions, 62% to 42%. What it says about small models.

A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis (Dr Magnus Lynch, October 25, 2026)

Magnus Lynch, a consultant dermatologist at Guy’s Hospital, gave 200 biopsy-proven lesions from the public MILK10k dataset to GPT-6 Astra, Claude Fable 5.1 and an Inception v3 CNN from 2015 that he fine-tuned on 68,011 ISIC Archive images. Each lesion had a clinical photo and a dermoscopic image, and every model picked from the same six diagnoses.

The CNN named the right diagnosis for 125 of 200 (62%). The two frontier models got 84 and 85 (42%). His conclusion:

A small, old, specialised network using an outdated architecture recognises skin lesions far better than two frontier LLMs.

What I like is how honest the setup is. The test lesions, and every other photo of the same lesion or patient, were removed from the CNN’s training and validation data. He lists his own caveats: 200 lesions is small, 30 melanomas give wide uncertainty, each LLM case ran once, and the frontier models may have seen MILK10k in training (which would have helped them). He also notes that regulated products built on state-of-the-art models “perform considerably better” than his baseline. The cases, prompts, every LLM reply and the scoring code are downloadable.

The lesson I take for building products: if the task is narrow, has labelled data and has a clear right answer, a small specialised model with a held-out eval is still the first thing to try. The frontier model is the baseline you have to beat, and here it lost. It also failed in a pattern. GPT-6 Astra called 75 lesions basal cell carcinoma when 25 were, and Fable 5.1 called 53 lesions actinic keratosis when 20 were. You only see that kind of bias with a confusion table, which is a good argument for always building one.

There’s an on-device angle too. Lynch says the CNN is “small enough to run entirely in a phone browser using WebGPU” with nothing uploaded. That’s the same direction as Block’s Dessa speech model, which runs locally on a laptop: specialised and small enough to run on the user’s own device.

Benchmarks always come with a setup, and this one is no different. I wrote about that with Theo’s definition of a harness: a score tells you about the model plus the setup it ran in. Lynch publishes his whole setup, so anyone can check it.

Via Hacker News.

Get new posts in your inbox

© 2026 Malhar Ujawane. Built with Astro.