On October 25, 2026, dermatologist Magnus Lynch of Guy's Hospital published a test that spread fast. A 10-year-old image model, fine-tuned for about 20 minutes, named the right skin lesion diagnosis more often than GPT-6 Astra and Claude Fable 5.1. The CNN scored 62%. Each language model scored 42%. The result is real but narrow. It is one clinician's blog post with 200 cases, not a peer-reviewed study, and it does not show that either approach is ready for clinic use.
What Lynch Tested
Lynch drew 200 biopsy-proven lesions at random from MILK10k, a public dataset released in 2025. Each lesion has a clinical photo and a dermoscopic image, a close-up through a skin microscope. Half should not be sent home: 30 melanomas, 25 basal cell carcinomas, 25 squamous cell carcinomas and 20 actinic keratoses. Half are safe to discharge: 50 moles and 50 seborrhoeic keratoses. Every lesion was biopsied, so these cases are harder than usual.
Each model chose from six diagnoses. The two language models got both images and the same prompt, at default settings, one run each. The CNN is Inception v3, first published by Google in 2015. Lynch fine-tuned it on 68,011 images from the ISIC Archive using one H100 GPU, and removed the test lesions from its training data.
The Scores
The CNN named the right diagnosis for 125 of 200 lesions. GPT-6 Astra got 84 and Claude Fable 5.1 got 85. With a second guess allowed, the CNN reached 173, against 134 and 138. On the 30 melanomas it was right 13 times, against 8 for Astra and 11 for Fable.
The biggest gap was on harmless lesions. The CNN named 37 of 50 moles and 26 of 50 seborrhoeic keratoses. Astra got 16 and 13, and Fable got 33 and 6. The CNN did not win everywhere: Astra got all 25 basal cell carcinomas right, against 24 for the CNN. The models also had favourite answers. Astra called 75 lesions basal cell carcinoma when 25 were, and Fable called 53 actinic keratosis when 20 were.
The Safety Catch
Naming the right diagnosis is not the same as keeping dangerous lesions in the clinic. At its preset threshold, the CNN sent home 18 of 30 melanomas. GPT-6 Astra sent home 3, and Claude Fable 5.1 sent home 4. The language models played it safe: they kept 97 and 96 of the 100 unsafe lesions, but also kept most harmless ones, discharging only 32 and 22 of 100.
When Lynch moved the CNN's cutoff to match the language models' discharge rate, it kept 98 or 99 unsafe lesions and sent home only 1 or 2 melanomas. So the CNN is better at telling lesions apart, but it needs its cutoff set for safety, and that adjustment was made after seeing the results.
What the Test Cannot Show
Lynch lists the limits himself. Two hundred lesions include only 30 melanomas, each language model ran once, and answers vary between runs. MILK10k has been public since 2025, so the language models may have seen it in training, which would favour them. The CNN is a baseline, and he says regulated products built on stronger models perform considerably better. On ordinary archive images his CNN separated lesions better, with an AUC of 0.96 against 0.86 here. No dermatologist baseline was tested.
Asked as a patient, with one photo and the question "do I need to see a doctor?", both language models said yes for nearly every lesion, including 99 and 98 of 100 melanomas. That is cautious, but not a diagnosis.
What It Says About General Models
Our reading, not a finding: this is a test of image recognition, where a small model trained on 68,011 labeled images beat general models that were never trained for this task. That fits a narrow job. It differs from Microsoft's MAI-DxO result on text cases, where the study's own data suggest the strength of the underlying model mattered. The practical lesson is about design: a specialist model may be better for the image, and a general model for the questions and explanation. Lynch released the cases, prompts, every language model reply and the scoring code, so others can re-run the test.
Disclosure: This article was drafted with Claude, an AI model made by Anthropic. Claude Fable 5.1, one of the two models tested, is also made by Anthropic. Lynch says he used Claude Code, Anthropic's coding tool, for the analysis and first drafts of his write-up. Figures here come from his write-up.
