Man against machine: AI is better than dermatologists at diagnosing skin cancer
Researchers have shown for the first time that a form of artificial intelligence or machine learning known as a deep learning convolutional neural network (CNN) is better than experienced dermatologists at detecting skin cancer.
In a study published in the leading cancer journal Annals of Oncology today, researchers in Germany, the USA and France trained a CNN to identify skin cancer by showing it more than 100,000 images of malignant melanomas (the most lethal form of skin cancer), as well as benign moles (or nevi). They compared its performance with that of 58 international dermatologists and found that the CNN missed fewer melanomas and misdiagnosed benign moles less often as malignant than the group of dermatologists.
A CNN is an artificial neural network inspired by the biological processes at work when nerve cells (neurons) in the brain are connected to each other and respond to what the eye sees. The CNN is capable of learning fast from images that it “sees” and teaching itself from what it has learned to improve its performance (a process known as machine learning).
The first author of the study, Professor Holger Haenssle, senior managing physician at the Department of Dermatology, University of Heidelberg, Germany, explained: “The CNN works like the brain of a child. To train it, we showed the CNN more than 100,000 images of malignant and benign skin cancers and moles and indicated the diagnosis for each image. Only dermoscopic images were used, that is lesions that were imaged at a 10-fold magnification. With each training image, the CNN improved its ability to differentiate between benign and malignant lesions.
“After finishing the training, we created two test sets of images from the Heidelberg library that had never been used for training and therefore were unknown to the CNN. One set of 300 images was built to solely test the performance of the CNN. Before doing so, 100 of the most difficult lesions were selected to test real dermatologists in comparison to the results of the CNN.”
Dermatologists from around the world were invited to take part, and 58 from 17 countries around the world agreed. Of these, 17 (29%) indicated they had less than two years’ experience in dermoscopy, 11 (19%) said they were skilled with between two to five years’ experience, and 30 (52%) were expert with more than five years’ experience.
The dermatologists were asked to first make a diagnosis of malignant melanoma or benign mole just from the dermoscopic images (level I) and make a decision about how to manage the condition (surgery, short-term follow-up, or no action needed). Then, four weeks later they were given clinical information about the patient (including age, sex and position of the lesion) and close-up images of the same 100 cases (level II) and asked for diagnoses and management decisions again.
In level I, the dermatologists accurately detected an average of 86.6% of melanomas, and correctly identified an average of 71.3% of lesions that were not malignant. However, when the CNN was tuned to the same level as the physicians to correctly identify benign moles (71.3%), the CNN detected 95% of melanomas. At level II, the dermatologists improved their performance, accurately diagnosing 88.9% of malignant melanomas and 75.7% that were not cancer.
“The CNN missed fewer melanomas, meaning it had a higher sensitivity than the dermatologists, and it misdiagnosed fewer benign moles as malignant melanoma, which means it had a higher specificity; this would result in less unnecessary surgery,” said Professor Haenssle.
“When dermatologists received more clinical information and images at level II, their diagnostic performance improved.
