Artificial intelligence screens for diabetic retinopathy
Google scientists say ‘deep learning’ algorithm has high sensitivity and specificity
WRITTEN BY ANDREW SKELLY ON NOVEMBER 29, 2016 FOR CANADIANHEALTHCARENETWORK.CA
In yet another sign of the growth of artificial intelligence research in medicine, Google researchers have published a paper in the Journal of the American Medical Association showing their image-recognition software could effectively screen for diabetic retinopathy.
Dr. Varun Gulshan (PhD) of Google Inc. in Mountain View, Calif., and colleagues employed a concept called “deep learning,” in which an algorithm programs itself by learning from a large set of examples, removing the need to specify rules.
They “trained” the algorithm using a data set of 128,175 retinal images, which were graded three to seven times for diabetic retinopathy, diabetic macular edema and image gradability by a panel of 54 U.S. ophthalmologists and ophthalmology senior residents.
The researchers then validated the algorithm in two separate data sets of 9,963 and 1,748 images, with two different settings for high specificity or high sensitivity. The goal was to detect referable diabetic retinopathy, defined as moderate and worse diabetic retinopathy, referable diabetic macular edema, or both, based on a reference standard of the majority decision of the ophthalmologist panel.
At the high-specificity setting, the algorithm had 90.3% and 87.0% sensitivity and 98.1% and 98.5% specificity in the two data sets.
At the high-sensitivity setting, the algorithm had 97.5% and 96.1% sensitivity and 93.4% and 93.9% specificity.
The researchers cited several previous studies of automated and semi-automated diabetic retinopathy evaluation, but none of those systems performed as well.
“These results demonstrate that deep neural networks can be trained, using large data sets and without having to specify lesion-based features, to identify diabetic retinopathy or diabetic macular edema in retinal fundus images with high sensitivity and high specificity,” Dr. Gulshan and colleagues wrote.
“Further research is necessary to determine the feasibility of applying this algorithm in the clinical setting and to determine whether use of the algorithm could lead to improved care and outcomes compared with current ophthalmologic assessment.”
In a rare move, JAMA published two editorials on the paper, plus an opinion column on the implications of artificial intelligence in radiology and pathology.
Commenting on the Google study, Dr. Tien Yin Wong of the Singapore Eye Research Institute and Dr. Neil Bressler of the Wilmer Eye Institute at Johns Hopkins University in Baltimore wrote that it “truly represents the brave new world in medicine.”
However, they noted it is important to assess how the algorithm performs in the critical situation of a patient with “sight-threatening” diabetic retinopathy. There were too few such severe cases in the current study.
Furthermore, a potential weakness of the software is that it does not detect other important eye conditions, such as signs of glaucoma or age-related macular degeneration. “Most diabetic retinopathy screening programs currently in operation using human assessment . . . include these conditions.”
In a second editorial, Dr. Andrew Beam (PhD) and Dr. Isaac Kohane of the department of biomedical informatics at Harvard Medical School in Boston wrote that it seems likely that artificial intelligence algorithms will reshape aspects of specialties such as pathology, radiology and dermatology that involve analysis of images. They noted that once an artificial intelligence has been “trained,” it can be deployed on a computer chip costing approximately $1,000 and process about 3,000 images per second. “This translates to an image processing capacity of almost 260 million images per day (because these devices can work around the clock), all for the cost of approximately $1,000.”
How might that transform the profession? Dr. Saurabh Jha, a radiologist at the University of Pennsylvania in Philadelphia, and Dr. Eric Topol, founder and director of the Scripps Translational Science Institute in La Jolla, Calif., noted in their opinion column that is clear many tasks in radiology and pathology can be handled by artificial intelligence.
“Perhaps these specialties should be merged into a single entity, the ‘information specialist,’ whose responsibility will not be so much to extract information from images and histology but to manage the information extracted by artificial intelligence in the clinical context of the patient.”
Comments
