Pivotal clinical trial
In 2017, IDx funded a clinical trial that compared the performance of IDx-DR to the gold standard diagnostic for diabetic retinopathy. It was the first study to prospectively assess the safety of an autonomous AI system in patient care.
For the study, 900 adult patients with diabetes but no history of diabetic retinopathy were examined at 10 primary care sites across the U.S. The clinics were chosen to ensure a study population with diversity in age, gender, ethnicity, and severity of diabetes.
Retinal images of the patients were obtained using a robotic camera, with an AI assisting the operator in getting good quality images. Once the four images were complete, a second AI—the diagnostic AI—then made a clinical diagnosis in 20 seconds. The diagnostic AI detects disease just as expert clinicians do, by having detectors for the lesions characteristic for diabetic retinopathy, including microaneurysms, hemorrhages, and lipoprotein exudates.
Camera operators in the study were existing staff of the primary care clinics, and not physicians or trained photographers. The camera operators received four hours of training on IDx-DR, and if they were unable to obtain a quality image of the retina after three attempts, the AI system recommended the patient’s pupils be dilated to provide a better view of the back of the eye.
Study participants also had retinal images taken at each of the primary care clinics using specialized widefield and 3D imaging equipment without AI and operated by experienced retinal photographers certified by the Wisconsin Fundus Photograph Reading Center (FPRC)—the gold standard in grading the severity of diabetic retinopathy.
IDx-DR passes the test
Complete diagnostic data accomplished by both the AI system and FPRC readers was available for 819 of the original 900 study participants. FPRC readers identified 198 participants with more than mild diabetic retinopathy who should be further examined by a specialist; the AI was able to correctly identify 173 of the 198 participants with disease, resulting in a sensitivity of 87 percent.
Among the 621 disease-free participants identified by FPRC readers, AI identified 556 participants, for a specificity of 90 percent. The AI had a 96 percent imageability rate: of the 852 participants who had an FPRC diagnosis, 819 had an AI system diagnostic output.
Rating the imageability of the AI—or how likely the system captures pictures that enable an accurate diagnosis—was factored into the clinical trial during years of discussions between IDx and the FDA.
“Normally, diagnostic systems are evaluated on sensitivity and specificity. Once you start thinking about autonomous systems that are making diagnoses without any human oversight, imageability becomes very important,” says Michael Abràmoff, MD, PhD.
“The AI system’s primary role is to identify those people with diabetes who are likely to have diabetic retinopathy that requires further evaluation by an eye-care provider. The study results demonstrate the safety of autonomous AI systems to bring specialty-level diagnostics to a primary care setting, with the potential to increase access and lower cost,” he says.
Upon reviewing the clinical trial data, the FDA granted IDx clearance in April 2018 to market the IDx-DR system in the U.S. Since 2013, IDx-DR has been available in Europe.
“Dr. Abràmoff and his collaborators within the department and University of Iowa have worked hard to develop this new technology and bring it to the point of patient care. We are proud of his focused efforts and are very excited about the implications this technology has on the diagnosis and treatment of diabetic eye disease,” says Keith Carter, MD (88F), FACS, UI professor and head of the Department of Ophthalmology and Visual Sciences, and the Lillian C. and Dr. C.S. O’Brien Chair in Ophthalmology.
•