• Youngjune Kim, Kyong Joon Lee, Leonard Sunwoo, Dongjun Choi, Chang-Mo Nam, Jungheum Cho, Jihyun Kim, Yun Jung Bae, Roh-Eul Yoo, Byung Se Choi, Cheolkyu Jung, and Jae Hyoung Kim.
• Department of Radiology, Seoul National University Bundang Hospital, Seongnam.
• Invest Radiol. 2019 Jan 1; 54 (1): 7-15.

ObjectivesThe aim of this study was to compare the diagnostic performance of a deep learning algorithm with that of radiologists in diagnosing maxillary sinusitis on Waters' view radiographs.Materials And MethodsAmong 80,475 Waters' view radiographs, examined between May 2003 and February 2017, 9000 randomly selected cases were classified as normal or maxillary sinusitis based on radiographic findings and divided into training (n = 8000) and validation (n = 1000) sets to develop a deep learning algorithm. Two test sets composed of Waters' view radiographs with concurrent paranasal sinus computed tomography were labeled based on computed tomography findings: one with temporal separation (n = 140) and the other with geographic separation (n = 200) from the training set. Area under the receiver operating characteristics curve (AUC), sensitivity, and specificity of the algorithm and 5 radiologists were assessed. Interobserver agreement between the algorithm and majority decision of the radiologists was measured. The correlation coefficient between the predicted probability of the algorithm and average confidence level of the radiologists was determined.ResultsThe AUCs of the deep learning algorithm were 0.93 and 0.88 for the temporal and geographic external test sets, respectively. The AUCs of the radiologists were 0.83 to 0.89 for the temporal and 0.75 to 0.84 for the geographic external test sets. The deep learning algorithm showed statistically significantly higher AUC than radiologist in both test sets. In terms of sensitivity and specificity, the deep learning algorithm was comparable to the radiologists. A strong interobserver agreement was noted between the algorithm and radiologists (Cohen κ coefficient, 0.82). The correlation coefficient between the predicted probability of the algorithm and confidence level of radiologists was 0.89 and 0.84 for the 2 test sets, respectively.ConclusionsThe deep learning algorithm could diagnose maxillary sinusitis on Waters' view radiograph with superior AUC and comparable sensitivity and specificity to those of radiologists.

17 keywords...

hide…