• Elvan Burak Verdi, Muhammed Yılmaz, Deniz Doğan Mülazimoğlu, Abdussamet Türker, Aslıhan Gürün Kaya, Özlem Işık, Aslı Bostanoğlu Karaçin, Övgü Velioğlu Yakut, Bülent Mustafa Yenigün, Çağlar Uzun, Atilla Halil Elhan, Özlem Özdemir Kumbasar, Akın Kaya, Ayten Kayı Cangır, Cantürk Taşçı, Ahmet Murat Özbayoğlu, and Serhat Erol.
• Department of Chest Diseases, Ankara University Faculty of Medicine, Ankara, Turkey.
• J. Investig. Med. 2024 Jan 1; 72 (1): 889988-99.

AbstractThe generalizability of artificial intelligence (AI) models is a major issue in the field of AI applications. Therefore, we aimed to overcome the generalizability problem of an AI model developed for a particular center for pneumothorax detection using a small dataset for external validation. Chest radiographs of patients diagnosed with pneumothorax (n = 648) and those without pneumothorax (n = 650) who visited the Ankara University Faculty of Medicine (AUFM; center 1) were obtained. A deep learning-based pneumothorax detection algorithm (PDA-Alpha) was developed using the AUFM dataset. For implementation at the Health Sciences University (HSU; center 2), PDA-Beta was developed through external validation of PDA-Alpha using 50 radiographs with pneumothorax obtained from HSU. Both PDA algorithms were assessed using the HSU test dataset (n = 200) containing 50 pneumothorax and 150 non-pneumothorax radiographs. We compared the results generated by the algorithms with those of physicians to demonstrate the reliability of the results. The areas under the curve for PDA-Alpha and PDA-Beta were 0.993 (95% confidence interval (CI): 0.985-1.000) and 0.986 (95% CI: 0.962-1.000), respectively. Both algorithms successfully detected the presence of pneumothorax on 49/50 radiographs; however, PDA-Alpha had seven false-positive predictions, whereas PDA-Beta had one. The positive predictive value increased from 0.525 to 0.886 after external validation (p = 0.041). The physicians' sensitivity and specificity for detecting pneumothorax were 0.585 and 0.988, respectively. The performance scores of the algorithms were increased with a small dataset; however, further studies are required to determine the optimal amount of external validation data to fully address the generalizability issue.

17 keywords...

hide…