• Bernardo Sousa-Pinto, Rafael José Vieira, Manuel Marques-Cruz, Antonio Bognanni, Sara Gil-Mata, Slava Jankin, Joana Amaro, Liliane Pinheiro, Marta Mota, Mattia Giovannini, Leticia de Las Vecillas, Ana Margarida Pereira, Justyna Lityńska, Boleslaw Samolinski, Jonathan Bernstein, Mark Dykewicz, Martin Hofmann-Apitius, Marc Jacobs, Nikolaos Papadopoulos, Sian Williams, Torsten Zuberbier, João A Fonseca, Ricardo Cruz-Correia, Jean Bousquet, and Holger J Schünemann.
• Department of Community Medicine, Information and Health Decision Sciences, Faculty of Medicine, University of Porto, Porto, Portugal (B.S.-P., R.J.V., M.M.-C., S.G.-M., L.P., M.M., A.M.P.).
• Ann. Intern. Med. 2024 Sep 24.

BackgroundGuideline questions are typically proposed by experts.ObjectiveTo assess how large language models (LLMs) can support the development of guideline questions, providing insights on approaches and lessons learned.DesignTwo approaches for guideline question generation were assessed: 1) identification of questions conveyed by online search queries and 2) direct generation of guideline questions by LLMs. For the former, the researchers retrieved popular queries on allergic rhinitis using Google Trends (GT) and identified those conveying questions using both manual and LLM-based methods. They then manually structured as guideline questions the queries that conveyed relevant questions. For the second approach, they tasked an LLM with proposing guideline questions, assuming the role of either a patient or a clinician.SettingAllergic Rhinitis and its Impact on Asthma (ARIA) 2024 guidelines.ParticipantsNone.MeasurementsFrequency of relevant questions generated.ResultsThe authors retrieved 3975 unique queries using GT. From these, they identified 37 questions, of which 22 had not been previously posed by guideline panel members and 2 were eventually prioritized by the panel. Direct interactions with LLMs resulted in the generation of 22 unique relevant questions (11 not previously suggested by panel members), and 4 were eventually prioritized by the panel. In total, 6 of 39 final questions prioritized for the 2024 ARIA guidelines were not initially thought of by the panel. The researchers provide a set of practical insights on the implementation of their approaches based on the lessons learned.LimitationSingle case study (ARIA guidelines).ConclusionApproaches using LLMs can support the development of guideline questions, complementing traditional methods and potentially augmenting questions prioritized by guideline panels.Primary Funding SourceFraunhofer Cluster of Excellence for Immune-Mediated Diseases.

3 keywords...

hide…