• Br J Anaesth · Feb 2025

    A comparison of large language model-generated and published perioperative neurocognitive disorder recommendations: a cross-sectional web-based analysis.

    • Sarah Saxena, Odmara L Barreto Chang, Melanie Suppan, Basak Ceyda Meco, Susana Vacas, Finn Radtke, Idit Matot, Arnout Devos, Mervyn Maze, Mia Gisselbaek, and Joana Berger-Estilita.
    • Department of Surgery, Research Institute for Health Sciences and Technology, University of Mons, Mons, Belgium; Department of Anesthesiology, Helora, Mons, Belgium.
    • Br J Anaesth. 2025 Feb 7.

    BackgroundPerioperative neurocognitive disorders (PNDs) are common complications after surgery and anaesthesia, particularly in older adults, leading to increased morbidity, mortality, and healthcare costs. Therefore, major medical societies have developed recommendations for the prevention and treatment of PNDs. Our study evaluated the reliability of large language models, specifically ChatGPT-4 and Gemini, in generating recommendations for PND management and comparing them with published guidelines.MethodsWe conducted an online cross-sectional web-based analysis over 48 h in June 2024. Artificial intelligence (AI)-generated recommendations were produced in six different locations across five countries (Switzerland, Belgium, Turkey, Canada, and the East and West Coasts of the USA). The English prompt 'a table of a bundle of care for perioperative neurocognitive disorders' was entered into ChatGPT-4 and Gemini, generating tables evaluated by independent reviewers. The primary outcomes were the Total Disagreement Score (TDS) and Quality Assessment of Medical Artificial Intelligence (QAMAI), which compared AI-generated recommendations with published guidelines.ResultsThe study generated 14 tables, with TDS and QAMAI scores showing similar results for ChatGPT-4 and Gemini (2 [1-3] vs 2 [2-3], P=0.636 and 4 [4-4] vs 4 [3-4], P=0.424, respectively). AI-generated recommendations aligned well with published guidelines, with the highest alignment observed in ChatGPT-4-generated recommendations. No complete agreement with guidelines was achieved, and lack of cited sources was a noted limitation.ConclusionsLarge language models can generate perioperative neurocognitive disorder recommendations that align closely with published guidelines. However, further validation and integration of clinician feedback are required before clinical application.Copyright © 2025 The Author(s). Published by Elsevier Ltd.. All rights reserved.

      Pubmed     Copy Citation     Plaintext  

      Add institutional full text...

    Notes

     
    Knowledge, pearl, summary or comment to share?
    300 characters remaining
    help        
    You can also include formatting, links, images and footnotes in your notes
    • Simple formatting can be added to notes, such as *italics*, _underline_ or **bold**.
    • Superscript can be denoted by <sup>text</sup> and subscript <sub>text</sub>.
    • Numbered or bulleted lists can be created using either numbered lines 1. 2. 3., hyphens - or asterisks *.
    • Links can be included with: [my link to pubmed](http://pubmed.com)
    • Images can be included with: ![alt text](https://bestmedicaljournal.com/study_graph.jpg "Image Title Text")
    • For footnotes use [^1](This is a footnote.) inline.
    • Or use an inline reference [^1] to refer to a longer footnote elseweher in the document [^1]: This is a long footnote..

    hide…