• J Pain Symptom Manage · Oct 2024

    Comparative Study

    AI-Generated Content in Cancer Symptom Management: A Comparative Analysis Between ChatGPT and NCCN.

    • David Lazris, Yael Schenker, and Teresa Hagan Thomas.
    • Department of Medicine, (D.L, Y.C), University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania, USA. Electronic address: lazrisdi@upmc.edu.
    • J Pain Symptom Manage. 2024 Oct 1; 68 (4): e303e311e303-e311.

    BackgroundArtificial intelligence-driven tools, like ChatGPT, are prevalent sources for online health information. Limited research has explored the congruity between AI-generated content and professional treatment guidelines. This study seeks to compare recommendations for cancer-related symptoms generated from ChatGPT with guidelines from the National Comprehensive Cancer Network (NCCN).InterventionWe extracted treatment recommendations for nine symptoms from NCCN, separated into four full Supportive Care sections and five subsections of the Palliative Care webpage. We entered "How can I reduce my cancer-related [symptom]" into ChatGPT- 3.5 for these same symptoms and extracted its recommendations. A comparative content analysis focused on recommendations for medications, consultations, and non-pharmacological strategies. We compared word count and Flesch-Kincaid Grade Level (FKGL) readability for each NCCN and ChatGPT section.OutcomesThe mean percent agreement between NCCN and ChatGPT recommendations was 37.3% (range 16.7%-81.8%). NCCN offered more specific medication recommendations. ChatGPT did recommend medications in the constipation and diarrhea sections that were not recommended by NCCN. Significant differences in word count (P=0.03) and FKGL (P<0.01) were found for NCCN Supportive Care webpages, with ChatGPT having lower word count and reading level. In the NCCN Palliative Care webpage subsections, there was no significant difference in word count (P=0.076), but FKGL was significantly lower with ChatGPT (P<0.01).Conclusions/Lessons LearnedWhile ChatGPT provides concise, accessible supportive care advice, discrepancies with guidelines raise concerns for patient-facing symptom management recommendations. Future research should consider how AI can be used in conjunction with evidence-based guidelines to support cancer patients' supportive care needs.Copyright © 2024 The Authors. Published by Elsevier Inc. All rights reserved.

      Pubmed     Copy Citation     Plaintext  

      Add institutional full text...

    Notes

     
    Knowledge, pearl, summary or comment to share?
    300 characters remaining
    help        
    You can also include formatting, links, images and footnotes in your notes
    • Simple formatting can be added to notes, such as *italics*, _underline_ or **bold**.
    • Superscript can be denoted by <sup>text</sup> and subscript <sub>text</sub>.
    • Numbered or bulleted lists can be created using either numbered lines 1. 2. 3., hyphens - or asterisks *.
    • Links can be included with: [my link to pubmed](http://pubmed.com)
    • Images can be included with: ![alt text](https://bestmedicaljournal.com/study_graph.jpg "Image Title Text")
    • For footnotes use [^1](This is a footnote.) inline.
    • Or use an inline reference [^1] to refer to a longer footnote elseweher in the document [^1]: This is a long footnote..

    hide…