• Adrian Baker, Jonathan Potter, Katharine Young, and Ira Madan.
• NHS Plus, Bristol, UK.
• J Eval Clin Pract. 2011 Aug 1; 17 (4): 758-62.

RationaleThis study focused on factors that most concern specialist societies when choosing an evidence grading system, such as methodological strengths and weaknesses, applicability and ease of use. The grading systems chosen were the Scottish Intercollegiate Guidelines Network (SIGN), the Grading of Recommendations Assessment, Development and Evaluation (GRADE) and the National Service Framework for long-term neurological conditions critical appraisal tool (NSF-LTC).MethodologyTwelve assessors, representing typical members of society-based guideline development groups, graded papers and a recommendation using a key question as a guide. Key questions and recommendations were extracted from existing clinical guidelines representing a variety of research fields. Assessors were given 3 months to grade the papers using the grading systems and to complete a semi-structured qualitative questionnaire. The results were independently assessed for emerging themes.ResultsAssessors felt all three systems had strengths and weaknesses depending on the type of evidence being graded. GRADE was seen as the most complex but rigorous system, while SIGN and NSF were seen as easier and more flexible to use, but less methodologically rigorous. In grading the evidence, 10% of assessors used GRADE incorrectly, 33% used SIGN incorrectly and 75% used NSF-LTC incorrectly. In grading the recommendations, 60% used GRADE incorrectly, 50% used SIGN incorrectly and 50% used NSF-LTC incorrectly.ImplicationsIt is recommended that specialist societies consider the type of evidence they will be evaluating and the research experience of the appraisers before selecting a grading system. Additionally, appraisers should have training in appraising and grading evidence using the system to be employed.© 2011 Blackwell Publishing Ltd.

13 keywords...

hide…