• Gerald Gartlehner, Isolde Sommer, Tammeka Swinson Evans, Kylie Thaler, and Kathleen N Lohr.
• Department for Clinical Epidemiology an Evidence-based Medicine, Austrian Cochrane Branch, Danube University Krems, Dr. Karl Dorrekstrasse 30, 3500 Krems, Austria; RTI-UNC Evidence-based Practice Center, Research Triangle Institute International, 3040 East Cornwallis Road, Research Triangle Park, NC 27709-2194, USA. Electronic address: gerald.gartlehner@donau-uni.ac.at.
• J Clin Epidemiol. 2015 May 1; 68 (5): 489-97.

ObjectivesWe sought to determine whether producers or users of systematic reviews using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach or a close variation give the same meanings to terms intended to convey uncertainty about treatment effects when interpreting grades for the quality or strength of evidence.Study Design And SettingFollowing exploratory interviews with stakeholders and user testing, we conducted an international Web-based survey among producers and users of systematic reviews. For each quality grade (high, moderate, low, very low/insufficient), we asked participants to assign a minimum likelihood that treatment effects will not change substantially as new studies emerge. Using multivariate analysis of covariance, we tested whether the estimated likelihoods differed between producers and users.ResultsIn all, 244 participants completed the survey. The associated minimum likelihoods that treatment effects will not change substantially for high, moderate, and low grades of quality of evidence (QOE) were 86% [95% confidence interval (CI): 85%, 87%], 61% (95% CI: 59%, 63%), and 34% (95% CI: 32%, 36%), respectively (very low was preset at 0%). Likelihoods for each grade were similar between producers and users of systematic reviews (P > 0.05 for all comparisons).ConclusionGRADE is, in general, a suitable method to convey uncertainties for systematic review producers to users. The wide ranges of likelihoods associated with GRADE terms suggest that current definitions of levels of QOE that rely exclusively on qualitative certainty expressions should be augmented by numerical predictions once such data are available.Copyright © 2015 Elsevier Inc. All rights reserved.

9 keywords...

hide…