• Norma Terrin, Christopher H Schmid, and Joseph Lau.
• Institute for Clinical Research and Health Policy Studies, Tufts-New England Medical Center, 750 Washington Street, Box 63, Boston, MA 02111, USA. nterrin@tufts-nemc.org
• J Clin Epidemiol. 2005 Sep 1; 58 (9): 894-901.

Background And ObjectivePublication bias and related biases can lead to overly optimistic conclusions in systematic reviews. The funnel plot, which is frequently used to detect such biases, has not yet been subjected to empirical evaluation as a visual tool. We sought to determine whether researchers can correctly identify publication bias from visual inspection of funnel plots in typical-size systematic reviews.MethodsA questionnaire with funnel plots containing 10 studies each (the median number in medical meta-analyses) was completed by 41 medical researchers, including clinical research fellows in a meta-analysis class, faculty in clinical care research, and experienced systematic reviewers.ResultsOn average, participants correctly identified 52.5% (95% CI 50.6-54.4%) of the plots as being affected or unaffected by publication bias. The weighted mean percent correct, which adjusted for the fact that asymmetric plots are more likely to occur in the presence of publication bias, was also low (48.3 to 62.8%, depending on the presence or absence of publication bias and heterogeneous study effects).ConclusionResearchers who assess for publication bias using the funnel plot may be misled by its shape. Authors and readers of systematic reviews need to be aware of the limitations of the funnel plot.

7 keywords...

hide…