In systematic review or meta-analyses, multiple cohort studies and randomized controlled studies are integrated and conducted more precise analysis. However, it’s impossible to avoid publication bias because non-significant studies are less likely to be posted and published. If systematic reviews or meta-analyses with publication bias were conducted, incorrect treatment would be accepted. Funnel plot is one of methods to assess whether there is publication bias or not.

Scatter plot inverse of standard error on vertical axis against odds ratio or hazard ratio on horizontal axis. If there was no publication bias, funnel plot would be symmetrical. However, if there was publication bias, funnel plot would be asymmetrical.

It is required to calculate standard error in order to draw funnel plot, if point estimated of the effect size and its 95 % confidence interval are known, they are described in most of systematic reviews and meta-analysis, you can find standard error as following;

ES: effect size, SE: standard error, 95%CI: 95 % confidence interval, 95%LL: 95 % Lower Limit, 95%UL: 95 % Upper Limit

References:
Bias in meta-analysis detected by a simple, graphical test (pdf)
A note on graphical presentation of estimated odds ratios from several clinical trials
Funnel plot (Wikipedia)

Bias in meta-analysis detected by a simple, graphical test

Matthias Egger, George Davey Smith, Martin Schneider, Christoph Minder

Abstract

Objective

Funnel plots (plots of effect estimates against sample size) may be useful to detect bias in meta-analyses that were later contradicted by large trials. We examined whether a simple test of asymmetry of funnel plots predicts discordance of results when meta-analyses are compared to large trials, and we assessed the prevalence of bias in published meta-analyses.

Design

Medline search to identify pairs consisting of a meta-analysis and a single large trial (concordance of results was assumed if effects were in the same direction and the meta-analytic estimate was within 30% of the trial); analysis of funnel plots from 37 meta-analyses identified from a hand search of four leading general medicine journals 1993-6 and 38 meta-analyses from the second 1996 issue of the Cochrane Database of Systematic Reviews.

Main outcome measure

Degree of funnel plot asymmetry as measured by the intercept from regression of standard normal deviates against precision.

Results

In the eight pairs of meta-analysis and large trial that were identified (five from cardiovascular medicine, one from diabetic medicine, one from geriatric medicine, one from perinatal medicine) there were four concordant and four discordant pairs. In all cases discordance was due to meta-analyses showing larger effects. Funnel plot asymmetry was present in three out of four discordant pairs but in none of concordant pairs. In 14 (38%) journal meta-analyses and 5 (13%) Cochrane reviews, funnel plot asymmetry indicated that there was bias.

Conclusions

A simple analysis of funnel plots provides a useful test for the likely presence of bias in meta-analyses, but as the capacity to detect bias will be limited when meta-analyses are based on a limited number of small trials the results from such analyses should be treated with considerable caution.

Key messages

• Systematic reviews of randomised trials are the best strategy for appraising evidences; however, the findings of some meta-analyses were later contradicted by large trials
• Funnel plots, plots of the trials’ effect estimates against sample size, are skewed and asymmetrical in the presence of publication bias and other biases
• Funnel plot asymmetry, measured by regression analyses, predicts discordance of results when meta-analyses are compared with single large trials
• Funnel plot asymmetry was found in 38% of meta-analyses published in leading general medicine journals and in 13% of reviews from Cochrane Database of Systematic Reviews
• Critical examination of systematic reviews for publication and related biases should be considered a routine procedure