13.6.2.4  When pooling is judged not to be appropriate

Before undertaking a meta-analysis, review authors must ask themselves the standard question about whether primary studies are ‘similar enough’ to justify pooling (see Chapter 9, Section 9.1). Forest plots in RevMan allow the presentation of estimates and standard errors for each study, using the ‘Generic inverse-variance’ outcome type. Meta-analyses can be suppressed, or included only for subgroups within a plot. Providing that effect estimates from the included studies can be expressed using consistent effect measures, we recommend that review authors display individual study results for NRS with similar study design features using forest plots, as a standard feature. If consistent effect measures are not available, then additional tables should be used to present results in a systematic format.

 

If included studies are not sufficiently homogeneous to combine in a meta-analysis (which is expected to be the norm for reviews that include NRS), the NRSMG recommends displaying the results of included studies in a forest plot but suppressing the pooled estimate. Studies may be sorted in the forest plot (or shown in separate forest plots) by study design feature, or some other feature believed to reflect susceptibility to bias (e.g. number of Newcastle-Ottawa Scale ‘stars’ (Wells 2008)). Heterogeneity diagnostics and investigations (e.g. a test for heterogeneity, the I2 statistic and meta-regression analyses) are worthwhile even when a judgement has been made that calculating a pooled estimate of effect is not (Higgins 2003, Siegfried 2003).

 

Narrative syntheses are, however, problematic, because it is difficult to set out or describe results without being selective or emphasizing some findings over others. Ideally, authors should set out in the review protocol how they plan to use narrative synthesis to report the findings of primary studies.