Biases associated with particular characteristics of studies may be examined using a technique often known as meta-epidemiology (Naylor 1997, Sterne 2002). A meta-epidemiological study analyses a collection of meta-analyses, in each of which the component studies have been classified according to some study-level characteristic. An early example was the study of clinical trials with dichotomous outcomes included in meta-analyses from the Cochrane Pregnancy and Childbirth Database (Schulz 1995b). This study demonstrated that trials in which randomization was inadequately concealed or inadequately reported yielded exaggerated estimates of intervention effect compared with trials reporting adequate concealment, and found a similar (but smaller) association for trials that were not described as double-blind.
A simple analysis of a meta-epidemiological study is to calculate the ‘ratio of odds ratios’ within each meta-analysis (for example, the intervention odds ratio in trials with inadequate/unclear allocation concealment divided by the odds ratio in trials with adequate allocation concealment). These ratios of odds ratios are then combined across meta-analyses, in a meta-analysis. Thus, such analyses are also known as ‘meta-meta-analyses’. In subsequent sections of this chapter, empirical evidence of bias from meta-epidemiological studies is cited where available as part of the rationale for assessing each domain of potential bias.