Proportions of participants for whom no outcome data were obtained should always be collected and reported in a ‘Risk of bias’ table; note that the proportions may vary by outcome and by randomized group. However, there is no consensus on the best way to handle these participants in an analysis. There are two basic options, and a plausible option should be used both as a main analysis and as a basis for sensitivity analysis (see below and Chapter 9, Section 9.7).
Available case analysis: Include data on only those whose results are known, using as a denominator the total number of people who had data recorded for the particular outcome in question. Variation in the degree of missing data across studies may be considered as a potential source of heterogeneity.
ITT analysis using imputation: Base an analysis on the total number of randomized participants, irrespective of how the original study authors analysed the data. This will involve imputing outcomes for the missing participants. There are several approaches to imputing dichotomous outcome data. One common approach is to assume either that all missing participants experienced the event, or that all missing participants did not experience the event. An alternative approach is to impute data according to the event rate observed in the control group, or according to event rates among completers in the separate groups (the latter provides the same estimate of intervention effect but results in unwarranted inflation of the precision of effect estimates). The choice among these assumptions should be based on clinical judgement. Studies with imputed data may be given more weight than they warrant if entered as dichotomous data into RevMan. It is possible to determine more appropriate weights (Higgins 2008); consultation with a statistician is recommended. However, none of these assumptions is likely to reflect the truth, except for imputing ‘failures’ in some settings such as smoking cessation trials, so an imputation approach is generally not recommended.
The potential impact of the missing data on the results should be considered in the interpretation of the results of the review. This will depend on the degree of ‘missingness’, the frequency of the events and the size of the pooled effect estimate. Gamble and Hollis suggest a sensitivity analysis for dichotomous outcomes based on consideration of ‘best-case’ and ‘worst-case’ scenarios (Gamble 2005). The ‘best-case’ scenario is that all participants with missing outcomes in the experimental intervention group had good outcomes, and all those with missing outcomes in the control intervention group had poor outcomes; the ‘worst-case’ scenario is the converse. The sensitivity analysis down-weights studies in which the discrepancy between ‘best-case’ and ‘worst-case’ scenarios is high, although the down-weighting may be too extreme.
A more plausible sensitivity analysis explicitly considers what the event rates might have been in the missing data. For example, suppose an available case analysis has been used, and a particular study has 20% risk in the intervention arm and 15% risk in the control arm. An available case analysis implicitly assumes that the same fractions apply in the missing data, so three suitable sensitivity analyses to compare with this analysis might consider the risk in the missing data to be 15% in both arms, or 15% and 10% in the experimental and control arms respectively, or 20% and 10% respectively. Alternatively, suppose that in the main analysis, all missing values have been imputed as events. A sensitivity analysis to compare with this analysis could consider the case that, say, 10% of missing participants experienced the event, or 10% in the intervention arm and 5% in the control arm. Graphical approaches to sensitivity analysis have been considered (Hollis 2002).
Higgins et al. suggest an alternative approach that can incorporate specific reasons for missing data, which considers plausible event risks among missing participants in relation to risks among those observed (Higgins 2008). Bayesian approaches, which automatically down-weight studies with more missing data, are considered by White et al. (White 2008a, White 2008b).