Time lag bias

Studies continue to appear in print many years after approval by ethics committees. Hopewell and colleagues reviewed studies examining time to publication for results of clinical trials (Hopewell 2007a). The two studies included in this review (Stern 1997, Ioannidis 1998) found that about half of all trials were published and that those with positive results were published, on average, approximately 2-3 years earlier than trials with null or negative results.


Among proposals submitted to the Royal Prince Alfred Hospital Ethics Committee in Sydney, Australia, an estimated 85% of studies with significant results as compared to 65% of studies with null results had been published after 10 years (Stern 1997).  The median time to publication was 4.7 years for studies with significant results and 8.0 years for studies with negative/null results. Similarly, trials conducted by multi-centre trial groups in the field of HIV infection in the United States appeared on average 4.3 years after the start of patient enrolment if results were statistically significant but took 6.5 years to be published if the results were negative (Ioannidis 1998). A recent study has found similar results (Decullier 2005).  The fact that a substantial proportion of studies remain unpublished even a decade after the study had been completed and analysed is troubling as potentially important information remains hidden from systematic reviewers and consumers.


Ioannidis and colleagues also found that trials with positive and negative results differed little in the time they took to complete follow-up (Ioannidis 1998). Rather, the time lag was attributable to differences in the time from completion to publication (Ioannidis 1998). These findings indicate that time lag bias may be introduced in systematic reviews even in situations when most or all studies will eventually be published. Studies with positive results will dominate the literature and introduce bias for several years until the negative, but equally important, results finally appear. Furthermore, rare adverse events are likely to be found later in the research process than short-term beneficial effects.