Searches for NRS are often time-consuming. Search results contain large numbers of irrelevant citations and abstracts often do not provide adequate detail about NRS design (which are likely to be required to judge eligibility).
Review authors need to collect all of the data required for a systematic review of randomized trials (see Chapter 7, Section 7.3) and also data to describe (a) the features of the design of a primary study (see Section 13.2.2), (b) confounding factors considered and the methods used to control for confounding (see Section 13.1.3), (c) aspects of risk of bias specific for NRS (see Section 13.5.1) and (d) the results (see Section 13.6.1).
Review authors normally collect ‘raw’ information about the results when reviewing randomized trials, e.g. for a dichotomous outcome, the total number of participants and the number experiencing the outcome in each group. If participants are randomized to groups, a comparison of these raw data is assumed to be unbiased. For a NRS, a comparison of the same raw data is ‘unadjusted’ and susceptible to confounding. Authors usually also report an ‘adjusted’ comparison estimated from a regression model which cannot be summarized in the same way. Review authors should still record the sample size recruited to each group, and the number analyzed and the number of events, but also need to document any adjusted effect estimates their standard errors or confidence intervals. These data can be used to display adjusted effect estimates and their precision in forest plots and, if appropriate, to pool data across studies.
Anecdotally, review authors’ experience is that NRS are poorly reported so that finding the required information is difficult, and different review authors may extract different information from the same paper. Data collection forms may need to be customized to the research question being investigated. Because of the diversity of potentially eligible studies and the ways in which they are reported, developing the data collection form can require several iterations in the course of reviewing a sample of primary studies. It is almost impossible to finalize these forms in advance.
Results in NRS may be presented using different measures of effect and uncertainty or statistical significance depending on the reporting style and analyses undertaken. Expert statistical advice may assist review authors to transform or ‘work back’ from the information provided in a paper to obtain a consistent effect measure across studies. Data collection sheets need to be able to handle the different kinds of information about study findings that authors may encounter.