All forms should be pilot tested using a representative sample of the studies to be reviewed. This testing may identify data that are missing from the form, or likely to be superfluous. It is wise to draft entries for the ‘Characteristics of included studies’ table (Chapter 11, Section 11.2.2) and the ‘Risk of bias’ table (Chapter 8, Section 8.5) using these pilot reports. Users of the form may provide feedback that certain coding instructions are confusing or incomplete (e.g. a list of options may not cover all situations). A consensus between review authors may be required before the form is modified to avoid any misunderstandings or later disagreements. It might be necessary to repeat the pilot testing on a new set of reports if major changes are needed after the first testing.
Problems with the data collection form will occasionally surface after pilot testing has been completed and the form may need to be revised after data extraction has started. In fact, it is common for a data collection form to require modifications after it has been piloted. When changes are made to the form or coding instructions, it may be necessary to return to reports that have already undergone data extraction. In some situations, it may only be necessary to clarify coding instructions without modifying the actual data collection form.
Some have proposed that some information in a report, such as its authors, be blinded to the review author prior to data extraction and assessment of risk of bias (Jadad 1996); see also Chapter 8 (Section 8.3.4). However, blinding of review authors to aspects of study reports is not generally recommended for Cochrane reviews (Berlin 1997).