Introduction: Evidence-based medicine relies on reproducible, high-quality reporting in the literature. Previous evaluations, which have assessed 11 basic elements of design and analysis in top impact clinical journals (both nonsurgical and surgical), have demonstrated that the reporting quality is less than perfect, although improving. The current study evaluates the quality of reporting in Diseases of the Colon and Rectum and other clinically related journals to identify specific areas where future improvements may be made.
Methods: Two independent evaluators assessed all randomized, controlled trials published in Diseases of the Colon and Rectum in the years 1990, 1995, and 2000. Additional assessments for 2000 were performed on all randomized, controlled trials published in Annals of Surgery, Archives of Surgery, and Gastroenterology. The frequency of reporting of 11 explicitly defined, traditionally important, basic elements of design and analysis were determined. These elements included reporting of eligibility criteria, admission before allocation, randomization (and method), blinded assessment (patient and observer), complications, loss to follow-up, statistical approach and tests, and power calculation.
Results: Interobserver reliability was strong (kappa, 0.76). The number of randomized, controlled trials published in Diseases of the Colon and Rectum increased from 5 (in 1990) to 13 (in 1995) to 17 (in 2000). Of the 1990 randomized, controlled trials, an average of 60 percent of the 11 basic elements were reported. Of the 1995 randomized, controlled trials, 72 percent of the items were reported (P = 0.05), whereas of the 2000 randomized, controlled trials, 77 percent of the 11 items were reported (P < 0.002 vs. 1990). The best-reported items were eligibility criteria, discussion of statistical tests, and accounting for all patients lost to follow-up. Only 11 percent of the 2000 randomized, controlled trials reported statistical power calculations. For the other journals that were evaluated, 72 to 88 percent of items were reported, with eligibility criterion being the best consistently reported item and power calculation being the worst.
Conclusions: For Diseases of the Colon and Rectum, the number of randomized, controlled trials and the quality of reporting is improving. However, although certain research standards are reported adequately, others are not. The calculation of statistical power is clearly important when interpreting randomized, controlled trial results (whether differences are reported or not), yet only 11 percent of studies contained this information. Improving the reporting of this single item would likely lead to improving the overall quality of clinical studies in colorectal surgery. Improved reporting might be best facilitated by having authors adhere to a list of explicitly determined elements that should be included.