Introduction

Methodological quality is the first ethical requirement in clinical trials.¹ Moreover, the approval of a research ethics committee (REC) and obtaining informed consent from patients (ICP) could be considered the main issues in the ethics of research with human beings.²

Both methodological and ethical aspects should be adequately reported to enable readers to make an accurate assessment of the clinical research about which they are reading.^{3, 4} However, there are several studies that have found deficiencies in reporting the design and conduct of trials.^5–10 It has also been shown that disclosure of the ICP and REC approval in published reports is sometimes incomplete, and sometimes omitted.^11–19 Nevertheless, very few studies have assessed the relationship between omission of REC or ICP and methodological quality of trials. We present the main results of a systematic evaluation of 767 clinical trials where the disclosure of REC approval and ICP was assessed, as well as the association between reporting these aspects and the methodological quality of the trials.

Methods

Two short preliminary reports, including some of our methods, have been published elsewhere.^{17, 20} They addressed different issues from the present study. A standardised protocol was applied for reviewing every article of trials published between 1993-1995 in the New England Journal of Medicine, the Lancet, the Journal of the American Medical Association, and the British Medical Journal. To identify eligible articles, all issues of these journals were hand-searched. In addition, the MEDLINE bibliographic database (1993-1996) was used to identify all trials published in other journals during 1993-1995 by at least one Spanish author, for the purpose of evaluating clinical trials conducted in our country. We included every publication of experimental trials on humans that had two or more study groups²¹ and all studies labelled as “clinical trials”, “field trials” or “randomised trials” by the authors. The exclusion criteria were: 1) non-experimental (observational) studies; 2) studies assessing a diagnostic test to estimate its sensitivity, specificity or other validity measures; 3) trials published as letters to the editor or short articles, and 4) secondary analyses of previously published studies.

The standardised protocol for assessing each trial and the process of data collection were based on a checklist (available from authors) that was constructed after reviewing 43 previously published checklists.²² After a pilot study, the selection of items to be included was performed with the help of an ad hoc expert panel. Some of the items selected were: year of publication; type of outcome to be assessed and end point(s) of the trial; number of centres participating in the study; country(ies) of origin; specialty of authors; source of funding; number of patients whose participation was requested; number of randomised participants; methods used to generate the allocation scheme and method for concealment of allocation; baseline comparison of groups; whether a hypothesis test was used in this comparison (hypothesis testing being inappropriate to exclude confounding²³;) type of treatment and control; blinding in the assessment of outcomes; justification of opened (unblinded) trials; information on statistical methods; sample size estimation; rules and monitoring to stop the trial after interim analyses, and disclosure of REC approval and ICP.

When information on REC approval or ICP was missing in an article, we sent the corresponding authors a standardised questionnaire enquiring whether the trial had in fact been approved by an REC and whether the participants had provided informed consent. If the response was negative, we requested an explanation as to why REC approval or ICP was not obtained. After two months, we repeated the mailing to those who had not replied.

For each trial, one of us (MRC) completed the evaluation of all relevant aspects of trial design, methods and ethical issues. Another researcher (MAM) independently re-evaluated a randomly selected sample of 30 trials while blinded to the results of the first assessment. The between-raters agreement was assessed by computing the kappa index. In the subsequent evaluation, any doubt as to how to classify an item was discussed between the investigators mentioned above. Items with a kappa index less than 0.35 were not used in subsequent analyses.

A multivariable logistic regression model was fitted to assess the methodological characteristics independently related to disclosing those ethical requirements (REC/ICP). In this model, we considered the failure to report that either REC or ICP had been obtained, or the explicit disclosure in the original paper that any of these requirements were not obtained, as the outcome. Multivariable analysis was performed with the aim of controlling for several variables that could be considered as potential confounders. In addition, we considered all indicators of methodological quality with a p value <0.25 in the univariate analysis as candidate variables for the multivariable model. After including these potential predictors and/or confounders in one multivariable model, the final model was obtained by gradually removing non-significant variables using the likelihood ratio test and by evaluating potential confounding with the criterion of a 10% change between crude and adjusted estimates of the odds ratios.

Our study was not approved by an REC, nor did we request informed consent from the authors of the articles because our research did not involve an experimental design using human beings.

Results

A brief descriptive assessment of the proportion of studies failing to report the ethical requirements has been published elsewhere.¹⁷ Table 1 shows the study quality indicators that were identified in the multivariable analysis as independent predictors of failure to report information about REC approval, ICP, or both. In addition to the variables shown in the table, the analysis was also adjusted for the length of the article, the journal in which the article was published, the country of authors, the number of participants and the source of funding.

Once adjusted for the other variables, an open randomisation procedure (failure to conceal allocation) was independently associated with not presenting information on ICP and/or approval by an REC. Among open trials, those that did not justify the unblinded assessment of the outcome were more likely not to report ICP/REC. Trials which did not use a treatment for the patients in the control group omitted this information significantly more often. Not disclosing information on statistical methods, nor on sample size estimation; not providing the rules to stop the trial, and not presenting a comparison of the distribution of baseline factors were characteristics independently and more frequently associated with the omission of ICP/REC. Trials with survival of patients as the outcome were less likely to provide information about these ethical issues.

When we fitted the logistic models using the actual responses of authors to our queries (mailed questionnaires) as the outcome, and reclassified the authors who replied that they had in fact obtained the ICP/REC in spite of not having reported it, the adjusted odds ratios for most of the indicators of lower quality remained high: no justification of unblinded trials (OR=2.62); no information about statistical methods (OR=1.62); omitting sample size estimation (OR=1.34), and not reporting the rules to stop the trial (OR=1.43).

When the two ethical issues were considered as separate outcomes, all associations maintained the same direction and most of them remained consistent and statistically significant (see table 2).

Discussion

The hypothesis of this study was that the higher the methodological and scientific quality of a trial, the more attention was given to its ethical aspects. Our findings suggest that, independently of the journal where they were published and the length of the report (among other descriptive variables), trials with higher standards of quality met the requirement of reporting ethical issues more frequently. Other authors have previously assessed some of these standards as indicators of quality (concealment of allocation,⁶ sample size estimation,^{7, 8} statistical methods,⁹ and justification of open trials,¹⁰) but have not related them to ethical aspects. Our results are consistent with those of another study regarding the association between lower quality and frequent omission of reporting REC approval in studies concerning cardiopulmonary resuscitation. Nevertheless, the assessment of methodological quality in that study was only based on the randomisation scheme.¹⁴

The ethical requirements were not mentioned in some articles, even though the authors responded to our questionnaire that they had in fact requested them. However, we chose the statements in the original article (or lack thereof) over the actual response of the authors, in terms of the information provided on ICP requested or REC approval. This decision was based on several reasons: a) This may promote more reporting of ethical issues in publications of clinical trials in the future; b) Some authors did not respond to our questionnaires, and therefore we could not ascertain what they had actually done in their studies, and c) Failing to report the ethical requirements in the original article may already imply a lower concern for ethical issues in trials.¹¹ When we fitted the logistic model using the actual responses of authors as the outcome, the adjusted odds ratios for most of the indicators of lower quality remained high. Therefore, not only the actual reporting of ethical issues in articles, but also the verified request for ICP or REC approval, showed an association with higher methodological quality.

We selected four journals that have had representatives on the International Committee of Medical Journal Editors (ICMJE) since its inception. This does not imply that these journals consistently follow the uniform requirements in all cases, but these are leading journals which publish research reports in all fields and have a broad readership. Medical journals which are published weekly also provide a large enough sample to assess the current trends in our topic of interest with precision. Reviewing trials with Spanish authors also has some advantages because it provides a subset of trials resembling a comparison (“pseudo-control”) group. Most trials in this group were published in specialised journals written in English.

Our results, in which trials with higher quality standards were more likely to give attention to ethical aspects, are reassuring because readers could consider these studies as models and examples of good research. It is important to note that in certain clinical trials, obtaining ICP may be more difficult²⁴ because of the patient's condition (ie emergency room or intensive care), or because the characteristics of the intervention may be more suitable for group rather than for individual randomisation (ie schools randomised to usual care or a special nutrition education programme). It is also possible that trials with survival of patients as the outcome omitted information about ICP/REC more often because authors considered it was implicitly assumed. However, the reasons why informed consent and REC approval were or were not obtained should always be stated.

Some authors have indicated the relevance and importance of complete reporting, as well as the importance of adequate methodological and ethical quality of clinical trials.^{11, 25} This reporting should reflect both the weak and the strong aspects of researchers' work. It could be considered as evidence of the concern for ethical aspects that must be present not only during the conception, design, and conduct of the research process,²⁶ but also in the publication and communication of results.²⁷ The association between methodological quality and reporting of ethical requirements probably reflects the respect shown for patients during the whole research process.