Objectives: To compare 2005 and 1995 ethics guidelines from journal editors to authors regarding requirements for institutional review board (IRB) approval and conflict-of-interest (COI) disclosure.
Design: A descriptive study of the ethics guidelines published in 103 English-language biomedical journals listed in the Abridged Index Medicus in 1995 and 2005. Each journal was reviewed by the principal author and one of four independent reviewers.
Results: During the period, the proportion of journals requiring IRB approval increased from 42% (95% CI 32.2% to 51.2%, p<0.001) to 76% (95% CI 66.4% to 83.1%, p<0.001). In 2005, an additional 9% referred to the Declaration of Helsinki or the International Committee of Medical Journal Editors’ Uniform requirements for ethical guidelines; 15% (95% CI 8.5% to 22.5%, p<0.01) provided ambiguous or no requirements. The proportion of journals requiring COI disclosure increased from 75% (95% CI 66.6% to 83.3%, p<0.05) to 94% (95% CI 89.4% to 98.6%, p<0.05); 41% had comprehensive requirements, while some addressed only funding source (6%), were vague (10%) or both (14%). Criteria for authorship rose from 40% (95% CI 30.5% to 49.5%, p<0.05) to 72% (95% CI 63.3% to 80.7%, p<0.05). Journals with higher impact factors were more likely to require IRB approval (p<0.01). Journals in anaesthesia and radiology all required IRB approval; requirements in other disciplines varied.
Conclusions: Instructions to authors regarding ethical standards have improved. Some remain incomplete, especially regarding the scope of disclosure of COI. The ethical guidelines presented to authors need further clarification and standardisation.
Statistics from Altmetric.com
The foundation for the conduct of human research is the Declaration of Helsinki and, in the USA, the Federal Policy for the Protection of Human Subjects (Common Rule).1 2 Both clearly stress the need for Institutional Review Board (IRB) or equivalent review committee assessment prior to commencing research involving human subjects. The International Committee of Medical Journal Editors’ (ICMJE) Uniform requirements for manuscripts submitted to biomedical journals stipulate that authors ascribe to the Declaration of Helsinki, thereby indirectly also requiring IRB approval.3 However, the stringency with which these ethical guidelines are adhered to varies.4–7 A 1997 study showed that fewer than half of journals required a statement of IRB approval as a prerequisite to publication of human research.4
Awareness of issues concerning the ethical standards in the conduct of research and its publication is increasing.8–10 However, it is unclear whether this intellectual consideration is reflected in improved ethical guidelines from journals to prospective authors. A literature review study comparing clinical trials published in five major medical journals before and after 1997 found improved reporting of informed consent and ethics committee approval, though a minority still mentioned neither.6 This has been confirmed in several smaller, specialty-focused studies.11–14
There has also been increasing attention in the scientific community directed towards the financial ties and other conflicts of interest of investigators.9 15–17 This has prompted many biomedical journals to review their own policies for authorial disclosure of potential conflicts and relationships with industry.9 It is unclear whether biomedical journals have improved their stated requirements for authors’ reporting of competing interests.9 16 17
Strict adherence to high ethical standards of publication is important for several reasons: (1) the scientific community is more apt to observe ethical standards if rules of publication forbid acceptance of non-compliant manuscripts, (2) high publication standards should be expected to support public confidence in the research endeavour and (3) journal editors should be interested in reinforcing high ethical standards in the interests of furthering high-quality science.
It is therefore essential to understand whether the completeness and clarity of ethical requirements provided to authors by journal editors have improved. In this study, we compare the current guidelines given to authors by journal editors regarding IRB approval and disclosure of conflict of interest (COI) with those published 10 years previously. The absence or lack of clarity of instructions to authors regarding ethical benchmarks for publication would suggest a need to develop standardised and comprehensive guidelines. These could be adopted by all journals publishing human research.
The methodology is based on Amdur and Biddle’s 1997 study design,4 with several important modifications, described below. Ethical approval was not obtained for this study, as it did not involve human subjects or data.
The sample consisted of 119 English-language journals from the 2005 Abridged Index Medicus (AIM) (last updated February 2005).18 (The AIM is a National Library of Medicine bibliography of major English-language biomedical journals.) Seventeen of these journals were not included in this study, as they do not publish original research involving human subjects. Since Amdur and Biddle’s study in 1997,4 two journals have been removed from the AIM listing. The Journal of Gerontology has four editions (Biological, Psychological, Medical Sciences, and Social Sciences), each with different instructions for authors, and each was evaluated separately. The journal Archives of Disease in Childhood has a Fetal and Neonatal Edition (with the same instructions for authors as the parent journal) and was evaluated as one journal. The current study focused on the 103 remaining journals from the 2005 version of the AIM. Hard-copy journal instructions were reviewed for 1995, while online instructions were used for the 2005 data collection. The investigator and reviewers were not blinded with regard to the year of the guideline being evaluated, as this was not feasible during the review.
We classified each journal into a medical specialty. The “mixed” category included those journals with multiple topic mandates (eg, British Medical Journal, JAMA, New England Journal of Medicine), and the “miscellaneous” category included journals from neurology, physical therapy and rehabilitation, and public and environmental health. Each journal was also classified by country of publication (USA, Canada, UK). The impact factor for each journal was obtained from the 2005 web-based listing of the Institute for Scientific Information’s Journal Citation Report.19
The instructions for authors of each journal were reviewed to determine the presence of specific guidelines regarding the ethical standards for reporting studies involving human subjects. Each journal’s most detailed instruction list was reviewed and assigned to one of the following categories: “IRB review”, “Uniform Requirements for Manuscripts Submitted to Medical Journals”, “Declaration of Helsinki”, or “informed consent”. Those without specific requirements were classified as “no guidelines” or “ambiguous”. This last category was added because a lack of clarity is equally problematic, leaving the “guideline” open for subjective interpretation on the part of the investigator. Examples of ambiguous guidelines included: “[journal] is concerned that the rights and dignity of all subjects involved in research be protected”, and “the corresponding author will be responsible ... that human or animal subject approval has been obtained.”
Documentation of compliance with journal requirements
Each journal’s method of assessing compliance with ethical guidelines was categorised as to whether the journal required a copy of an IRB letter, a manuscript statement or a separate signed statement, or assumed compliance (ie, no stipulated documentation of compliance).
COI disclosure and publication issues
We also examined journal guidelines regarding COI. We defined COI disclosure as the requirement to report any of the following nine domains: “employment or leadership position in commercial firm”, “employment as consultant for commercial firm”, “family connection”, “stock ownership”, “honoraria”, “research funding or grant support”, “expert testimony”, “patents” or “other remuneration”. “Very comprehensive conflict-of-interest disclosure” was defined as a requirement to report eight or more of the domains included above. Vague statements were defined as those that allowed for a sense of optional compliance or more than one interpretation. Statements classified as vague included: “the author will declare ... any financial connection to, or interest in, a company that might have biased the work”, and “anything that would embarrass you or any of your co-authors if it were to emerge after publication and you had not declared it”. We assessed how journal editors requested that COI be disclosed. Guidelines were categorised as requiring an author disclosure statement, a statement by authors in the manuscript, a statement by authors on the title or cover page, or as giving no specific instructions for disclosure.
The Consolidated Standards of Reporting Trials (CONSORT) statement, first published in 1996, was developed to improve the quality of randomised controlled trial reporting.20 We examined whether journal instructions in 2005 referred prospective authors to the CONSORT guidelines. Note was also made of any reference to authorship assignment and publication prohibitions in the instructions.
A principal reviewer (AR-L) classified and reviewed all 103 journal guidelines from both 1995 and 2005. In addition, four external reviewers each reviewed 51 (or 52) of the journal instructions from either 1995 or 2005, so that two independent reviewers reviewed each journal’s complete instructions from both years. The reviewers used a definition and classification template to ensure consistency. We calculated the inter-rater reliability between the principal reviewer’s 1995 data and those of Amdur and Biddle’s 1997 study, as well as the percentage agreement between AR-L and the reviewers for both the 1995 and 2005 journals.
Published guidelines related to the ethics of human research from the 103 AIM journals from both 1995 and 2005 are summarised in table 1. The proportion of journals requiring IRB approval increased from 42% (95% CI 32.2% to 51.2%, p<0.001) in 1995 to 76% (95% CI 66.4% to 83.1%, p<0.001) in 2005. The proportion of journals with ambiguous or no requirements related to ethical approval decreased from 29% in 1995 to 15% in 2005.
A kappa value of 0.90 (95% CI 0.84 to 0.97) showed high agreement between the primary reviewer and Amdur and Biddle’s reviews of 1995 journal instructions. The weighted kappa values evaluating inter-rater reliability between the primary reviewer and the external reviewers for 103 journals were 0.70 (95% CI 0.59 to 0.81) for 1995 and 0.73 (95% CI 0.59 to 0.88) for 2005, indicating consistent interpretation of the instructions to authors regarding IRB approval. When the data were analysed for inter-rater reliability based on whether IRB approval was required or not, the kappa values between the primary and external reviewers were 0.84 (95% CI 0.73 to 0.94) for 1995 and 0.70 (95% CI 0.53 to 0.86) for 2005. The percentage agreement based on whether IRB approval was required or not between the primary and external reviewers was 92% in 1995 and 89% in 2005.
Both the Uniform Requirements21 (updated 2006) and the Declaration of Helsinki1 (updated 2004) contain statements that stress the need for assessment by an IRB or equivalent prior to commencing research involving human subjects. IRB approval was therefore required by 62% of the journals studied in 1995 and 84% in 2005 (combined IRB, Uniform Requirements and Declaration of Helsinki). There has been a trend towards journals providing their own guidelines for IRB review rather than referring to established guidelines, with 20% of journals referring authors to the Declaration of Helsinki or the Uniform Requirements for ethical guidelines in 1995, and only 8% in 2005.
Of the seven journals with ambiguous guidelines in 1995, four required IRB approval in 2005, two still had ambiguous guidelines and one had no requirements with respect to research ethics approval. Of the 23 journals with no requirements in 1995, 12 required IRB approval a decade later, two required that the Declaration of Helsinki be followed and nine still had no requirements.
Documentation of compliance with journal requirements
The method of documenting IRB approval in those journals that specifically require ethics approval in their instructions is shown in table 2. The Journal of Bone and Joint Surgery (US edition) was the only journal to require that the IRB approval letter be provided on submission, and the American Journal of Obstetrics and Gynecology asked that it be available upon request. The four journals with ambiguous guidelines in 2005 assumed compliance for documentation.
Journal requirements by specialty
Most disciplines were variable in their stated ethical requirements for publishing studies involving humans (table 3). All specialties had improved since 1995. One-fifth of medicine journals and 17% of surgical journals still had either ambiguous or no requirements.
Journal requirements and impact factor
Those journals with higher impact factors from 2005 were more likely to require IRB approval (fig 1). Since the distribution was skewed, a non-parametric Mann–Whitney test was used (p<0.01).
Country of origin
The journals studied were published in the USA (85%), the UK (14%) or Canada (1%). There was a range of editorial policies regarding ethical guidelines in each jurisdiction; neither US nor British journals had a uniform approach in their requirement for IRB approval. Between 1995 and 2005, both the USA and the UK had improved their ethical requirements; the number of journals requiring IRB approval increased from 42% to 75% in the USA and from 43% to 86% in the UK. In the 10-year interval, the number of journals with ambiguous or no ethical requirements decreased from 30% to 15% in the USA and from 29% to 14% in the UK.
COI disclosure was addressed more frequently and in more detail in 2005 than in 1995, with 94% of journals (95% CI 89.4% to 98.6%, p<0.05) requiring a disclosure of COI in 2005 versus 75% (95% CI 66.6% to 83.3%, p<0.05) in 1995. The requirements for COI disclosure are classified in table 4. Note that in the 1995 instructions, 20% were vague about what constituted a reportable COI. A further 12% of journals asked authors to declare funding or grant support but had a vague statement regarding other COI disclosure. This left only 28% that specifically addressed other potential COIs, such as leadership positions, family connections, honoraria, etc. The 2005 instructions were typically more comprehensive (Table 4). However, a total of 24% had vague instructions with respect to what constituted a COI. This left only 64% of journals with instructions clearly addressing at least some potential COIs beyond funding. In 1995, only two journals had very comprehensive instructions regarding COI disclosure, while in 2005, 42 journals had very comprehensive instructions addressing at least eight domains of COI (see table 4, footnote).
There was variability in the requirements for reporting of COI disclosure. In 1995, of those journals that required COI disclosure, 53% required a statement in the title page or the cover page and 17% had an author disclosure statement. In 2005, 57% of journals had an author disclosure statement. Excluding those journals that required a declaration of funding to be included in the manuscript, only 10% of journals in 1995 and 21% in 2005 required that other conflicts of interest be disclosed in the text. There were no specific instructions for authors of the mechanism for disclosure of conflict of interest in 25% of 1995 journals and 14% of 2005 journals.
Authorship was addressed in 40% of 1995 journal instructions to authors (95% CI 30.5% to 49.5%, p<0.05), and 72% of the 2005 journals (95% CI 63.3% to 80.7%, p<0.05). The requirements for authorship were variable and often poorly defined, ranging from vague statements such as “authors must have contributed significantly to the study” to requiring a detailed statement of each individual author’s contribution. The majority of journals prohibited simultaneous and previous publication in their instructions to authors (90% and 88%, respectively, in 1995, and 96% and 95% in 2005). We found that only 37% of journal instructions stated that authors should adhere to the CONSORT guideline in their instructions to authors in 2005. As the CONSORT guidelines were first published in 1996, there were no comparison data for the 1995 cohort.
Three-quarters of the biomedical research journals in this study required IRB approval in their published instructions for authors. This represents an improvement over the decade.4–7 11–14 This improvement is likely to be attributable to the publication of more comprehensive instructions to authors and acknowledgement of recognised standards (Declaration of Helsinki and ICMJE’s Uniform Requirements). Furthermore, in the decade from 1995 to 2005, several bodies (the Council of Science Editors, the Committee on Publication Ethics and the World Association of Medical Editors) have published revised statements with more detail in the areas of IRB approval, COI disclosure and authorship issues.20 21 22 In 2004, the ICMJE introduced a requirement to have all clinical trials registered; this included the need for research ethics review.23 The most recent Uniform Requirements update (2006) has added the following statement, explicitly confirming the requirement for IRB review: “If doubt exists whether the research was conducted in accordance with the Helsinki Declaration, the authors must explain the rationale for their approach, and demonstrate that the institutional review body explicitly approved the doubtful aspects of the study.”3
A proportion of journals, however, continue to have either ambiguous or no stated requirements for ethical approval. This omission on the part of journal instructions to authors, and hence the editors, is significant. An editor’s requirement for IRB approval implies that the scientific community values the institutional review board’s contribution to protecting participants’ rights. Its inclusion suggests that ethical consideration has been given to the design and implementation of the study.
We found that COI disclosure was addressed in journal instructions to authors more frequently than a decade earlier, but that a substantial proportion of journals contained vague statements of disclosure and/or required only funding disclosure. In his 2003 study of 124 high-impact journals, Atlas found that COI (financial disclosure, commercial relationships and funding source) was covered in only 58.8% of journals investigated.7 Our findings show a further improvement but underline the fact COI disclosure remains narrow in scope.
The means by which COI is disclosed by authors to editors remains variable. The Uniform Requirements asks that all potential sources of COI be disclosed on a “conflict of interest notification page”. In our study, we found that only 57% of journals provided such a disclosure page for authors. These forms (such as those in the BMJ or JAMA) were the most complete with respect to their detail. We believe that investigators are often not in the best position to decide what constitutes a substantive COI that merits disclosure, and these declaration pages represent a thorough and objective solution.
In our study, it was noteworthy that many instructions to authors contained the phrase “the editor will discuss with the authors the manner in which this information will be communicated to the reader” with respect to COI reporting. The instructions for authors concerning disclosure of COI typically do not address the extent to which or how editors will disclose to the readers what has been reported to them.
The assignment of a journal to a particular category based on review of its instructions to authors is potentially subjective. We attempted to decrease this subjectivity by incorporating multiple external reviewers. The strong kappa values and percentages of agreement between reviewers were in the range that would be expected, given the methodology of evaluating and classifying the instructions to authors based on fixed criteria. In addition, this study focused on published journal guidelines and may not reflect actual editorial practice; it may fail to reveal more flexible interpretation of the rules or the application of non-published standards on the part of journal editors.
The results of this study substantiate a renewed call for clarification and standardisation of the ethical guidelines presented to authors from journal editors. There is significant opportunity for improvement and transparency that may enhance both the quality of disclosure of compliance with ethical standards and the public trust. Our recommendations include:
Journals should standardise their requirements with respect to the necessity of IRB approval and should stipulate consistent practice for its disclosure in the Methods section of published articles.
If journal editors choose to refer readers to the Uniform Requirements or the Declaration of Helsinki, they should make these documents easily accessible (by reprinting in the journal or via a web-link).
Journals should add IRB approval requirements to author, editorial, peer-reviewer, and CONSORT checklists to ensure its inclusion in the research process.
Journals should standardise conflict-of-interest elements to be disclosed by means of a template and describe how these will be reported within the journal.
Funding: This study was supported by a Category A Grant from the IWK Health Centre’s Research Office. The investigators also wish to recognise the external reviewers (Monique MacFarlane, Rob Clements, Kate Collins and Stephanie Carpentier) for their work in the data collection.
Competing interests: None.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.