Research ethics committees (RECs) are regularly subjected to scrutiny in the medical journals.1^–10 Many researcher complaints focus on the apparently “nit-picking” nature of the revisions prescribed by RECs before a favourable opinion is issued.11^–13 However, there has been little systematic examination of the types of process “errors” RECs ask researchers to rectify, as distinct from the ethical issues they ask them to resolve. This paper presents an analysis of the types of process errors that UK RECs identified in applications from researchers.

METHODS

We analysed letters written by NHS RECs in the UK to applicants following the first meeting at which an application was considered. These letters are uploaded to the Research Ethics Database maintained by the National Research Ethics Service (NRES).

RECs can make three possible decisions when they first consider applications: favourable, provisional, or unfavourable (box 1).

We focused on applications that received either an unfavourable or provisional decision at first review, as such decisions indicate that there was something in the application that has troubled the REC and requires a response from the applicants before a final decision can be rendered. The final outcome was recorded where available.

Criteria for inclusion of a letter in our sample were as follows:

• The letter conveyed a “provisional” or “unfavourable” opinion.

• The letter concerned an application considered by a REC for the first time during our “eligible periods”: July 2005, October 2005, January 2006 and April 2006. These periods were chosen to minimise seasonal effects in application submission, such as an increase in student projects prior to the academic year, and so on.

Fifty-five RECs did not upload letters to the research ethics database during our eligible periods and were excluded, leaving 115 RECs from which letters could be sampled. The first letter that met our eligibility criteria for each of the 115 RECs was chosen for inclusion in the study. Unfavourable opinions were purposefully over-represented to yield sufficient letters for analysis, so that they formed 20% of the initial sample. The remaining 80% of letters were those applications with provisional opinions. In addition, because applications that were initially “provisional” but were subsequently issued with an unfavourable opinion were of particular interest, this type of application was also purposefully over-sampled by including all such applications between March 2004 and July 2006 for which a letter was available.

In order to analyse the letters, the techniques of ethnographic content analysis were employed, which involves the development of a coding scheme or framework grounded in the data (in contrast to approaches that use a pre-specified framework).14 Our framework was generated initially through close inspection and comparison across the texts of letters used in a previous study,15 but was modified extensively in response to the new data in this project. Explicit specifications were devised to aid data assignment to analytic categories, a process which was facilitated by the use of QSR N6 software (QSR International, Melbourne, Australia).

Our project was deemed by the NRES not to require research ethics committee review as it was considered to be service evaluation. Letters were fully anonymised before analysis, and where appropriate, in illustrative quotations, identifying details have been modified or removed.

RESULTS

Our sampling identified 141 letters for analysis (table 1). There are some indications that decisions may vary between different types of study.

In other papers15 16 we have reported on the ethical issues raised by RECs in their letters about these applications; in this paper, we were concerned with the issues that were not (strictly speaking) “ethical” in character, but instead referred to various kinds of process errors which RECs perceived as having been made by the applicants (though we recognise that such errors may have ethical dimensions or consequences). We are using the term “process error” here to describe lapses in paperwork, application processes or management issues. A process error by applicants was pointed out by the REC in 123 (87%) of the 141 letters. Of the applications identified as having an error, 101 were given a provisional opinion and 22 an unfavourable opinion.

We classified errors into four types: procedural violations, missing information, slip-ups and discrepancies.

Procedural violations

Failure of applicants to comply with correct procedures was noted in 104 (74%) letters. Problems identified by RECs included failure to register trials properly, to follow NRES application requirements, to obtain necessary signatures, and so on.

The Committee suggests that the trial should be registered as a randomised controlled trial and gain an ISRCTN number. (Letter 11, provisional opinion)

.... this study has been identified as a multi-site study. However if this had been the case, it would have been necessary to submit the study via the Central Allocation System, which did not happen. (Letter 35, provisional opinion)

Missing information

REC letters pointed out missing or inadequate information in 96 (68%) of the 141 letters in our sample. Types of missing information included entire documents (eg, participant information leaflets) or failure to provide full information within documents (such as missing details from the protocol).

The Committee noted that the study would involve healthy adults, but an information sheet for the control group had not been provided. Any resubmission of this proposal must be accompanied by copies of the complete documentation. (Letter 51, unfavourable opinion)

The Committee felt there was insufficient information available for a decision to be made on this proposal (eg, it would appear that no literature review has been done nor have the current procedures and the rationale behind them been explained). (Letter 112, unfavourable opinion)

Slip-ups

“Slip-ups”, including minor errors such as errors in grammar and spelling as well as typos, failing to tick boxes, and so on, were noted by RECs in 62 (44%) letters.

In the protocol there is a spelling error on Page 4, Question 4: “there” should be “their”. This is duplicated in the Research to be Undertaken section. (Letter 49, provisional opinion)

A24 of the application form was missed out. Healthy volunteers should be ticked. (Letter 52, provisional opinion)

Discrepancies

References to discrepancies between or within documents submitted to the REC were found in 35 (25%) letters. Examples included inconsistencies in descriptions of aspects of the research between various parts of the form, differences in the description of the research intervention between the application form and the participant information leaflet, and so on.

Flow Chart should be the same as [answer] given in A20, ie, participants should only be contacted by post in the first instance. (Letter 20, provisional opinion)

Please advise if results will automatically be given to participants following the completion of the study? Patient information sheet and application form provide conflicting answers. (Letter 32, provisional opinion)

Errors by type of applicant

There appeared to be some variability in the number of errors identified in applications according to job role of the chief investigator (table 2), though the nature of our sample limits the generalisability of these findings.

DISCUSSION

These data show that RECs are often concerned with process errors that researchers make in their applications. RECs identified such problems in most (87%) applications that did not get a favourable opinion at first review.

It could be argued that what is happening in the letters when RECs point out process errors (especially those that are perhaps “non-ethical” in nature) is primarily a display of RECs’ care and attention, and of their conscientious execution of their duties (down to the last spelling mistake).16 This could be seen as evidence of “control-freakery” and “nit-picking”,17 and part of a more general tendency for RECs to concern themselves with issues not strictly within their remit, including scientific design.18 A perhaps kinder interpretation might be that REC members may feel themselves responsible for ensuring that approaches made to patients are correct in every detail. Further research would be needed to explore the processes of RECs and the views and intentions of their members.

What RECs should do about process errors is an important question. Some errors, such as the “slip-ups”, are perhaps rather trivial, in the sense that they can easily be corrected. Such errors alone, though undesirable and possibly indicating a certain lack of care, perhaps should not hold up the granting of a favourable opinion for a study but instead their correction should be a condition of approval. Missing information is a rather different problem, since it is arguably not unreasonable for a REC to wish to see all relevant information before making a judgement on the ethical provenance of a study. Indeed, a missing information leaflet would prevent a REC from fulfilling one of its central roles—that of ensuring that the consent process is appropriate. Similarly, discrepancies are of concern to RECs, since it may not be clear which plan the researcher intends to follow if there is more than one version in play. Finally, it may not be possible for RECs to tolerate procedural violations if they are to ensure consistency and fairness, as well as account for their actions. In addition there may be serious consequences for the researchers if they do not register their clinical trial or gain approval from the appropriate regulator(s).

A further important question clearly concerns why the level of error identified by RECs is so high. It could be argued that the rules are too complex and demanding, and that some level of error is therefore inevitable. This explanation would require further investigation, particularly focused on how processes could be simplified and streamlined. It might also be the case that better administrative screening of applications before they reach the committee stage would help to reduce the number of errors seen by RECs. Of course, it could also be argued that better thoroughness and attention by researchers would also help to bring down the error rate.

Our data do not allow conclusions to be drawn about the extent to which “errors”, rather than ethical troubles, can lead to deferring or withholding of favourable opinions from RECs. However, it is not impossible that RECs interpret process errors as indicating possible incompetence, lack of attention or lack of respect, and thus may tend to be less inclined to be sympathetic to such applications. What is perhaps clear is that avoiding mistakes, oversights and omissions is in researchers’ interests, not least because they are probably mostly avoidable. Future research might investigate whether RECs vary systematically in their identification of errors, as well as how researchers can be best supported to avoid making errors.