The medical research ethics review system in the UK is perhaps best described as having been in transition for the past 15 years.1 2 The changes have been driven by user dissatisfaction, refinement of the system and legal changes.3 4 This paper aims to explore whether the latest suggestions for improvement found within two documents of the Central Office for Research Ethics Committees (COREC)—Implementing the recommendations of the Report of the Ad Hoc Advisory Group on the Operation of NHS Research Ethics Committees: a consultation5 (here referred to as “the consultation plan”); and Building on improvement: implementing the recommendations of the Report of the Ad Hoc Advisory Group on the Operation of NHS Research Ethics Committees)6 (“the implementation plan”) will achieve the aimed-for goal of increased efficiency. (Details about the now closed consultation can be found at http://www.nves.npsa.nhs.uk/recs/about/develop.html consultation) This is important because of the impacts that the efficiency of the ethics review system has on the development of new medical initiatives in the UK.7

Unfortunately, I conclude that the changes proposed within the implementation plan are unlikely to significantly increase the efficiency of the National Health Service (NHS) ethics review system in most cases, and in some cases may actually worsen it. Furthermore, the proposed changes are likely to introduce specific issues of fairness. Finally, there is little solid evidence one way or the other regarding the efficiency of this new system. While I argue that these reforms will not introduce remarkable increased efficiencies into the system and will introduce some inefficiencies, the true effects cannot be determined for certain without trialling the system. Remarkably, however, in these days of evidence-based practices, trials of this new system have to the best of my knowledge not yet been carried out, but instead will be carried out as part of the now-agreed-upon implementation process. In other words, we are implementing a system that is supposed to improve efficiency with little evidence that it will in fact do this.

THE RELEVANT CHANGES

At present, all NHS-related research involving people, whether they are patients, staff or care givers, and even if the research is simply conducted on NHS property, is required to receive ethical approval by an NHS Research Ethics Committee (REC).8 Crucial to the whole implementation plan is the introduction of a research ethics service, with national research ethics advisors (NREAs) playing a gatekeeper role. NREAs would be able to approve applications which do not have material ethical issues without the application needing to go through the full review of an ethics committee. This is based on the notion of proportionality of review: the amount of ethical scrutiny on projects should increase as the risks posed by a project increase. As the consultation states,5

Ethics committees would be one part of the decision making process but applications would only be referred for full ethics committee review where there are ethical issues requiring debate: National Research Ethics Advisers would be able to recommend and issue favorable opinions of studies presenting no material ethical issues. (p 14)

The implementation plan,6 in response to concerns raised by the REC community, changes this slightly in that an executive subcommittee, rather than individuals, will have the ability to approve applications with minimal ethical concerns:

… a new independent group of national research ethics advisers will work as an executive sub-committee in making decisions on applications with minimal ethical dimensions in a timely manner. (p 2)

This is a radical change from the current system, which is based solely on the notion of full ethics committees. The proposed procedure is also intended to be the primary means through which greater efficiency gains will be made within the system. (Although the report also recommends further training for members of RECs and further refinement and improvement of the forms, which should aid efficiency, these initiatives are already proceeding.) The gains in efficiency are supposed to be achieved in two ways. First, applications that present no material concerns will be processed faster, since they will be approved by an NREA or executive subcommittee without having to wait for the availability of a normal review committee. Second, projects that fall outside the purview of the NHS committees and applications in which there is a need to explain and address scientific issues more clearly will be weeded out by a filtering system before the applications reach an ethics committee, saving time for all involved.

The NREA executive subcommittees are likely to react more quickly than a standard REC, since

[t]he proposal is for NREAs to consider applications perceived to be low risk by forming an executive sub-committee of a minimum of two members, one of whom will normally be required to be a lay member. However, in order to facilitate a fast tracked review, decisions will be reached by virtual meetings consisting of audited email exchanges or documented telephone conversations. The NREAs will meet formally from time to time to review the processes involved.6

How this filtering system is intended to work is that when an application comes in, the coordinator reads through it and decides whether it needs review via a committee or is suitable for filtering. Those applications which are assessed as suitable for filtering are then reassessed by a senior coordinator, who may decide to send it to a committee instead. If they don’t, then it goes to an NREA, who decides either that it is appropriate for this application to be considered by the executive subcommittee or that it needs to be referred to a proper REC. Finally, the executive subcommittee convenes and decides either to approve the application or that it raises significant ethical concerns and needs to be considered by an REC. The decision would be made in much the same way as a decision by a normal REC, with open discussion or debate, mediated perhaps by the meeting perhaps being “virtual”—carried out by telephone or email, as some subcommittee meetings are carried out at present.

The robustness of this model is clearly reliant on the NREAs.9 Comments from the existing members of research ethics committees have indicated there is concern whether any one person or small group of individuals (since the implementation plan indicates that it is anticipated to need only eight NREAs and that a minimum of two would be required to make a decision) could have the appropriate level of scientific, medical and ethical expertise to be able to evaluate the range of specialist research across clinical specialties and research methodologies encountered in medical research.9^–11 Likewise, there is some debate about whether this system of ethical review meets the requirements of the World Medical Association’s Declaration of Helsinki, which states (paragraph B13):

The design and performance of each experimental procedure involving human subjects should be clearly formulated in an experimental protocol. This protocol should be submitted for consideration, comment, guidance, and where appropriate, approval to a specially appointed ethical review committee, which must be independent of the investigator, the sponsor or any other kind of undue influence.12

For this reason, the Scottish Ethics Advisory Group has rejected the proposal that some applications can be approved without committee approval.13 Despite these other valid concerns, it is important to ask whether the suggested changes are actually likely to increase efficiency significantly.

INCREASED EFFICIENCY?

Something the implementation plan lacks is any indication of evidence of how many applications are likely to be considered to have no material ethical issues and thus suitable to be approved by an NREA. This is surprising, because the increase of administrative burden proposed by the implementation plan is quite significant. As indicated above, all applications will be filtered first by coordinators who at present screen applications only to make sure that all the required elements are in place (as specified in the standard operating procedures for these committees) (1.48–1.53).14 To determine whether the application presents any material ethical concerns, presumably it will need to be read in great detail by the coordinator. Since applications are typically 40 to 60 pages long, this is a considerable expansion of their duties. Furthermore, it will require considerable training or very clear guidelines to enable coordinators to correctly identify applications with material ethical concerns. This is essential to avoid the system’s ending up with significant levels of double handling of applications—for example, an application being treated as if it has no significant ethical concerns until it reaches the NREA, who then determines that it requires booking in for a Committee review, substantially delaying the application. This increased administrative burden is going to delay all applications at least slightly, perhaps adding as much as a day to the time each application takes to process (given the general increased workload of the coordinators and the time taken to read in depth an application up to 60 pages long). In a system that is currently averaging around 30 days for an application to be processed, the addition of a day to the processing of each is not insignificant.15

However, this general cost might be outweighed if a significant proportion of applications were going to be sped up considerably by the new system, which, after all, is aimed at speeding up those applications with minimal ethical concerns. Determining this would require knowledge of two elements: how long it will take for applications that have been triaged to get approval, and how many applications will be suitable for triage.

HOW LONG?

Determining how long approval would take really can only be done by trialling the system. The implementation plan suggests:

The opportunity for delivering efficiency savings through screening is demonstrated from management information. This shows that while decisions are delivered within an average of 40 days, in most cases 14 to 21 days of this period are taken up before the REC meeting (p 11).6

In the proposed system, filtered applications get examined in depth four times: (1) the coordinator filters out those applications which clearly present material ethical concerns for consideration by a committee; (2) the applications get re-filtered by senior coordinators and then (3) go on to the NREAs; (4) finally, the NREA executive subcommittee decides to either approve the application or send it to a REC. Given this, and the time it takes for reports to be filled out, and for applications to travel from one desk/location to another, it is not unreasonable to presume that it will take applications at least 15 days to make it through the new system, even when they are very straightforward. This is particularly the case because the NREAs are intended to be retained only part time and are likely to have busy schedules, into which executive subcommittee meetings will need to be fitted.

Though it is no more than anecdotal evidence, I note that at the University of Ulster, my school’s research ethics filter committee (of which I was vice-chair) operated as a virtual committee for a year. While this was more efficient than sitting as a full committee, it still typically took us about 2 weeks to reach a final decision on a project, and this was with far briefer applications (approximately 15 to 20 pages), and the application was only going through one layer of scrutiny rather than the four layers proposed in the implementation plan.

While 15 days is not definitive, it seems a defensible enough figure to work with, given the expectation that 14 to 21 days could be saved. I will also use the extremely optimistic figure of 5 days to give a balanced view.

HOW MANY?

Determining the second factor, how many applications will be suitable for triage, again can only be done effectively by trialling the system. There is, however, some tension raised by the words “no material ethical issues” (emphasis added). The more broadly “ethical issues” is defined, the greater the number of applications that will be triaged.5 There is, however, a trade-off. The committee model is widely recognised as the most robust method of ethics review. The more applications bypass this route, the more likely it is that applications which ought to have received full ethics review will not receive an appropriate level of review. Certain types of application will never be filtered—for example, those required by law to be reviewed by a full committee, such as clinical trials of medicinal products or devices—and arguably some applications required to be submitted by the Human Tissue Act 2004 and the Mental Capacity Act 2005.

We can perhaps get some preliminary indications of which applications do not have significant ethical concerns by looking at the quantity of applications which receive a favourable outcome immediately from a research ethics committee. However this should only be treated as indicative, since for at least some of these favourable outcomes, the applications may well have had material ethical concerns that the committee considered were well addressed within the application, or that were necessary to generate the great benefits that the research might allow, and so the application might have been approved despite significant ethical concerns.

Of the 5420 applications received between April and September 2005—on average 135 per month—about 15% received favourable outcomes.16 At best, then, we can hope that the new system will save time for 15% of applications.

Given a reasonable precautionary approach, which one hopes would be adopted at each stage of the filtering, and given that some of the applications that received favourable opinions would be required for statutory reasons to seek full committee approval, it is likely that fewer than 15% of applications will actually be filtered. This is backed up by the implementation plan’s anticipated need of approximately eight NREAs (p 10).6 Given that the REC system received over 13 000 applications in 2004, they are presumably allowing for a filtering rate lower than 15%.17 Otherwise an eight-person team will be making final decisions on about 1950 applications a year! Even split into four two-person teams, this is still 487 applications per team. This seems unrealistic, given that a full REC looks at 70 to 80 applications a year, and that the NREAs will be employed part time. I will show below what will be the case in the best-case scenario of 15% of applications filtered and the more realistic, but less charitable, 5%.

ERRORS

A significant factor which is likely to reduce the efficiency of the new system is the existence of errors. When I refer to error, I don’t mean to imply that the NREA executive subcommittee will make errors in its final approval of applications, though from time to time I am sure they will, since everyone is fallible. Rather, I refer to the inevitable mis-categorisation that will sometimes happen. Determining whether something raises ethical issues is a tricky business, and the system as it is set up takes account of this; that is why there are four layers of filtering before a final decision is made. But at each of those points, some applications that at the previous filter were thought not to raise serious ethical issues will be identified, rightly or wrongly, as raising serious ethical issues and bounced to a full committee. It is unlikely that these applications will instantly go to a full REC; instead they will join the queue as other applications do. The net effect of this is difficult to determine, because it depends where in the filtering system these applications are caught. However, those applications caught only at the NREA phase or, worse still, the NREA executive subcommittee phase, will take a longer rather than a shorter time going through the system, since they are likely to enter the REC queuing system 14 days after being identified as raising ethical issues off the bat. This is particularly significant because at present the NREA executive subcommittee is envisaged to have the power only to either approve an application or to send it to a full REC (p 11).6 This means that if there is something easily addressable that is wrong with the application, the NREAs cannot simply require the relevant changes to be made, but instead must send it to a full committee.

I’m going to assume that serious errors will be made with 1% of the applications, and that these will cause applications to take an extra four days. It is almost certain that the incidence of serious errors will be lower than 1%, but given that the effect of these is likely to be greater than four days, and that less serious errors, such as misdirection being picked up by the senior advisor, are likely to be more abundant, this doesn’t seem an unreasonable or uncharitable simplification to use.

DOING THE MATHS

If we assume that the filtering system will take on average fifteen days to produce a final decision and that 15% of all applications will be appropriate to be filtered, then the system will provide some slight improvement in terms of efficiency and applications will, on average, be sped up by a day (see box) in comparison with the present 30 days. Even if the processing time of the filtered applications is a heroic five days, on average, only two and a half days will be saved. If filtering is appropriate for only 5% of all applications, the new system will take either slightly more or slightly less time—slightly less than a day’s difference either way—than the present system, depending on how long the filtering system takes to process applications (see box).

FAIRNESS

As I have indicated, this is likely to come at the general cost of the majority of applications, which will take longer to process because coordinators will have to read them in depth to ascertain whether filtering them is appropriate. This raises a further issue of fairness: should all researchers be delayed so that some applications can be sped up? Is there a good reason that the burdens of improving the system for a small subsection of the research community (between 15% and 5%) should fall on the rest of the research community?

Furthermore, that burden falls on both the research community and those who benefit from the research, mainly patients. While I do not want to fall into the fallacy of suggesting that studies that are likely to raise few ethical concerns (presumably qualitative and social-science-type research for the most part, although clearly such studies can raise significant ethical concerns) are also likely to have low benefits, the question does have to be asked whether processing these applications much faster warrants making 85% to 95% of medical research slightly slower, with the effects this is likely to have for patient mortality and morbidity.

Some applications, namely those that are wrongly filtered in the first place, are likely to spend considerably longer being processed, since they will first go through some stages of the triage system and then have to enter the queue for a place at an ethics committee meeting. Again there is an issue of fairness to researchers and also of the potential impact of this delayed research on patient mortality and morbidity.

CONCLUSIONS

While in practice, then, the recommendations of the implementation plan may increase efficiency for the minority of research applications, unfortunately it seems that it will slow down most applications that need to be reviewed by a full committee. Furthermore, the efficiency of a system depends in part on user awareness and comfort with that system. For a system that has changed, in some ways drastically, five times in the past 15 years, perhaps one of the best ways to improve efficiency would simply be to let it be.

Perhaps changes could be made to the filtering system given in the implementation plan to make it more streamlined and efficient. For example, one or more of the filtering layers could be cut out, either by using some form of self-assessment by researchers or simply removing a layer of filtering. Likewise, changes in the abilities of executive subcommittees to given provisional opinions could, as I have suggested above, increase the efficiency of the system. However, these measures would not be without cost. Any move that streamlines the filtering system is also likely to make it less robust and efficacious at identifying those applications that do contain serious ethical issues.9 This is clearly demonstrated by the move from (in the consultation plan) giving a single NREA the power to decide that an application has no ethical concerns and approve it on the spot to (in the implementation plan) a requirement that an application go before an executive subcommittee of at least two NREAs, one of whom ought to be a lay member, before approval can be given. This change was warranted because of the significant ethical concerns about one person making the final decision, and although it makes the system more robust, it does introduce, as I have outlined above, significant inefficiencies. Until evidence is presented that clearly shows that the proposed changes will increase efficiency, rather than bureaucracy, they should be rejected.