The current system of ethical review for medical research in the United Kingdom is changing from the current system involving large committees of 7–18 members reviewing every individual application to a system involving pre-review by small sub-committees of National Research Ethics Officers (NREAs), who have a remit to approve studies if they believe there are no material ethical issues imposed by the research. The reliability of this new system depends on the reliability of the NREAs and in particular the ability of small groups to effectively identify and appropriately assess the seriousness of all the material ethical issues that may be posed by an application. There is anecdotal evidence to suggest that many individual research ethics committee members have had experience of believing that a study presents no material ethical concerns, then on reaching the committee and discussing the application they realise that the committee feels it does present significant ethical concerns. If this is the case then this casts doubt on the reliability of NREAs or small groups to effectively identify ethically problematic research and appropriately respond to this to protect research participants. In this paper we describe a small questionnaire based piece of research carried out to assess how common this and other relevant experiences are.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Although almost everybody in the field of medical science and certainly those who are involved in research will have to deal with research ethics committees (RECs), it could be seen as a general consensus among the scientific community that they are overly rigorous and non-specific in that they have a “one size fits all” approach. Considerable criticism has, through the years, been made of National Health Service (NHS) Research Ethics Committees system.1–8 In response to these criticisms the NHS Research Ethics system has undergone significant change and development.i In 2005 the Warner Report was published recommending further changes to the NHS Research Ethics System.10 In response to this the Central Office for Research Ethics Committees (COREC) formulated a draft proposal of changes to meet with the suggestions of the Warner Report and carried out a consultation of interested parties.11 The net result was the Implementation Plan.12 As it stands all research and procedures within the NHS must go through a full REC evaluation. These committees can consist of anything from 7–18 members.13 There are approximately 120 NHS RECs throughout the UK and an application could potentially be submitted to any of them.11 In a full review, all aspects of the proposal are considered. In the new system, based on the principle of proportional review, a pre-committee review will be carried out by a small group of National Research Ethics Advisors (NREAs) to determine whether applications pose significant material ethical concerns. If they do not then they will be approved by the NREAs without being seen by a full ethics committee. 12 ii
This proposed new system has met with considerable critical attention, both in regards to the Warner Report, the plan put forward in the consultation and the revised plan laid out in the Implementation Plan—suggesting that the new system will not be substantially more efficient, that the notion of “no material ethical concerns” is too vague and that it will fail to provide adequate protection to research participants.14–21 It is this last claim that this piece of research focuses on. In particular the new system depends on the reliability of NREAs in identifying all the ethical issues posed by applications, and their accuracy at assessing the significance of these ethical issues.
A commonly expressed opinion, which applies to this recommendation, and many other instances in the implementation document, is that it is a frequent experience, as a lead or back-up reviewer, to believe that their submission is sound and ethical, only to discover during the ensuing committee discussion, that your fellow members think nothing of the sort. How then could an individual, or even two or three people, be confident that the correct and safe decision had been reached? 15
However, as the above quote from the Association of Research Ethics Committees response to the consultation shows, there is at least anecdotal evidence which gives us reason to doubt that individuals are either reliable detectors of all ethical issues in a particular research application, or are reliable in making the right judgement concerning the seriousness of these ethical issues. The purpose of this research is to help establish the incidence of this sort of experience by the use of a short questionnaire. If the experience is even minimally present for a proportion of ethics committee members then this gives us a reason to doubt that the reformed NHS research ethics system will adequately protect research participants.
The World Medical Association states in its Declaration of Helsinki:
The design and performance of each experimental procedure involving human subjects should be clearly formulated in an experimental protocol. This protocol should be submitted for consideration, comment, guidance, and where appropriate, approval to a specially appointed ethical review committee, which must be independent of the investigator, the sponsor or any other kind of undue influence. 22
As such, RECs can be viewed as the gold standard for ethics review, and an alternative system should be shown either to be as reliable or have some other significant advantage before it is adopted.
While this research is focused on the changes to the current NHS research ethics system in the UK, it does have further implications both for research ethics review in UK universities, and abroad.
In the case of research ethics review in UK universities many differing structures of ethical review are present.23 Some universities rely on one central ethics committee to review all or some of its research. Others delegate ethical review responsibility to a faculty, school or department level, each operating their own ethics committee. Some of these committees can be quite small (we know of one committee where regularly only two members attend committee meetings and make decisions). In one report published in 2004 it was found that;
One third (22/63) of the Universities had one central REC only. Just under two thirds (59%) had more than one—in some cases this consisted of a combination of Faculty/School/Department. 24
It is often proposed that this is a better system for the university environment, as it is thought that most university based research projects do not generally raise any serious social or ethical issues.9 While there are no legal requirements at this stage for university based research in the UK to seek ethics approval (excluding some research involving human tissue, mental incapacity, clinical trials and of course medical research involving the NHS) many funders—for example, the Medical Research Council and the Economic and Social Research Council—require it as do many academic journals.25 However this research may also cast doubt on the reliability of these small committees in UK universities, especially if the committees primarily contain members from only one discipline. In particular this points out significant concerns for common methods of expediting seemingly unproblematic research such as taking Chairs action or subcommittee review. Insofar as this research indicates that individuals or small groups are unreliable at identifying ethical issues and assessing their significance, it indicates that these forms of ethical scrutiny may be inappropriate.
There will be a variation in opinion between individual members of the RECs surveyed on the ethical implications of certain projects and members will have experienced a change in opinion of the ethical implications of a project through the course of an REC meeting.
A questionnaire was designed and administered to those involved in the review of research ethics at the University of Ulster, Northern Ireland. The University of Ulster operates a devolved research ethics system and has 14 research ethics filter committees ranging from 5–20 members per committee which can approve low risk, ethically non-contentious projects and a central committee which considers applications which raise very significant ethical issues.26
A two-page questionnaire was constructed consisting of nine questions. Of these nine questions, six required a written response and three were Likert scales.iii Question 3 asked if the respondent played any special role within their committee, this, along with a question about the length of service allowed for the degree of experience of the members questioned to be ascertained. Question 5 asked whether they had ever had the experience of believing an application was acceptable, until they reached the ethics committee meeting and realised from discussions there that the committee felt there were serious ethical issues with the application; question 6 asked how often they had this experience. In question 7, participants were asked how much variance they would consider there to be between their opinion and that of others within their committee. This was aimed to allow us to gauge the degree of variation in the opinions of particular projects that committee members experienced. This could then be used to further test our hypothesis. Question 8 was also a multiple choice question asking how often the participants felt that their opinion on a project had changed through the course of an REC meeting, with five possible responses. This question was necessary in order to show that through the course of committee meetings the opinion of certain members can and do change. It also indicates the potential for false positives, where the new system might wrongly identify an innocuous application as contentious reducing the likelihood of the hoped for increased efficiency.21
Ethical approval was applied for from the University of Ulster Biomedical Sciences Ethics Filter Committee and was granted. The names of the Chairs of the committees currently in operation within the University of Ulster were obtained from Research Governance and their email addresses were obtained from the university website. Each committee Chair was emailed to let them know about the questionnaire and that their assistance would be appreciated. Questionnaires were sent to each committee Chair through the internal university mail along with a letter to them asking them to distribute the questionnaires to the members of their committee at their next meeting.
There are approximately 50–70 members of research ethics committees at the University of Ulster, of whom 23 returned the questionnaire. It is difficult to be more determinate about how many people are involved because several people sit on more than one committee, and because when you are dealing with 15 different committees there will be a certain level of turnover of members at any given time. So the response rate was between 46–32%.
Figure 3 shows, oddly, that despite 50% indicating in the questionnaire that they had had the experience (in response to question 6) of believing an application was acceptable until they reached the ethics committee meeting, but realised from discussions there that the committee felt there were serious ethical issues with the application (question 5), yet 54% of those questioned said that they had never had that experience.
Due to the reasonably small scale nature of this study and the focus on university REC members as opposed to the Health and Personal Social Services (HPSS) or NHS REC members, it may well be questioned how generalisable the results are. Keeping this in mind, the results nonetheless suggest that this may pose a problem for systems of proportional review. This is particularly the case in regards to the suggested amendments to the NHS research ethics system, because there are reasons to believe that members of university RECs may be more reliable at identifying the specific ethical issues posed by the research applications that they review, than those involved in HPSS or NHS RECs. Firstly, while we don’t want to make the fallacious claim that all medical research involves more significant risks or complexity than the social science research examined by many university ethics committees such as the ones we surveyed, it is fair to say that medical research applications often present significantly more risks and complexity than much university based research. Secondly, the forms involved in the typical university review process are often far less detailed and complex than the NHS REC form. To give an example, the application form for research at the University of Ulster is only six pages long (although supporting information such as a short CV, a research protocol, participant information sheet and consent form is also required).26 The NHS REC form in contrast is 35 pages long and is likewise supplemented by supporting information (although the form may shorten depending on what options are chosen). The more information there is to process the more difficult that processing becomes. Thirdly, university committees are often based in subject areas and so have considerable expertise in the ethical issues presented within the typical research methodologies and areas that most of their applications focus on. In contrast, a wide range of methodologies and subject areas are routinely reviewed by HPSS & NHS RECs so that it is unlikely that there will always be an expert in the specific area on the committee. This will be even more pronounced with the new pre-review system given that there may only be two people who are deciding whether an application presents significant ethical issues. They are very unlikely to be experts in the specific or even general area of the research project.17
The average length of time the members surveyed had served on an ethics committee was 1–2 years with 43% falling into this bracket. 18% of those questioned had surveyed for 2–3 years. These figures are illustrated in fig 1. This would suggest that 61% of those who serve on the committees of the University of Ulster have over 1 year of experience; this means that the majority of those on the filter committees have relevant experience in assessing research projects ethical implications. This indicates that their experience is likely to be sufficient to successfully identify significant ethical issues with applications and be relevantly similar to the experiences of those on NHS & HPSS committees.iv
Figure 2 illustrates the share of special roles within the 16 filter committees in the University of Ulster. It shows that 23% of those questioned were or had been the Chair of their filter committee, 13.5% were Vice Chairs, 9% were lay members and in the questionnaires these lay members generally stated that they felt their position helped them see the value of the work of the ethics committees from an outsiders perspective as opposed to a researchers and through this they were able to be more objective to the issues from the perspective of the human participants in the proposed studies. Student representatives numbered 4.5%, and 50% of those questioned felt that they did not play any special role within the research ethics filter committees.
Of those surveyed, 50% said that they had had the experience of thinking a proposal was unproblematic and then realised on review that the committee felt the application presented serious ethical issues to a greater or lesser degree. There are two possible explanations of this phenomenon which have differing implications for how serious a problem this is for the proposed new research ethics system in the UK. The first explanation is simply that individuals failed to identify ethical issues, that were picked up either by other committee members prior to the meeting, or during the discussion at the meeting issues were discovered. The second explanation is that while they identified an ethical issue they either underestimated the seriousness of this issue, or were over-confident in the ability of measures detailed within the research application to deal with that particular ethical issue.v It is likely that both of these explanations occur, for some members they will have experienced the first, for others the second, and for others both.,vi We will now discuss the significance of each of these explanations for the proposed changes to the UK NHS research ethics system.
The first explanation is clearly problematic for the proposed new system. It seems to shows the importance of the REC meetings in order to identify the ethical issues raised by research projects.27 While one member may not see a problem with a proposal another may be able to identify ethical issues and discrepancies for the attention of the entire committee. This suggests that the changes to the NHS research ethics system may put some research participants at risk, by failing to detect all the ethical problems in an application.vii However, if the second explanation is not also problematic then the data may not suggest a great concern for the proposed changes, if the second explanation is far more prevalent than the first.
It is more difficult to assess how much of a problem the second explanation causes for the proposed new system. It will be a significant problem, if the new system requires NREA to make assessments of the seriousness of ethical issues with applications and/or how well measures undertaken by the researchers resolve those ethical issues. Whether NREA will be doing this largely depends on how we interpret the ambiguous phrase “no material ethical issues”.20 If it is interpreted to mean that whenever an ethical issue of any sort or magnitude, resolved or not is identified by the NREAs the application is sent on to full committee then the adequacy of individuals in assessing the severity of ethical issues, and the appropriateness of responses to these would be irrelevant. This is because the NREA would in effect only be identifying ethical issues, not making any judgments regarding them. If in contrast, we view the role of an NREA to be making an all things considered judgment about whether there are significant ethical issues left unresolved in the application then clearly the adequacy of individuals in making these decisions is of the utmost concern.
It is a matter of some debate as to how this phrase ought to be interpreted and we do not hope to settle this debate here. However it seems likely that neither extreme is the appropriate interpretation. The first explanation is unlikely to be correct if only because it is hard to imagine a research project that doesn’t raise some ethical issues, regardless of how well they are addressed by the researcher. As such it would seem that on this interpretation almost all applications would not be suitable for approval by NREA, removing any gains in efficiency that might be hoped for in this new system.21 Furthermore the examples given in appendix F of the Consultation Document of applications that may be appropriate to be approved by NREA seem to contain ethical issues (presumably adequately dealt with, though it is hard to tell from the brief descriptions).11 viii The “all things considered” interpretation is also likely to be wrong because as the Consultation Document and the Implementation Plan make clear, certain types of research will automatically receive review by a full committee.11 12 ix As such it seems the NREA will be making a “some things considered” judgement. While some studies will be identified as needing to go to full committee reasonably easily, in other cases some judgement will be required as to the severity of the ethical issues involved, and the appropriateness of how the researchers propose to deal with them. While not as troublesome as if the NREA were required to make an all things considered decision the second explanation of the data is thus still problematic for the proposed system, sometimes the judgements of individuals will not align with those that would have been made by a full REC. It might be objected that the NREA will be very experienced and so will be less likely to miss problems or misjudge their severity. However fig 5 showed that even the more experienced members on the committees—that is, those who have served as a chair and/or have at least three years experience—still sometimes have this experience. 60% of these most experienced members have had the experience of thinking that a proposal was unproblematic only to discover on meeting with their colleagues that they felt it had serious ethical issues. Of course this may simply represent how long they have been involved in ethics review, but it still suggests that although the NREA can be highly experienced there may still be problems in identifying and reacting appropriately to some ethical issues. This indicates, worryingly, that some applications with serious ethical issues could be wrongly identified as non-problematic if they are only subjected to review by a few people. In the new system proposed for NHS research in the UK this could lead to applications with serious ethical issues or inadequate precautions being approved to proceed.
As shown in fig 4 all of those surveyed had the experience at some time of their opinion being changed during the course of an ethics committee meeting. The majority (77%) of those questioned said that this occurred sometimes while 4.5% had the experience often and 18.5% rarely. This again underwrites the importance of the REC meeting. By meeting and comparing their opinions of the ethical implications of research projects the committee as a whole are able to minimise the occurrence of missing a shortcoming in the ethics of a proposed project. This is important because it mitigates both against missing serious ethical issues, but also against wrongly identifying applications as having serious ethical issues when there aren’t any in play.
We were surprised at how much variance there was between the views of individuals and what they perceived to be the views of their fellow committee members. We anticipated a considerably more homogenous result, with many members feeling that there was little or no difference between their views and those of other committee members. Especially since those who responded to the questionnaire mainly came from committees comprised, for the most part, of members of a single academic discipline. However, as fig 6 shows 59% of those questioned thought there was some variance between their opinion and those of other committee members. All of those surveyed had had the experience of a variance of opinion between them and their colleagues although 41% said the variance was little. This may present further problems for the intended changes to the research ethics system in terms of the wide range of views displayed. Making the reasonable assumption that part of this variance is explained by different committee members being sensitive to a variety of different concerns then this would seem to indicate that the meeting of the ethics committee allows for the consolidation of this variation and for a general consensus to be made in regards to the ethical issues in research projects. If the views of ethics committee members are as rich and diverse as this result seems to indicate it seems difficult to see how two or even four or five people could genuinely represent all of these diverse views in an appropriate fashion.17
The data shown illustrates the occurrence and frequency of variations between the opinions of members within the RECs of the University of Ulster. It has been argued that this data casts doubts on the ability and reliability of individuals or small groups to identify ethical issues, to recognise their significance and appropriately assess what the researchers intend to do to mitigate those issues. It also casts doubt on the ability of individuals or small groups to adequately represent and identify the wide range of possible ethical viewpoints that members of committees presently represent. The data gathered in this study would suggest that the proposed changes to the structure of ethics committees within the NHS may fail to protect the research participants to the current standard. This would suggest that if these measures ought to be put in place, it would only be because there were some other significant advantages in terms of, for example, the efficiency of the new system that outweighed its decreased reliability.21 However it may be legitimately questioned whether efficiency gains can ever justify a decreased level of protection for the participants in research.
We would like to thank the members of the University of Ulster Filter and Research Ethics Committees for participating in the research. We would particularly like to thank the Chairs of these committees for facilitating the research by distributing and collecting questionnaires on our behalf. We would like to thank the Faculty of Life and Health Science, University of Ulster for funding this project.
Competing interests: DH is the Alternate Vice Chair of HPSS Research Ethics Committee 1 in Northern Ireland. He also sits on the University of Ulster Research Ethics Committee, and the School of Biomedical Sciences Ethics Filter Committee.
Ethics approval: Ethical approval for this research project was granted by the University of Ulster Biomedical Sciences Ethics Filter Committee, in accordance with the University of Ulster Research Governance procedure.
↵i A brief history of these changes can be found in William-Jones B, Holm S. A university wide model for the ethical review of human subjects research and in Department of Health, Report of the Ad Hoc Advisory Group on the Operation of NHS Research Ethics Committees—The Warner Report.9 10
↵ii It is presently intended that there will be up to 8 NREAs and that they will be able to decide on whether an application contains no significant ethical concerns in subcommittees of at least 2 members.12
↵iii See the supplemental data for a copy of the questionnaire.
↵iv It is only “hopefully” relevantly similar since there is little data available on the experience level of members of HPSS & NHS RECs and more importantly at what level of experience REC members are most effective at identifying ethical issues. It is commonly thought that new members of RECs will be less effective at first, but as with marking essays newcomers may well judge harder than the rest of the committee. Likewise while we would typically think those who have considerable experience will be more effective members of RECs they may well have become jaded, or having seen some truly appalling applications lower their standards because at least this one isn’t as bad as that other one.
↵v We would like to thank an anonymous reviewer for the Journal of Medical Ethics for pointing out these two distinct explanations which we feel has added considerable strength to the overall argument of this paper.
↵vi Unfortunately the data we have does not indicate which explanation is more prevalent. We think it is likely to be the case that both explanations are correct in some cases. However as we will shortly argue both explanations cause concerns for the proposed changes.
↵vii We do not here intend to indicate that research ethics committees are by any means infallible in terms of identifying ethical issues, however we feel that this shows that they are likely to be more reliable than small groups.
↵viii One example includes questionnaires with patients which obviously raises questions of adequate informed consent being taken. Another involves the use of archival tissue without consent.
↵ix The Consultation Document suggests the following types of research would go to full review: “any clinical trial of an investigational medicinal product or clinical investigation of a medical device; any other clinical research; research involving vulnerable groups; studies where participants are to receive payments or incentives exceeding reimbursement of expenses, or investigators are to receive payments or incentives linked to recruitment of participants; and any study in which it is proposed not to obtain informed consent”(p16).11 The Implementation Plan is more circumspect only suggesting that studies requiring physical interventions will definitely automatically go to full committee review.12