Article Text

Download PDFPDF

Process of risk assessment by research ethics committees: foundations, shortcomings and open questions
  1. Pranab Rudra,
  2. Christian Lenk
  1. Institute of the History, Philosophy and Ethics of Medicine, Ulm University, Ulm, Baden-Württemberg, Germany
  1. Correspondence to Pranab Rudra, Institute of the History, Philosophy and Ethics of Medicine, Ulm University, Ulm, Baden-Württemberg 89073, Germany; rudrapranab{at}gmail.com

Abstract

Risks and burdens in the study participation, as well as an adequate risk-benefit balance, are key concepts for the evaluation of clinical studies by research ethics committees (RECs). An adequate assessment and continuous monitoring to ensure compliance of risks and burdens in clinical trials have long been described as a central task in research ethics. However, there is currently no uniform and solid theoretical approach to risk assessment by RECs. Regulatory standards of research ethics such as the Declaration of Helsinki provide only minimal guidance on how risk decisions are considered. Due to discrepancies in the existing literature and guidance documents, adequate risk assessment by RECs remains to be elusive. In this article, we address current definitions of risk and present our own concept of aggregate risk definition. Moreover, we highlight the concept of benefit, the standard of reasonableness with respect to ethics literature and different approaches of risk-benefit assessment. In order to present a comprehensive theoretical approach of risk assessment by RECs, further understanding of the definitions of risk may improve adequate decision-making tasks by RECs. To improve the process of risk assessment by RECs, a dynamic framework will be illustrated, showing step-by-step risk assessment functions. This approach may be a promising tool to ensure adequacy in risk assessment by RECs.

  • research ethics
  • clinical ethics
  • clinical trials
  • ethics committees/consultation
  • technology/risk assessment

Statistics from Altmetric.com

Introduction

The concept of risk within the clinical research community is considered to be a central element and is frequently mentioned in scientific publications, guidelines and regulations.1–8 Over the last two decades, increased globalisation of clinical research paved its path to the increased expansion of clinical trials.9 Although regulatory documents for every clinical research addressing benefit and harm comparison are helpful, there is no uniform methodology inherent on how research ethics committees (RECs) should use the heterogeneous information on risks and burdens gathered from clinical studies. Lacking, however, is a comprehensive and systematic framework for risk-benefit evaluations based on the relevant literature and on guidance documents. As a result, the role of RECs in risk evaluation and monitoring is not clearly understood, particularly in terms of the ongoing clinical trial. This issue is highlighted by Rid and Wendler: “The regulations seem to leave those charged with assessing the risks and potential benefits of medical research with only their own intuitions to guide them. However, while normative judgment is a crucial part of ensuring that medical research is ethical, these assessments should be based on a systematic evaluation of the risks and benefits posed by individual research studies, not based on mere intuition regarding which procedures and studies are, and are not, excessively risky”.2 This critical description of the framework of risk assessment by RECs is supported by a number of further publications.5 10–12 On the one hand, research ethics looks back at a long and fruitful discussion concerning the criteria of risk assessment. On the other hand, there are still uncertainties on how to use these criteria in the field of clinical trials in a regular and systematic manner in order achieve adequate risk monitoring and patient protection.

The importance of a rigid risk assessment and monitoring in clinical trials was dramatically demonstrated by a Dutch study in pregnant women that had to be terminated after fatal results during summer 2018. Due to a decision of the Data Safety and Monitoring Board of the Dutch Strider Trial, the risk-benefit ratio of the trial was regarded no longer acceptable. The data revealed that 17 infants born to women given the drug sildenafil (Viagra) had lung problems, and 11 of those infants died, in comparison to 3 infants with lung problems in the placebo group, where no infants died.13 The study’s goal was to find an effective treatment against fetal growth restriction, where the published study design of the drug sildenafil proved to be effective and successful in a number of animal and preclinical studies.14 In case an ongoing trial fails to show a significant beneficial effect or shows a signal of harm, RECs may decide to stop the trial and allow detailed review and validation of the findings before any further exposure of the given drug occurs.15 The dramatic chain of events that took place in the Dutch study stands as an alarming example of possible weaknesses of monitoring and constant risk-benefit assessment in clinical trials. Reports of the deaths of research subjects raise serious concerns regarding the system and processes by which the participant’s safety is monitored.

This article considers three especially contested areas of research risk. First, we will review the concept of risk and benefits used in the relevant literature with respect to clinical practice (see section ‘Understanding risks and benefits in clinical studies’). Second, we will review the term ‘reasonable risk’ using different evaluation procedures that RECs may rely on when weighing the risks and benefits of medical research (see section ‘Reasonable risks as a common standard for clinical research’). Finally, we illustrate a new approach for risk assessment in clinical trials, which can be applied to the different study phases with respect to different kinds of risks at different time points (see section ‘Process of risk assessment’). We argue that such a comprehensive model may assist RECs and physicians to improve the monitoring of clinical trials as well as patient safety. In this article, we do not focus exclusively on pharmacological studies, but want to present concepts usable for all kinds of clinical studies.

Understanding risks and benefits in clinical studies

Regarding the international regulation and monitoring of clinical research, we find a well-defined system for the classification of pharmaceutical risks and medical devices, but no comparable provisions in other fields (eg, other interventional studies in medicine, surgery or psychology). This leaves out subjective and emotional burdens as well as additional physical or social risks for patients and study participants. At present, it is an open question, how the pharmacological understanding of risk can be complemented by risks and burdens of additional research procedures such as performing biopsies, spinal taps or experimental surgical interventions.

A systematic literature research review3 5 16–19 reflects a widely endorsed definition of risk—a product formula for possible harm that may occur, expressed as chance (probability), and severity (magnitude) of the envisioned harm, denoting to the formula

Embedded Image

In the economic sciences, it is widely accepted that the second factor, ‘severity of damage’, can be adequately described as the amount of financial damage (eg, the damage of a house, the loss of a car or the compensation for an injury). Evidently, such a concept of risk is in the end not very well suited for medicine and research ethics. As King and Churchill explain, ‘[b]oth potential benefits and risks of harm should always be examined and characterized, to the extent possible, according to their nature, magnitude (size and duration), and likelihood’.20

Based on this definition, we propose an extended concept of risk and benefit and a comprehensive understanding of risk which does not only cover pharmacological risks and burdens in clinical trials. We define the extended clinical risk as the product of the frequency, the kind of harm and the magnitude (size and duration) of the expected side effect (Box 1, (2)). For example, it could be known that a drug causes in the case of 1%–10% of the treated patients impaired vision to a severe degree, but that this is a transient condition. Such a description would be adequate from a medical ethics’ perspective. The frequency of risk and benefit can be deduced from the former experience with a therapeutic intervention. In the case of common medical interventions, the rate of side effects is often published in medical guidelines.

Box 1

Definitions of risk, types of risk and types of benefit

  1. Economic concept of risk: risk=frequency×severity of damage

    • Example: risk of dying in a car crash=10 deaths p.a./100 000 vehicles=0.0001%

    • ­

  2. Extended concept of risk and benefit (King and Churchill 2008): risk=frequency×[(kind of harm+magnitude (size+duration)] benefit=frequency×[kind of benefit+magnitude (size+duration)]

    • Examples:risk=1%–10% of treated patients×(impaired vision+severe+transient) benefit=10% of treated patients×(6 months higher life expectancy+median quality of life)

    • ­

  3. Comprehensive understanding of risk: The overall risk and burden of study participation consists in the aggregate of partial risks and burdens (drug risk, additional physical risks/burdens, social risk, psychological risk/burden)

Additionally, to develop a comprehensive understanding of burden and risk in a clinical study, we come to the definition of comprehensive risk in Box 1, (3). The overall risk and burden in a clinical study is then the sum of partial risks and burden in four different areas (figure 1): first, the drug risk, which contains adverse events and side effects. Second, there are the burdens of study measurements like additional X-rays or spinal taps. Third, genetic examinations can also include some social risks, and the same is true for psychiatric or psychological examinations which strongly intrude into the privacy sphere of the participants. Finally, there exist psychological risks and burdens, for example, when psychiatric patients are questioned about sexual abuse in their childhood.

Figure 1

Different types of risk in clinical studies.

In general, this is contrasted with three forms of benefit (figure 2): first, the direct clinical benefit, that is, when the patient directly profits from a new therapy in a study. Second, the group benefit, that is, when the patient herself or himself does not directly profit from the intervention, but future patients having the same disease, and third the scientific benefit, that is, to provide generalisable knowledge which is normally produced in all studies of basic research.20 These three forms of benefit are also incorporated in the Declaration of Helsinki and are connected to the acceptability of risk, that is, larger forms of risk are only acceptable when study participants have the chance to realise a direct clinic benefit. When the study participation bears only the chance for a group benefit, only minor risk is seen as acceptable (Art. 16–18).4 From our point of view, with these definitions and descriptions of benefit and risk, research ethics has the necessary concepts and terminology to carry out an adequate benefit-risk assessment, over and above the reductionist understanding of pharmacological risk alone.

Figure 2

Different types of benefit in clinical studies.

Reasonable risks as a common standard for clinical research

International conventions and guidelines, as well as national law, usually demand an ‘adequate’ risk assessment and a ‘weighing’ of risks and benefits for the individual participant. From the participant’s point of view, it has to be determined, that in her or his situation, the probability and amount of harm in a clinical trial is acceptable or appropriate and does not exceed defined ethical, legal and medical thresholds (eg, pain, fear or toxicity).21 The amount of risk for a special intervention (ie, the application of a new drug or an innovative surgical intervention) is often difficult to determine even for medical experts and has to be concluded—among other sources—from detailed statistical analysis and the available literature. A clinical trial has the aim to show the superiority or at least the non-inferiority between two or more different and prima facie equivalent medical interventions (the so-called clinical equipoise). When dealing with patient populations, risks are compared with those of the existing standard of care, which may itself involve a great deal of risk and the level of risk tolerance may therefore be quite high; when no such effective treatment or procedure exists, the benefits and risks of an innovative treatment approach are often compared with placebo. In this context, a comparable problem exists in the case of placebo use in a clinical trial when study participants are at risk of disease progression due to the placement in the placebo group.22 23 However, there are currently no clearly defined criteria and procedures on how this should be organised. Therefore, the implementation of risk evaluation for RECs still remains a challenge. In the international research ethics literature, there are especially three approaches that discuss possible and adequate forms of risk assessment:

  1. Weijer and Miller wanted to make RECs’ decisions more understandable and objective, and therefore suggested component analysis in the evaluation of clinical research. The component analysis divides the individual research interventions included in a given study into two groups: therapeutic and non-therapeutic. The approach involves an independent risk assessment of each research intervention or procedure involved in a study (eg, drawing blood, performing CT scan, taking liver biopsy).24 25 According to Weijer, therapeutic procedures are justified in terms of the clinical benefits of the participants, that is, clinical equipoise must be met. In contrast, non-therapeutic procedures are regarded as justified when the procedure in the study includes knowledge gain (ie, administered solely for the purpose of answering scientific questions and the potential to advance society’s interests) provided that the risk is acceptable and does not or only slightly exceed risks of daily life.

  2. In response to Weijer’s component analysis, Wendler and Miller developed the ‘net risks test’ that assesses the risk-benefit profile of each intervention by comparing its risks with the potential clinical benefits and then comparing them with existing alternatives.26

  3. The direct benefit assessment is used in risk assessment by RECs, which is derived from the US guidance documents for medical research on humans.27 This assessment instrument offers a possibility to avoid the therapeutic and non-therapeutic distinction and takes the direct clinical benefits of a study for the participants as the deciding criterion for ethical considerations.

Concerning the three shortly described approaches, it catches one’s eye, that they do not give a clear and substantial definition of ‘reasonable risk’. The criteria are always relative or proportional to possible clinical or scientific benefits. Therefore, these instruments do not give a direct answer of what is meant by a ‘reasonable’, ‘justified’ or ‘balanced’ relation of risk and benefit. Rather, they can be regarded as decision aids for experts with longstanding experience in the field. It remains an open question, whether such approaches can ensure a homogeneous level of regulation based on similar decisions from different RECs and in the assessment of different clinical trials. Additionally, especially the Belmont Report and the different versions of the Declaration of Helsinki take up a stance regarding an adequate risk assessment. The different aspects of risk assessment in clinical research studies from the Declaration of Helsinki are compiled in table 1.

Table 1

Criteria for risk assessment in clinical studies according to the Declaration of Helsinki

A closer look at these criteria of risk assessment in table 1 reveals that there are in general two different kinds of ‘reasonable risk’: first, we find a relative or proportional understanding of risk in the majority of passages in the cited documents. This suggests that it is asked or stated that an existing risk might be acceptable in comparison to the clinical benefit for the concerned patient, the benefit for society or in comparison with existing alternatives (table 1, (1–3)). We can call this a proportional understanding of risk, because it results in the point of view that the existing risk must be proportional to an aspired good or benefit. For example, when the patient suffers from only a minor disturbance of health and well-being, it would be irrational to accept major risks and burdens for the treatment of this condition. However, in the case of severe or untreatable diseases, it is rational to accept larger risks and burdens in the treatment of the disease. Such an understanding of risk seems to be adequate for medical research, but maybe produces further questions in some situations, for example, when patients suffer from a potentially lethal disease with no existing treatment options. Where is then the limit in such a relative understanding of risk and what is acceptable for such patients in the research for new therapies? The proportional perception of risk comes here to the conclusion, that also the sacrifice of important patient interests could be acceptable in such cases. From the ethical point of view, such a result is a dilemma of benefit-risk assessment (for further problems with the ‘weighing’ of risks and benefits, see King and Churchill20). Given that such patients are an especially vulnerable group in research ethics, alternative approaches would be necessary to solve this dilemma.

Second, one can notice an absolute or definite understanding of ‘reasonable risk’ in the quotations number 4 and 5 from the Declaration of Helsinki (table 1, (4 and 5)). The first quotation number 4 refers to vulnerable patients who have no direct clinical benefit when participating in a research project. Art. 28 of the Declaration of Helsinki claims that in such cases only ‘minimal risk and minimal burden’ is acceptable. This indicates an absolute understanding of risk. Quotation number 5 refers to placebo studies and studies where standard treatment is withheld. According to the Declaration ‘additional risks of serious or irreversible harm’ must be excluded in such cases, which indicates an absolute understanding of risk as well. Apart from these two articles, the relative or proportional understanding of risk is seen as adequate or reasonable. The term absolute represents a predetermined or fixed risk level to ensure clinical safety for the participants. For example, exposing a group of children or individuals to the burdens of study participation (ie, an additional blood sampling) would not violate the requirements of patient protection when the study has the potential to promote the health of this patient group with minimal risk and burden. In such cases, Kopelman proposes an upper limit for risk, which ‘(a) represents no more than a minimal risk for children with conditions who are enrolled in studies and (b) be no more than a minor increase over minimal risk for healthy children were they enrolled’.28 This constitutes an absolute standard of risk which is applied to children and other vulnerable groups.

Process of risk assessment

It remains unclear whether the models proposed in the literature are already used by RECs when making decisions in interdisciplinary consultations. The common risk-benefit approaches, namely component analysis, net risk test and direct benefit assessment do not give a complete account of risk-benefit assessment of medical research.2 18 24–26 Most importantly, none of them accounts for risk identification on a continuous basis throughout the clinical trial. An ideal risk assessment process should start at the protocol design so mitigation can be built into the protocol and monitoring documents.

In the following, we describe the different steps in the assessment procedure of RECs, as it is portrayed in figure 3. So far, the internal process of risk assessment has not been adequately described in relation to the different components of clinical trials (initiation phase, start of trial, conduct, reporting, end of trial). Our description refers to the processes as they are currently practised by RECs and other responsible authorities (eg, the respective national body responsible for the market approval of new drugs, ie, the Food and Drug Administration in the USA, the Medicines and Healthcare products Regulatory Agency in the UK and the Federal Institute for Drugs and Medical Devices in Germany). Additionally, the description includes the proposal of the extended concept of risk and benefit (see section ‘Understanding risks and benefits in clinical studies’) and a systematic order of REC risk assessment at different points in time in the course of the clinical trial. Therefore, the description represents a normative or idealised risk assessment procedure for RECs.

Figure 3

The process of risk assessment by research ethics committees. DSURs, Development Safety Update Report; EMA, European Medicines Agency; SAEs, Serious Adverse Events; SUSARs, Suspected Unexpected Serious Adverse Reaction; t1, Evaluation of Risk before Trial Start; t2-t3, Reporting in the Course of the Trial; t4, Foreseen End of Trial.

The process begins with the initiation phase of the clinical trial (figure 3, (1)) and the submission of the study protocol together with clinical and preclinical data and further documents to the official authorities and to the responsible RECs. In the assessment of the clinical trial by the REC, risks are identified and compared (figure 3, (2)) with standard treatment (insofar there is an established standard treatment). To come to a comprehensive understanding of risk, as described above, the different types of risk (figure 1) have to be examined. It follows (figure 3, (3)) the proof of the risk classification, that is, what is to date known regarding the frequency of risk. The frequency at which the risk occurs is an important factor in classifying risk according to its weight. The European Medicines Agency guidelines classify drug risks according to the likelihood and severity of adverse events (ranging from very common to very rare). The risk-benefit assessment demands knowledge and a proper understanding of setting priorities in terms of risk identification and classification (figure 3, (4)). Considerations such as what can go wrong or the probability of a negative outcome occurring or trial subject’s safety, well-being and rights are all part of the assessment process. Different criteria of risk-benefit assessment profile of clinical trials, drug trials, first-in-human use, children, vulnerable groups and individuals according to Declaration of Helsinki are taken into account.29

In summary, the concerned REC and the respective national body assess the information for a well-founded risk-benefit balance and take possible steps to reduce the risk to an acceptable level (figure 3, (5)). Once the right risk-benefit balance has been determined, the authorities finally decide on whether the clinical trial is allowed to start (figure 3, (5 and 6)). The regular reports of expected and unexpected adverse events (figure 3, (7)) during the trials are used to classify the risks appropriately (ranging from mild risk to the death of a participant). According to the international drug regulations, adverse events in drug trials have to be regularly communicated to the RECs and the responsible authorities. By systematic documentation of the adverse events, RECs gain insight into the existing risks and burdens in a clinical trial. This is complemented by annual reports of the researchers regarding the safety of the trial. In the course of the continued risk observation and assessment, the question arises, whether the original risk-benefit assessment at the trial start (t1) was accurate in comparison with the later on reported types and level of risk (t2–t3) (figure 3, (8)). At the present point of time, it is not defined in the Declaration of Helsinki or other documents (at least in the European Union) how much effort RECs should invest into this task of risk monitoring in clinical trials. However, as the above-mentioned example of the Strider Trial shows, this task is of extreme importance. In case of a fundamental deviation from the previously expected risk level and an excess of severe adverse events, the responsible authorities have the possibility of a termination of the clinical trial (figure 3, (9)). Otherwise, the trial reaches its foreseen end as it was defined in the study protocol (figure 3, (10)).

From our point of view, this systematic approach to risk assessment in clinical trials can facilitate better and more informed decision-making and make an important contribution to a steady and continuous improvement of patient safety. Although the approach presented here is an evolving framework, the concept is expected to benefit the RECs how and to what degree of transparency the information is shared. One should also consider that numerous values ground risk judgements and that a more comprehensive approach towards risk assessment will be more sensitive to the different needs and burdens of study participants. Additionally, a more systematic approach can strengthen the coherence of REC decisions and therefore the trust between researchers and the institution of the REC as such. Such an approach could also identify remaining elements of uncertainty in the concept of a research project, where more assertive measures from the REC could be needed. It is, however, in the responsibility of all involved parties to best contribute to the delivery of an effective risk assessment in research studies.

Conclusion

The task of risk-benefit assessment by RECs is a complex endeavour. To better understand this assessment, we took initiative to explore the literature definition and reasonableness of risk in clinical studies. We summarised how authors from ethics literature perceive risk and benefit definitions. A key part of the assessment process is to understand what reasonable risk means. Generally, with the term ‘reasonable risk’ we understand a proportionality of risks and benefits while for vulnerable groups there should only be minimal risks with the exclusion of any irreversible risks. In any case, a systematic approach where the definition and reasonableness of risk are uniform and its heterogeneity addressed promises to reduce unreliable decisions made by RECs. A model system to articulate the process of risk has been proposed here which provides a compelling dynamic foundation for a framework for assessing risk in clinical studies. The model may prove effective for continuous risk assessment in different clinical studies while maintaining a ‘homogenous approach’ by the RECs. The appropriate conduct of the risk assessment process suggested here may ensure step-by-step compliance with regulations and can prevent—in the given limits of risk assessment—major unjustified risks to participants in the study. When following the systematic framework, the RECs will be able to indicate any amendments related to risks and can, therefore, respond to them accordingly, timely, accurately and effectively.

To sum up, by looking at the theoretical aspects of risk in all dimensions, in particular, definition and reasonableness, we can better identify the problems of discrepancies and strategise consolidated solutions of risk assessment. Following the described process of risk assessment would be fruitful for RECs to conflate ethical considerations and risk-benefit tasks, thereby improving the functioning and the ethical consultation of RECs. As described above, the aim of a responsible risk-benefit assessment cannot be reached by only punctual and intuitive activities. Instead, a steady and systematic approach by RECs is needed, particularly in improving ongoing or intermediate clinical trials and patient safety. Next to that, our description of the domain of risk assessment in clinical trials shows that there should be more efforts for the establishment of absolute risk thresholds, for example, in the case of specific diseases (ie, cancer research, dementia research). This could lead to a more homogeneous risk assessment between different RECs, also in international comparison. Such an approach may be extremely important especially in the case of patients with severe and incurable diseases, where up to now a proportionate approach for risk assessment is not possible.

Acknowledgments

The authors would like to thank the two anonymous reviewers for their helpful comments, which have been incorporated into this article.

References

Footnotes

  • Contributors All authors contributed significantly to this article.

  • Funding This paper was written with support from the Medical Faculty of Ulm University.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Other content recommended for you