FIRST FAILING AND FIRST LESSON: PUBLICATION BIAS

The first major failing was the failure of researchers who conducted the 1978 San Francisco study of hexamethonium to report similar adverse reactions.

The tendency of researchers to fail to publish disappointing results² or results which are unattractive to them or their sponsors' interests has been well documented.³ It is called publication bias. An important part of publication bias is the failure to report all relevant or potentially relevant results. As this case illustrates, researchers must report or make available all research results, even negative results, and all potentially relevant outcomes of that research. If researchers believe an adverse event is unrelated to the study, they should explain why they believe this rather than omitting the event from their report. Elimination of publication bias is essential to scientific progress, to the protection of participants in research and in maintaining (or perhaps restoring) public faith in the integrity of scientists and doctors. Ethics committees and journals should make it clear (in a written undertaking) that all adverse events—related or unrelated—to a project must be acknowledged, evaluated and further details made available to other researchers or the public if requested.

SECOND FAILING: FAILURE TO CONDUCT A SYSTEMATIC REVIEW

But there is a more important failing. The most disturbing feature of this case was that literature documenting the pulmonary toxicity of hexamethonium was available before the trial began.

In the study protocol submitted to the ethics committee, the researcher cited four reports involving 20 patients given inhaled hexamethonium without adverse event. However, Johns Hopkins investigators found a number of reports between 1953 and 1962, and a review article from 1972, which had not been referred to.⁴ Clark, et al found additional data on adverse events, even in studies published after 1980, on MEDLINE, Old Medline, and Cochrane Controlled Trials Register.⁵ They found at least seven reports, and an additional nine in the references of these studies. Five referred to pulmonary complications of hexamethonium in the title.⁵ Neither the researchers nor the ethics committee accessed that literature. Perhaps as a result of that error, a healthy woman died and many good research programmes were halted.

The failure of ethics committees to require an adequate review of the existing literature relevant to a research project is not new. In 1995, Iain Chalmers, Jennifer Blunt, and I highlighted some failings of ethics committees to do precisely that and urged committees to require a systematic review of the literature from researchers.⁶ Togias did not systematically review the literature. The ethics committee did not evaluate, or inadequately evaluated, his search strategy. In the State of Victoria, Australia, we have developed a common application form for researchers which nearly all major institutions use. That form requires documentation of the detailed search strategy employed by researchers to capture relevant literature (http://www.dhs.vic.gov.au/phd/ethics/). Ethics committees must require transparent and scientifically defensible systematic reviews of existing literature, and details of the search strategy employed to retrieve that literature.^5–7

TIME FOR A NEW SYSTEM OF ETHICS REVIEW?

In June, I reviewed the death of Jesse Gelsinger, a young man with a mild genetic disorder who died in a phase 1 gene therapy trial. I argued that one of the major errors that resulted in the death of Gelsinger was a failure of the ethics review process in research, and the failure of the ethics committee involved to adequately conceptualise what constitutes harm and/or a failure to give enough importance to reducing risk to potential participants. This led the committee to approve the participation of an adult with a mild form of a genetic disorder rather than a neonate with a severe form who would likely have died without gene therapy. The lesson from the Gelsinger case was that ethics review is not a matter of subjective consensus or finding an answer that is publicly acceptable. Proper ethics review requires a sound philosophical approach, and committees need an understanding of the ethical principles and concepts that underpin research ethics review.

A month later, another healthy volunteer has died. What can we learn from this second tragic death?

The failings in this case are not new. They are well documented. There were other failings.⁴ It is tempting to point the finger at the ethics committee. The ethics committee failed in several important regards. But can its members be blamed? How can we resurrect what Art Caplan describes as a “dead” system of ethics review?⁸

Ethics review is complex, demanding and time-consuming. It may involve interpretation of layers of complex legislation. Ethics committee members in the UK and Australia are generally unpaid and devote large amounts of personal time to conscientiously reviewing ethics applications. But as these two deaths show, evaluation of the conceptual and safety issues is a technical and specialised job. Ethics committees are simply not set up or resourced to adequately protect the interests of participants in research.

In Australia, the Australian Health Ethics Committee is a committee of the National Health and Medical Research Council, the major body funding medical research. The Australian Health Ethics Committee has adopted several strategies which address these problems:

1. Guidelines. It has formulated detailed guidelines in the form of the National Statement on Ethical Conduct in Research Involving Humans(“the National Statement”).⁹

2. National application form. It is developing a standard electronic application form which is based on this National Statement and includes detailed guidance to researchers on what is required.

3. Training. It is actively training members of ethics committees.

4. Co-option of expertise. The National Statement allows for the co-option of members with specialist expertise.

5. Resourcing. The National Statement requires institutions to adequately resource their ethics committee.

These are positive steps. But problems remain. There are over 200 ethics committees in Australia with more than 2000 members. It is difficult to ensure adequate training of ethics committees members on this scale.

Moreover, ethics committees are not set up to identify the risks that led to the deaths of Gelsinger and Roche. The National Statement requires a minimum membership of: a chairperson, at least two lay members, at least one member with knowledge of, and current experience in, the areas of research that are regularly considered . . .for example, health, medical, social, psychological, epidemiological, as appropriate at least one member with knowledge of, and current experience in, the professional care, counselling or treatment of people (for example, medical practitioner, clinical psychologist, social worker, nurse, as appropriate), at least one member who is a minister of religion, or a person who performs a similar role in a community such as an Aboriginal elder, and at least one member who is a lawyer.¹⁰

In 1998, at the time of revising the previous guidelines, I made a public submission arguing that the new guidelines should include a person with skills in systematic review of the literature and that the requirement to have a religious representative on the committee be replaced with the requirement to have an ethicist. In light of the death of Ellen Roche, some have argued that clinical pharmacologists and pharmacists would have had an important role to play in preventing Roche's death.⁴ But how can a lawyer or a priest be expected to decide whether inhalation of hexamethonium will be toxic? The composition of ethics committees is not set up for modern research ethics review.

Similar problems exist with harnessing external expertise. Experts are usually expected to give up their time unpaid and uncompensated. Selection of experts occurs in an uncontrolled and unevaluated way.

Concerns about institutional liability have encouraged institutional research ethics committees to evolve. But this is an inefficient system resulting in duplication of review of multicentre research with the consequences of inefficiency, wasted resources and inconsistent review.¹¹ The standard of ethical review of research has been roundly criticised in Australia^12,13 and internationally.^14–23 It is also a system which invites “cronyism” among researchers and the members of the ethics committee, some of whom are researchers themselves. It is time to ask whether institutional research ethics committees should be abandoned in favour of expert committees that cover many institutions—suprainstitutional specialist committees. Supraregional specialist ethics committees could specialise—for example, in genetic research, cancer clinical trials, dermatology, respiratory physiology, and each of the specialist areas of medical research. An example of such a committee is provided by the Thoracic Society of Australia and New Zealand research ethics committee which reviews of multicentre trials of respiratory drugs.²⁴

When researchers apply for public funding their proposal must be reviewed at several levels by experts to decide whether the research question is important, whether the methodology is sound and whether the aims are likely to be achieved. But surely the lives of potential participants are just as important as getting good value for money from our research dollar.

If ethics committees are to maximally reduce risk and promote effective communication of risks to participants, they require specialist expertise in pharmacology, systematic review, ethics, communication skills, and the science behind the research. Members should be paid as specialist professionals whose job it is to protect research participants.

We have two options. One is to increase our support and resourcing of ethics committees as they are currently structured. The other more radical option is to rethink the structure and function of research ethics committees. These comments are not intended to denigrate the important and selfless work many ethics committee members have contributed. But if we want to further reduce the risk to participants, we need to consider the possibility that ethics review is becoming too technical for the people currently asked to perform it, rather than blaming them when avoidable deaths occur. We need to consider the possibility of few expert suprainstitutional specialist ethics committees.