Article Text

Empirical evidence against placebo controls
  1. Sadhvi Batra1,
  2. Jeremy Howick2
  1. 1 University of Alabama at Birmingham School of Medicine, Birmingham, Alabama, USA
  2. 2 Nuffield Department of Primary Health Sciences, University of Oxford, Oxford, UK
  1. Correspondence to Sadhvi Batra, University of Alabama at Birmingham School of Medicine, 1720 12th St. S, Apt D Birmingham, AL 35205, USA; sadhvi010{at}


The revised Declaration of Helsinki allows placebo-controlled trials to be used even when there is an established therapy, provided there are adequate ‘methodological’ reasons for doing so. This seems to violate the principle of beneficence: where there is an established therapy, physicians treating patients with a placebo are withholding a known effective therapy. Because of this problem, we hypothesised that clinical researchers may be unwilling to risk violating the principle of beneficence and employ placebo-controlled trials in cases where there is an established therapy. In this paper, we began to investigate this hypothesis. After summarising the arguments for and against using placebo controls in clinical practice, we exploredthe extent to which placebo-controlled trials are used in cases where there is an established therapy. To do this, we conducted as systematic search for all placebo-controlled trials published in 2015 in the five highest impact general medical journals. We identified 70 placebo-controlled trials. Of these, 66 were for indications where there was no established effective therapy. Only four used a placebo control in spite of there being an available effective therapy. The infrequent use of placebo controlled trials where established therapy exists highlights a seeming discrepancy between what the Declaration of Helsinki allows and what clinical investigators believe to be ethically acceptable. The evidence presented in this paper suggests that the Declaration of Helsinki be reconsidered, and perhaps revised, in light of actual practice.

  • placebo

Statistics from

Rationale: the ethical arguments for and against placebo controls where established therapies are available

The original Declaration of Helsinki adopted in 1964 stated that the use of placebo controls was not ethically permissible if standard effective treatment is available: ‘in any medical study, every patient-including those of a control group, if any-should be assured of the best proven diagnostic and therapeutic method’.1 This seems reasonable: why would a patient knowingly enrol in a clinical trial in which they might receive a placebo when an effective standard therapy is available? Indeed, there is evidence to suggest that patients may not be aware of their options. Exacerbating this problem, patients are often not adequately informed about the purpose of placebo and the process of randomisation in control trials. If patients are unaware of the purpose of placebo controlled trials and that another effective therapy is available, it is questionable whether the ethical duty of informing the patient is being satisfied.2 Yet even if patients were aware, it still might be unethical for a medical practitioner or research to withhold a known effective therapy.

Physicians are ethically bound by the principles of beneficence and non-maleficence. Giving a patient a placebo when there is an established effective therapy seems to violate these principles. To be sure, the ethical duties of the clinical investigator may be different from the ethical duties of physicians.3 However, physicians are involved in placebo controlled trials, and it is fair to state that a physician giving a patient a placebo when there is an established therapy is, on one interpretation, violating the ethical principles of beneficence and non-maleficence.

In spite of the strong prima facie rationale for patients and research clinicians to avoid placebo-controlled trials where established treatments are available, the original Declaration was revised to allow the use of placebo controls in just those circumstances provided there are ‘sound methodological reasons’.4 The revised version in 2013 indicates that placebo use is warranted in instances in which this use ‘is necessary to determine the efficacy or safety of an intervention and the patients who receive placebo or no treatment will not be subject to any risk of serious or irreversible harm’.4 However, this does not put the ethical worries to rest. The requirement of non-maleficence requires that physicians do NO harm, rather than do irreversible harm. Anticipating the ethical objections, the authors of the Declaration stress that ‘extreme care must be taken to avoid abuse of this option’. However, they neither provide guidance as to what counts as a sound methodological reason nor formally require that researchers provide these in reports of placebo controlled trials.

Temple and Ellenberg attempted to provide these ‘sound methodological reasons’ in a series of two papers.5 6 One of us has reviewed these arguments carefully elsewhere,6 so we will limit ourselves to a brief summary here. While describing these arguments, we will refer to placebo-controlled trials as ‘PCTs’ and trials that use established effective therapies as controls as ‘ACTs’.

Temple and Ellenberg’s first argument against the relative value of ACTs is that only PCTs allegedly possess ‘assay sensitivity’. They define ‘assay sensitivity’ as the ability of a trial to distinguish between effective and ineffective treatments. The only example Temple and Ellenberg invoke to support their argument (and we believe that the failure to discuss other examples in any detail is a further weakness of their position) is that of selective serotonin reuptake inhibitors (SSRIs). They begin by asserting that they know SSRIs to be effective, yet SSRIs sometimes do and sometimes do not demonstrate superiority to placebos in clinical trials. So if we were to compare a new SSRI trial with an ‘active’ control (say another SSRI), and the new SSRI was as good as (the statistical term is ‘noninferior’) to the established one, we could not infer that the new SSRI was effective, because in that trial the established SSRI might not have been superior to placebo.

There are two problems with this argument. First, it does not apply to superiority ACT trials, where we seek to learn whether the new treatment is superior to the control. If a new SSRI were more effective than an established SSRI, then we could infer that the new SSRI was effective even if the established SSRI were no better than placebo in that trial (provided, of course, that the established therapy were no worse than placebo). Sometimes we may not want to require that new treatments demonstrate superiority to established therapies but merely rough equivalence (or ‘non-inferiority’). Demonstrating rough equivalence is sometimes justified on the grounds that a new treatment might be no more effective, but have fewer side effects, is more cost-effective, less invasive or can be used in patients who are resistant to the established therapy. These justifications have validity in some cases; however, they are likely to be exaggerated. We might like to have a few options in the case of resistance or side effects; however, we do not need dozens of options, and in the case of antidepressants, there are dozens of options such as ‘tricyclic agents, monoamine oxidase inhibitors (MAOIs), serotonin-norepinephrine reuptake inhibitors (SNRIs), noradrenergic and specific serotonergic antidepressants (NASSAs), norephinephrine (noradrenaline) reuptake inhibitors (NRIS), and norepinephrine-dopamine reuptake inhibitors’.7 Furthermore, there are additional treatment options for depression that are non-pharmaceutical. Moreover, if the justification for a new equivalent treatment is that it has fewer side effects, then we can test for a superior side effect profile. Hence, while conducting non-inferiority trials may be justified in some cases, in many others we would like to know whether a new intervention is better than what we already have.

Temple and Ellenberg acknowledge the problem that their argument only applies to superiority trials implicitly and restrict their discussion to non-inferiority trials. Their justification for this restriction amounts to a few sentences about the alleged benefits of non-inferiority trials and seem to ignore the arguments against non-inferiority trials (see ref 7 for a more complete discussion).7

The more serious problem seems to be that Temple and Ellenberg confuse a property of trials with a property of interventions. They claim that assay sensitivity is the inability of a trial to detect whether a treatment is superior to placebo. However, an alternative explanation for failure to demonstrate superiority to placebo is that the treatment in the trial is not more effective than placebos. Indeed, it has been argued that SSRI antidepressants are not (much) better than placebos for mild to moderate depression.8

The next alleged advantage of PCTs over ACTs is that only the former provide a measure of ‘absolute effect size’.5 6 This is only true if placebo effects are constant, which is not the generally case. Indeed, one way to interpret the data on SSRIs that Temple and Ellenberg cite is to propose that placebo effects in antidepressant trials are not constant. In general, different placebos have different effects and even the same placebos have highly variable effects. For example, in studies comparing cimetidine for treatment of ulcers to placebo, the effects of the placebo ‘in the same trials ranged from 10% to 90%’.7 In another example, neurogastroenterological study of treatments for irritable bowel syndrome found that the ‘placebo’ response ranged from 16.0% to 71.4% of the (roughly constant) experimental treatment.8 Although the substantial variation in ‘placebo’ effects could be that the placebos in question were illegitimate, placebos could also have inherently variable effectiveness. To be sure, placebo effects vary according to the condition, and in some cases the placebo effect might be more constant. However, we are not aware of any studies that test the stability of placebo effects per condition. Until such a study is done, the examples we cite suffice to show that placebo effects do not appear to be constant, and therefore placebo controls do not provide a baseline from which absolute effect sizes can be measured.

To summarise, Temple and Ellenberg’s attempt to provide ‘sound methodological reasons’ for conducting PCTs even when there are established therapies is problematic at best and at worst fail. Hence, the case that there are sound reasons for supporting the methodological superiority of PCTs over ACTs is not without controversy. If we are correct about this, then the rug is pulled out from under the rationale for revising the Declaration of Helsinki.

Independently of whether you accept our arguments against Temple and Ellenberg, it is also important to investigate the extent to which patients and doctors believe PCTs to be ethically acceptable when there is an established therapy. We began to measure the extent to which doctors believe this to be the case. Since it is always possible in principle to use a PCT, and since Temple and Ellenberg do not restrict the scope of the argument about the superiority of PCTs, their alleged ‘sound methodological reasons’ can always be invoked to justify using PCTs even when established therapies are available. If PCTs were frequently used even when there is an established therapy, this would seem to provide support for the claim that the ‘sound methodological reasons’ are accepted by patients and medical practitioners involved in research. However, if it turns out that PCTs are not (or rarely) actually used when established treatments are available, then we have tentative evidence that patients and doctors rarely accept that PCTs should be used if there is an established therapy. Furthermore, given the incoherence between the actual practice and what the revised Declaration allows, one might also recommend that the Declaration be revised so that it reflects what is believed to be acceptable to actual patients and doctors involved in research. This idea is supported by Rawls’ Theory of Reflective Equilibrium, suggesting that theories are revised if the theories and moral judgments conflict.9


We aimed to measure the prevalence of recent PCTs published in major general medical journals in which cases where established effective therapies are available.


Data sources and searches: PCTs

We focused on the five English-language general and internal medicine journals with the highest impact factors for 2015, as ascertained from the ISI Web site ( The New England Journal of Medicine (N Engl J Med), Journal of the American Medical Association (JAMA), The Lancet (Lancet), Annals of Internal Medicine (Ann Intern Med) and PLoS Medicine (PLos Med). One independent reviewer conducted PubMed searches using the ‘trial’ filter the term ‘placebo’ in the abstract for the 12-month period beginning in 2015. Articles were eligible if they presented original data from randomised trials in humans and reported using placebo controls placebos. We did not consider ahead-of-print publications.

All the journals we searched were listed in PubMed, and we used the search terms: ‘placebo’ in the title, 2015, the name of the journal and the clinical trial filter. For example, ‘placebo [TI] 2015 and JAMA’ was entered into the search box and the ‘clinical trial’ filter was selected under ‘article types’. Additional online searches of PCTs in each of the top five journals mentioned were conducted for thoroughness.

We focused on the year 2015 to examine recent practice, and we chose the five most widely cited journals to examine common/best practice and for feasibility considerations.

We excluded placebo ‘add-on’ trials, where either a new treatment or a placebo is added to an established therapy to test whether the new agent adds some benefit to the standard treatment regimen. This is because such trials do not count as trials where the established therapy was withheld from the control group.

Data sources and searches: systematic reviews of a potential control

For purposes of this paper, we assumed that there was an established therapy for the condition treated in the PCT if a systematic review of randomised trials supported the effectiveness of such a treatment. We searched for systematic reviews (using the ‘review’ filter in PubMed or manually typing ‘systemic review’ in Google Scholar) and the control condition (any field). We limited our search to reviews published up to 12 months before the clinical trials were published, since the treatment’s effectiveness had to be known at the time of the trial’s inception.

Data extraction: checking whether there was a systematic review establishing the effectiveness of a potential control therapy

One reviewer independently abstracted data from each section of eligible trials and examined the full text when necessary. The reviewer then recorded the PICO (population, intervention, control and outcome) information for each study (see table 1). Data about the PICO were compared with the PICO of the trial to check whether the review in fact established the effects of a control treatment for the same condition. A second reviewer (JH) was consulted to resolve conflicts.

Table 1

Studies that met inclusion criteria

Criteria for inclusion as a PCT that could have used an established effective therapy as a control

Based on our search for systematic reviews, we classified PCTs into one of the following two categories. Either:

  1. There was no established effective therapy. In these cases there was no question of violating the principle of beneficence.

  2. There was an established effective therapy. For example, Eggermont10 investigated intravenous infusions of ipilimumab in comparison with placebo in patients with stage III cutaneous melanoma. The primary outcome was recurrence-free survival, and the study showed statistically significant improved recurrence free survival. However, in a systematic review published by Mocellin,11 patients with high-risk cutaneous melanoma received interferon-alpha (IFN-α) therapy compared with observation and any other regiment other than IFN-α. The primary outcome was disease-free survival, and the review showed that IFN-α had significantly improved recurrence-free survival. We did not include placebo ‘add on’ trials, since all groups in such a trial receive the established therapy.


We identified 70 PCTs published in 2015 in the highest impact journals. Four of the 70 PCTs (5.7%) met our inclusion criteria: two from The Lancet, two from JAMA and none from the others (see table 1, figure 1). The trials were in the fields of cancer (n=2), mental illness (n=1) and surgery (n=1). Of the PCTs that did not meet our inclusion criteria, the reasons were either (A) there was no systematic review supporting the effectiveness of an established therapy published in a 12-month period prior to the clinical trial or at all, (B) there was a systematic review but the population was different or (C) systematic review did not conclude that the therapy was effective (see table 2).

Table 2

Sample of studies that did not met our inclusion criteria

Figure 1

Summary of methods. PRISMA flow diagram for data extraction shows that of the 70 studies analysed in the top five medical journals, four were found to meet the inclusion criteria.

The studies that met the inclusion criteria were conducted worldwide. For example, Eggermont et al compared intravenous infusions of ipilimumab to placebo in patients with stage III cutaneous melanoma across 91 hospitals in 19 countries.10 Symonds et al compared the addition of cediranib with carboplatin and paclitaxel chemotherapy with placebo in women with metastatic or recurrent cervical cancer at 17 UK treatment centres.12 Berwaerts et al conducted a PCT that studied the efficacy of paliperiodone palmitate among patients with schizophrenia in eight countries.13 Finally, Valente et al compared the effects of preoperative intravenous injections of dexamethasone to normal saline solution on oedema and ecchymosis in patients seeking rhinoplasty in Brazil.14

Valente et al was the only study that referred explicitly to the Declaration of Helsinki; they stated, ‘All aspects of the study were conducted in accord with the tenets of the Declaration of Helsinki’.14 Though the study was in accordance with the revised Declaration, a systematic review was identified that showed other steroids, not dexamethasone, as effective prevention against oedema and ecchymosis following rhinoplasty perioperative.15


Summary of findings

In cases where there is an established effective therapy, PCTs (as published in high impact journals) are extremely rare and amount to fewer than 6% of the published PCTs. In the rare cases where the Declaration of Helsinki was explicitly mentioned, the authors of the trial did not mention that they could have opted for an established therapy as a control rather than a placebo. Combined with the evidence that patients may not understand that the PCT is being used in a case where there is an effective therapy, our evidence points to a discrepancy between what the revised Declaration of Helsinki allows and what is considered acceptable by practitioners and patients.

Strengths and weaknesses of our study

This is the first attempt to quantify the prevalence of the use of PCTs in cases where established effective therapies are available that we are aware of. As such, it provides an empirical estimate of the prevalence of the extent to which the revised Declaration of Helsinki permission to use placebo controls where active treatments are available is actually invoked. The apparent discrepancy between the Declaration of Helsinki and current practice suggests that more empirical work needs to be done to determine (A) what patients consider acceptable and (B) what actually counts as methodological justification for using placebo controls when established treatments are available.

Nonetheless, this study is not without potential limitations. First, we might have failed to identify a systematic review. This is unlikely and would serve to strengthen our conclusions that the use of placebo controls where established effective therapies are available is very rare. Second, we chose to search the top five journals for the year 2015. Choosing a wider range of journals and other year might have yielded different results. We opted for a recent year to determine current practice, and limited our search to the top five given that research published in these journals are typically held to a higher standard. Using high impact factors of the journals may suggest that the findings are more generalisable.16 Yet another limitation is that our definition of ‘established effective therapy’ could be questioned. There could be established effective therapies that have not been supported by systematic reviews. We chose our definition because it was conservative. By expanding the definition of established effective therapies, we would have found fewer trials, and thus strengthened our conclusions. Finally, we limited our search to general medical journals. Other specialties might also yield different results.

Ethical and research implications of our study

Regarding limitations, our study demonstrates an apparent discrepancy between what the Revised Declaration of Helsinki allows and what is actually practiced. At least in principle, placebo controls are always an option, and the alleged methodological advantages of PCTs always apply. Hence, if one accepts the legitimacy of the Declaration’s revision and the arguments for the methodological benefits of PCTs, there should be far greater number of PCTs being used even when there are established effective therapies. The lack of PCTs where established therapies are available suggests that either such practice is not considered ethical by actual patients, practitioners or ethics committees, or that there are not many ‘sound methodological reasons’ for using PCTs where established therapies are available.17

Future research should expand the scope of this study by examining a wider range of journals, as well as survey patients and practitioners about their views on this issue. In addition, we do not know whether the lack of PCTs where established therapies are available reflects ethical attitudes or the lack of ‘sound methodological reasons’. This would also seem to be a reason to reconsider the revised Declaration. Moreover, drawing inferences from what people believe is ethical use of placebo controls to claims about ethical use would be committing the naturalistic fallacy.

Sometimes the risk and benefit in PCTs is difficult to determine. Often, such assessments are determined by the values of the patient and/or physician. For instance, in oncology, where prognoses are better understood, there are risk calculators as well as more data on anticipated benefits of chemotherapy that patients can use to make informed choices. However, in cardiac surgery, there are risk calculators but no tool for assessment of the benefits. Every procedure has rates of harm, but not necessarily quantification of the benefits. Perhaps this suggests another reason why placebo trials may be rare when there is an established therapy—it may be assumed that treatment may be better than no treatment or placebo due to the lack of a standardised tool of measuring risk versus benefit.

Nonetheless, one might also argue that placebo controls are acceptable in spite of there being established therapies provided that the condition in question is not very serious (eg, mild pain). However, even in these cases, one should not ignore the views of practitioners or patients.


The use of placebo controls where established therapies are available is rare, occurring in fewer than 6% of PCTs published in major medical journals. This counts as preliminary evidence that the Declaration of Helsinki does not reflect views of practitioners, who may be violating the principle of beneficence by engaging with PCTs where there is an established therapy. Certainly, researchers and physicians opting for placebo treatments in these circumstances should carefully inform trial participants that established and effective option is available. Further empirical investigations of patient and practitioner attitudes towards placebo controls should inform future revisions of the Declaration of Helsinki.


View Abstract


  • Contributors JH designed the protocol and assisted with the data analysis. SB did the data analysis. Both JH and SB drafted the manuscript.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.