Scientific imperatives impose practices on trials that potentially frustrate the therapeutic mandate of medicine, including randomisation, blinding, restrictions on the use of other treatments, and unalterable dosing schedules.^1,2 Yet because novel and hazardous agents are often first tested in people who have exhausted standard interventions, subjects are prone to misunderstanding clinical trials as last resort treatments.

Such misattribution of therapeutic intent to clinical trials, known as the “therapeutic misconception”,³ can be abetted inadvertently by physicians loathe “to debilitate a patient’s hope” or eager to attempt a new treatment that they have become invested in intellectually, emotionally, or financially.⁴ Resulting misunderstandings distort the decision making processes of patients by causing them to overestimate a study’s benefits, underestimate its risks, or neglect non-curative options that offer greater probability of personal gain.^1,5

The therapeutic misconception is well documented among severely ill patients entering clinical trials.^6,7,8,9,10 Though less well documented, therapeutic misconceptions may also be more common in trials of highly publicised, novel agents.¹¹ Similarly, trials where investigators have developed an experimental agent may be at increased risk of therapeutic misconception because investigators’ personal stake in the success of their innovation can affect their portrayal of a trial’s benefits. Human gene transfer (HGT) trials are characterised by all three risk factors and therefore offer an arena in which to study the therapeutic misconception. In this study we examined how therapeutic benefit is portrayed in cutting edge clinical trials by performing a content analysis¹² of consent documents used in HGT clinical trials.

METHODS

Materials

Inclusion criteria: all available consent documents for phase 1 HGT protocols between 1989 and 2001 were collected from the NIH’s Office of Biotechnology Activities (OBA). Phase 1 trials were identified using the OBA clinical trials database, which was also used to identify principal investigators (PIs), the disease being investigated, and the date of protocol submission to Recombinant DNA Advisory Committee (RAC).¹³ Exclusion criteria: phase 1/2 trials were excluded from the analysis, as were nine forms used for genetic disease trials that were explicitly described as scientific feasibility experiments and targeted tissues that were unrelated to the pathologies of human subjects. A total of 277 consent forms were available.

The OBA database was also used to identify PIs who had conducted five or more HGT clinical trials before 2002; these PIs were termed “high volume” trialists. Identities of these investigators were coded to preserve their anonymity and entered into the database. When multiple PIs were associated with a particular trial, a consent form was only considered to belong to a high volume trialist if they or their institution were clearly indicated on the consent form.

Content analysis

Therapeutic optimism was defined as the extent to which therapeutic potential or intent was implied. Therapeutic forbearance was defined as the extent to which therapeutic potential was downplayed. Each form was analysed for several variables expressing optimism or caution. Consent form introductions were defined operationally as all material preceding any discussion of specific procedures or risks. Each introduction was examined for whether it used the term “gene therapy” at least once, and whether it stated anywhere that direct benefit to research participants was unlikely. Whereas use of the term “gene therapy” (as opposed to more neutral terminology like “gene transfer”¹⁴) was taken to signal therapeutic optimism, statements that benefits were improbable were interpreted as a marker of therapeutic forbearance.

Benefits paragraphs were defined as (1) any discrete paragraph or consecutive paragraphs containing a cluster of statements about possible benefits and not containing detailed information on methods or procedures, or (2) any section titled “Benefits”. Benefits paragraphs were analysed for benefits language by scoring each sentence for six categories (see table 1). The first three measured description of direct benefit: possible benefit statements were scored +1, uncertainty of benefits statements 0, and improbability of benefit statements were scored –1. The remaining three categories scored each sentence that described aspirational benefits (A), restatements of purpose (P), or warnings about risks of participation (H). Aspirational benefits refer to benefits accruing to others, including science, other patients, and society. When sentences contained multiple clauses that could be scored in more than one category, each was coded. Statements or clauses not fitting into the above categories (for example, descriptions of collateral benefit, information about an agent’s mechanism, statements about distant future benefits possibly accruing to subjects, and so on) were left uncoded.

Three measures of therapeutic optimism were derived from this coding scheme. Firstly, a “benefits possibility” score was obtained by averaging each sentence scored for direct benefits. For example, a benefits paragraph such as the following: “You may experience shrinkage of your tumour, however this can not be guaranteed. Your participation will benefit society …” would score 0.5 by averaging together the score for the first sentence (which describes a benefit as possible and thus scores +1) and the second (which describes a benefit as uncertain and thus receives a 0). The second variable measured the proportion of described benefits that pertained to the experimental subject (termed “direct benefits proportion”). This was calculated by dividing the number of direct benefit sentences by the sum of all sentences describing possible direct and indirect (for example, aspirational) benefits. Thus, the above example would receive a score of 0.5 for direct benefits proportion by dividing the number of sentences describing possible benefits (1) by the sum of possible benefit sentences (one direct benefit plus one aspirational benefit). A direct benefits proportion below 0.5 would indicate that over half of all sentences describing possible benefits within a form’s benefits paragraphs were aspirational. Thirdly, each statement coded in any of the six categories above was additionally coded for whether it referred to the investigational agent using experimental (−1), neutral (0), or therapeutic terms (+1). Agents were only scored when they referred specifically to the product being tested. Average agent terminology was calculated for each form.

Therapeutic forbearance within benefits paragraphs was measured by two variables. Firstly, restatement of a trial’s purpose within a benefits paragraph was interpreted as an attempt to remind prospective subjects about the scientific nature of the trial. The assumption that such statements thwart therapeutic misconceptions is consistent with observations that patients who were unable to describe the purpose of a trial are more likely to expect therapeutic benefit as a result of trial participation.⁶ Secondly, some oncology trial benefits paragraphs contained sentences warning prospective subjects about possible harms; such statements in the context of benefits descriptions were taken to provide another indication of therapeutic caution.

All benefits paragraphs were coded by the authors blindly with respect to the identity of PIs and date. Reliability estimates were calculated for each scoring category (table 1). Discrepancies in coding were resolved by consensus or, where consensus was not possible, by averaging the value of each author’s codings. When irresolvable differences were obtained for nominal values, a positive scoring was assumed. Nominal variables within the introduction were coded by one author (JK), with the other author coding 17% of the forms a second time for the purposes of error analysis. Cohen’s kappa scores obtained were 0.928 for the statement that benefits were unlikely within introductions, and 0.950 for the use of the term “gene therapy.”

Comparison of benefits language scores for high and low volume, and analysis of trends

To examine how benefits language varied according to an investigator’s HGT research activity, benefits language scores needed to be corrected for variation according to disease category. For example, if genetic disease trials express more caution with respect to benefits language, and a disproportionate number of trials conducted by high volume trialists are for genetic disease, a high volume trialist benefit language average across disease categories would be deceptively low. Disease category language variation was corrected by standardising all continuous variables as follows: we obtained average benefits language scores for low volume trialists within each disease category, and subtracted this value from each continuous variable score belonging to the appropriate disease category. Whereas the resulting average score for low volume trialists within disease categories for each of the three variables would logically be zero, the average for high volume trialists reflects their departure from low volume trialist scores within the appropriate disease category. To enable a similar analysis of trends, average disease category benefits language scores were subtracted from each trial score according to its disease category.

Data analysis

All data were entered into Excel 98, which, along with Statview 5.0 (SAS Institute Inc, Cary, NC, USA) was used to derive summary statistics and to perform all analyses. Statistical significance was defined as p⩽0.05.

RESULTS

Measuring therapeutic intent and caution in phase 1 HGT consent forms

This study was designed to characterise the extent to which consent forms used in HGT trials suggest or caution against expectation of therapeutic benefit, and whether active HGT investigators are more likely than less active HGT developers to project therapeutic optimism in their consent documents. Only phase 1 studies were included in the analysis, because such trials typically enrol more severely ill subjects and involve experimental agents that, owing to uncertainties surrounding their properties and administration, are least likely to produce personal benefits to enrolled subjects.

Benefits paragraphs—most of which were titled as such—were available for 87.5% of consent forms (benefits descriptions in the remainder were either absent or buried within paragraphs describing procedures or methods). As table 2 indicates, benefits possibility scores averaged less than 0.5, meaning in general that most forms devoted more sentences in their benefits paragraphs to discussing uncertainties than possible benefits. Slightly more than half of all descriptions of possible benefits concerned personal (as opposed to aspirational) benefits. Agents were on average described using therapeutic terminology, and a minority of forms described a particular benefit as unlikely or restated their purpose in the benefits section. Forms were generally more emphatic about their optimism than about their caution. For instance, where 16.7% of forms containing benefits sections scored greater or equal to 0.5 for both agent terminology and benefits possibility, no forms scored negatively for both categories, and only 6% of forms scored negatively for benefits possibility and neutral (0) for agent terminology. Though therapeutic optimism and caution varied considerably according to disease categories, only genetic disease consent forms showed noticeably greater caution, with 63% stating clearly that personal benefits were improbable. Analysis of variance with post hoc comparisons using Fisher’s PLSD showed that genetic disease trials were significantly more likely than other disease trials to score lower for benefits possibility, but higher for use of the term “gene therapy.” In addition, genetic disease trials were significantly more likely than cancer and HIV trials to state that benefits were unlikely.

Benefits language of investigators highly active in HGT research

These data were next used to determine whether investigators who are deeply involved in gene transfer development—and therefore more personally invested in their success—are more prone to expressing therapeutic optimism in consent forms.

The identities of investigators who had conducted five or more HGT trials before 2002 (for shorthand, “high volume trialists”) were culled from the RAC database. Though this crude metric of HGT activity excluded several acknowledged HGT pioneers, we assumed that whatever tendencies these excluded pioneers had with respect to therapeutic language would be diluted by the numerous investigators who had conducted only one or two trials (“low volume trialists”). This classification system was tested by performing “gene therapy OR gene transfer” PubMed searches on each of the high volume trialists and an equal number of random low volume trialists listed as PIs on consent forms. High volume trialists averaged significantly more HGT publications before 2002 than their low volume counterparts (35.5 v 15.6, p = 0.026), suggesting that high volume trialists were more actively engaged in HGT research. Our database contained 75 consent forms drawn from 23 high volume trialists for a per investigator average of 3.26 forms.

After correcting for variation according to disease category by standardising all continuous variables, high volume trialists were significantly more optimistic with respect to benefits possibility and direct benefits proportion (table 3). However, high volume trialists tended to use more guarded terminology for experimental agents. Our dataset was insufficiently large to permit simple tests of statistical significance for nominal variables. In general, nominal averages were consistent with the above conclusions, with measures of therapeutic caution tending to be less frequent but agent terminology (for example, use of the term “gene therapy”) being more cautious for high volume trialists. Differences in benefits language within the largest disease category, cancer, are indicated in table 4. Although statistical significance was lost for benefits possibility, it held up for direct benefits proportion, agent terminology, and restatement of purpose.

Trends in therapeutic language

Numerous commentators have observed that the boundary between therapy and experiment may be eroding, characterising the trend as “benefits creep”.^15,16 Gene transfer experiments have generated considerable excitement and have been conducted for over a decade; they therefore provide a record with which to test such allegations.

To examine whether therapeutic optimism has increased over the past decade, benefits indicators were plotted over time after correcting values for variation according to disease category. As indicated in figure 1, a subtle trend toward less therapeutic optimism was apparent for all three measures (Pearson’s r values for benefits possibility, direct benefits, and agent terminology were −0.210, −0.190, and −0.054), though linear regressions of each failed to show statistically significant correlation. We also examined trends in therapeutic language using only oncology trial data. Expressions of therapeutic forbearance generally increased with time (fig 2B), while expressions of therapeutic optimism declined somewhat (fig 2A). However, only statements within benefits paragraphs that harms were possible correlated significantly with time (p = 0.005). These data provide encouraging indications that consent forms may not have deteriorated with respect to the therapeutic misconception. However, they also indicate that the authors of these forms have not responded aggressively to concerns about the therapeutic misconception, or to controversies surrounding HGT in the wake of a widely publicised 1999 volunteer death (Weiss R, Nelson D. Teen dies undergoing gene therapy. Washington Post September 29, 1999; page A1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

 Benefits language in HGT consent documents over time: benefits possibility, direct benefits, and agent terminology scores for each form were adjusted for variation according to disease category by subtracting the average of the appropriate disease category. Differences were plotted over time. Year 1992 is inclusive of all RAC approved trials conducted up to and including 1992.

Figure 2

 Benefits language in HGT oncology consent documents over time: measures of benefits optimism (A) and forbearance (B) are plotted over time. Year 1992 is inclusive of all RAC approved trials conducted up to and including 1992. Pearson’s r values were −0.307, −0.275, and −0.362 for benefits possibility, direct benefits, and agent terminology, respectively; −0.173, 0.893, 0.331, and 0.355 for purpose restatement, harm possible, benefits unlikely, and aspirational benefit, respectively.

DISCUSSION

In this study, consent forms used in phase 1 trials of an unproven class of experimental agents were analysed for language suggesting therapeutic optimism or caution. Expectation of benefit by participants and investigators would be unrealistic under such conditions of clinical immaturity: not only are safety, dosage, and efficacy of the agents not established, but the entire category of gene transfer remained without validation for the period reviewed here. Notwithstanding the claim—often made by oncologists—that phase 1 trials have therapeutic intent,^17,18 there is little empirical evidence that patients generally benefit medically from participation in phase 1 trials,^19–21 and survey data indicate that only 6% of oncologists believe that patients benefit medically from enrolment in phase 1 trials.²² Moreover, in addition to undergoing standard Food and Drug Administration and IRB review, HGT consent forms are evaluated at the national level by the RAC. Observing therapeutic optimism in HGT consent forms, therefore, might provide an a fortiori case that the consent process is not being appropriately guarded against the therapeutic misconception.

Although this proposition is supported by the present study, several issues limit the strength of our conclusions. Firstly, consent forms fail to capture the content of any discussions between PIs and research subjects. Nevertheless, this study presumes that consent document language frames the consenting process and provides a surrogate indicator of how various factors are discussed during consent discussions. A related study limitation concerns uncertainty about whether the variables measured actually influence the perception of potential research participants. There would appear to be a logical case that benefits language influences the perception of human subjects, and terminology has been shown to influence risks and benefit perception.^23–25 Without actual survey data, however, we cannot be certain that our coding methods reflect how actual patients perceive benefits language.

Like any content analysis, this study is reductive in that it sacrifices nuance and complexity of language in order to extract objective measurements. Also, the coding method used here did not exhaustively catalogue all gestures consent documents used to thwart therapeutic misconceptions, and we cannot rule out the possibility that some documents contained other types of statements designed to forestall therapeutic misconceptions. Additionally, many consent forms buried benefits related information within their introductions or statements of purpose. Apart from statements about the improbability of benefit, these sentences were not scored because of the difficulties of disentangling descriptions of study purpose, agent design, and benefits.¹⁵ Though we consider it unlikely, benefits descriptions may have varied systematically between consent forms having a benefits paragraph and those that did not. We therefore cannot rule out the possibility that the exclusion of 35 forms lacking benefits paragraphs from some of our analyses did not bias our study.

Finally, the coding scale used to score benefits language is biased to detect small gradations in expression of benefit. For instance, positive scores for benefits possibility reflect statements of possibility rather than probability; in contrast, negative scores contain probabilistic information. Therefore, high benefits language scores do not generally reflect extravagant expressions of benefit.

A recent study reported that phase 1 trial consent documents aptly describe their purposes in terms of generalisable knowledge, and state the uncertainties with respect to benefits for trial participants.²⁶ We emphasise here that our review of HGT consent forms is not inconsistent with this favourable appraisal. Nevertheless, measures used in the aforementioned study may not capture the degree to which consent forms endeavour to counter therapeutic misconceptions.

Within the parameters of these qualifications, the present study indicates that although clinicians communicate uncertainties to prospective research participants, only a minority use consent forms to advise research participants about any improbability of response, improvement, or even cure. These results also confirm that consent documents overwhelmingly use therapeutic terminology in describing experimental agents. We also provide evidence that benefits descriptions vary across medical specialties, with genetic disease trials using the term “gene therapy” more commonly, but otherwise showing significantly greater caution in communicating benefits. The latter may reflect the fact that many of these trials are conducted in people with stable and relatively mild forms of genetic illness.

The results offer evidence that therapeutic optimism is conveyed more strongly in trials conducted by investigators who are active in the development of HGTs. High volume trialists might have been expected to practice greater therapeutic caution on three grounds. Firstly, the HGT field has twice been publicly held to account for its exuberance: once in 1995, when an NIH panel concluded that “overselling” of HGT “[had] led to the mistaken and widespread perception that gene therapy is further developed and more successful than it actually is”,²⁷ and again after the 1999 death of a patient volunteer heightened questions about the field’s adherence to ethical practices. High volume trialists would be more motivated to protect the credibility of their field. Secondly, because of their familiarity with HGT’s complexity, they might take a longer and more modest view of gene transfer research. Thirdly, their attendance at RAC meetings and experience with upheaval might have sensitised high volume trialists to the hazards of overstating benefit.

Indeed, high volume trialists were significantly more cautious with agent terminology; within disease categories, they were also less likely than low volume trialists to use the term “gene therapy”, although significance could not be established. Nevertheless, high volume trialists used significantly more optimistic benefits language than others. These results suggest that active HGT investigators might somehow communicate their private and scientific enthusiasm for HGT to prospective subjects, though we can not rule out the possibility that high volume trialists might be affiliated with institutions whose IRBs are less attentive to therapeutic misconceptions, or high volume trialists’ studies may have differences that warrant more emphatic benefits descriptions. On the basis of these findings, we believe PIs and IRBs should be especially attentive to moderating benefits language where investigators are known to be involved in the development of the agents they test. In addition, an extra sentence might be added to consent documents disclosing an investigator’s personal investment in the line of inquiry being tested; such a disclosure would begin to address a source of conflict of interest that is probably more pervasive in clinical trials than those relating to financial relationships.²⁸

These data contain several encouraging findings as well. Firstly, this study provides indications that consent documents are increasingly mindful of the various ways that the therapeutic misconception can be avoided. Trends consistently indicate less emphasis on direct benefit and therapeutic terminology, and more emphatic expressions of therapeutic caution. Secondly, these data indicate that consent documents used in trials for terminal diseases, where the therapeutic misconception is of particular concern, are not necessarily more prone to therapeutic optimism. For instance, over half of the cardiovascular consent documents were drawn from limb revascularisation trials; nevertheless, cardiovascular trials received average scores that were similar or more optimistic than those for cancer, which is life threatening. In addition, oncology consent forms selected for statements that eligible subjects had an incurable condition scored for benefits language on average very similar to non-terminal cancer trials (data not shown). This observation is somewhat surprising, since disease severity tends to influence whether trials are conceptualised as “experiments” or “treatments.”²⁹

Although these results pertain to HGT trials, our intent was to use gene transfer as a model for studying the consent process when new and unproven, research intensive medical technologies reach the clinic. The present results provide empirical evidence of ethical problems that arise in trials of novel agents, particularly when physician scientists conduct trials of their own investigational agents.