Article Text

Download PDFPDF

Evaluating solutions to sponsorship bias
  1. M Doucet1,
  2. S Sismondo2
  1. 1
    Department of Philosophy, John Watson Hall, Queen’s University, Kingston, Ontario, Canada
  2. 2
    Department of Philosophy, Queen’s University, Kingston, Ontario, Canada
  1. Mr M Doucet, Department of Philosophy, John Watson Hall, Queen’s University, Kingston, Ontario, Canada K7L 3N6; 1wmd1{at}queensu.ca

Abstract

More than 40 primary studies, and three recent systematic reviews and meta-analyses, have shown a clear association between pharmaceutical industry funding of clinical trials and pro-industry results. Industry sponsorship biases published scientific research in favour of the sponsors, a result of the strong interest commercial sponsors have in obtaining favourable results.

Three proposed remedies to this problem are widely agreed upon among those concerned with the level of sponsorship bias: financial disclosure, reporting standards and trial registries. This paper argues that all of these remedies either fail to address the mechanisms by which pharmaceutical companies’ sponsorship leads to biased results—design bias, multiple trials with predictable outcomes, fraud, rhetorical effects and publication bias—or else only inadequately address those mechanisms. As a result, the policies normally proposed for dealing with sponsorship bias are unable to eliminate it. Only completely separating public clinical research from pharmaceutical industry funding can eliminate sponsorship bias.

Statistics from Altmetric.com

EVALUATING SOLUTIONS TO SPONSORSHIP BIAS

More than 40 primary studies, and three recent systematic reviews and meta-analyses, have shown a clear association between pharmaceutical industry funding of clinical trials and pro-industry results.13 Many commentators have rightly seen this association as evidence of a problem, and there are some common suggestions of remedies to that problem.1 2 410 This paper takes a needed novel approach, because the policies standardly suggested as remedies to sponsorship bias either fail to address the mechanisms by which pharmaceutical companies’ sponsorship leads to biased results, or else only inadequately address those mechanisms. As a result, the policies normally proposed for dealing with sponsorship bias—financial disclosure, reporting standards, and trial registries—are unable to eliminate it. More radical measures are required if sponsorship bias is to be eliminated.

MECHANISMS OF BIAS

Pharmaceutical company sponsorship of clinical trials is a spectrum of activities. On one end we might find a trial that is conceived and designed by a company, executed by a contract research organisation, written up by ghostwriter, merely approved by an academic author and then submitted to a journal by a medical education and communication company; such a paper, fairly common in the literature, is entirely ghost managed by the company and its agents.11 12 On the other end of the spectrum we might find a trial that is fully conceived, designed, executed and written up by independent researchers, who have simply sought financial support from a pharmaceutical company. In both cases and all between, sponsorship introduces interests in particular results into the scientific process, strong interests in the first case and subtle ones in the second. Thus there is pressure, familiar from studies of conflict of interest, to produce results favourable to the company.8 13

Our focus is the distorting effect of commercial interests on the clinical trial literature. We therefore say little about the effect personal or professional conflicts of interest. Industry sponsorship has a significant biasing effect when measured against non-sponsored trials, not merely against an imagined bias-free baseline; that effect is in consistent and predictable directions. If other sources of conflict of interest distort results, they do so on industry and non-industry funded trials alike, and so financial conflicts of interest have a significant biasing effect over and above the effect of other conflicts.

Commercial interests explain why the clinical trial literature displays bias, but not how that bias occurs. We have previously identified five mechanisms that translate interest into biased results, all of which have been demonstrated to contribute to sponsorship bias.14

  1. Design bias: Trials may be designed to make favourable conclusions more likely. For example, a placebo-controlled trial is likely to show a drug to be effective, but a comparison to the standard of care usually produces more meaningful information. Designs can be biased through choices of comparators, dosages, protocols, populations, endpoints, and trial durations.10 While industry-sponsored trials tend to have designs that bias results, it should be noted that they also tend to score higher on general methodological evaluations.2 1519

  2. Multiple trials with predictable outcomes: Many post-marketing trials are created as marketing tools—directly to researchers and/or to readers of journal publications—rather than to create new knowledge. They follow established patterns and thus are likely to arrive at established results.20 21

  3. Fraud or scientific misconduct: Trials are sometimes terminated without justification, and data is sometimes concealed, altered, or strongly misrepresented.22 23 There is no evidence that such misconduct is more common in industry-sponsored trials than in others, but it is reasonable to assume that misconduct in a pharmaceutical company’s funded trials will tend to support that company’s interests.24

  4. Interpretive and rhetorical effects: Data do not speak for themselves. Articles and abstracts reporting clinical trials are always rhetorically constructed, though sponsored ones have been shown to be more likely to be constructed to favour sponsors.4 25

  5. Publication bias: Pharmaceutical company sponsored trials are more likely to be published prominently than are comparable independent trials.11 Their results are more likely to be published if they are favourable than if not; positive results, reworked in different combinations, have been found to be published multiple times, and many times more than negative results.26

With these mechanisms in mind, we can evaluate some of the common proposed remedies for sponsorship bias.

FINANCIAL DISCLOSURE

Many articles on the problem of sponsorship bias propose that journal editors require full disclosure of the financial relationship between researchers and the pharmaceutical industry as a condition of publication.1 2 5 79 18 2730 Full disclosure of such relationships is undoubtedly a good idea. Yet the current problem of sponsorship bias in medical research exists in a climate where disclosure, although not always mandatory or rigorously enforced, is widespread; indeed, the evidence of bias typically depends upon disclosure. Stronger rules and stricter enforcement might uncover some currently hidden cases of bias, but they would not mark a dramatic shift in policy, and so would not have a dramatic effect of the prevalence of sponsorship bias, except possibly to reveal more.

More important, even a rigorous disclosure policy will not eliminate sponsorship bias, since disclosing interests is not the same as eliminating them. Disclosure does not directly address any of the above mechanisms by which sponsorship has its effects, because it merely points to the open barn door after the horses have gone. To see this, we can look at the mechanism most likely affected by full disclosure, the rhetorical creation of bias. Disclosure of financial interest (stronger than disclosure of funding) has been shown to affect readers’ perceptions of the value and strength of medical articles, and thus might mitigate the power of rhetoric.31 However, there is also evidence that people disclosing conflicts of interest are sometimes more trusted than those who do not, and that disclosure can be taken as a “moral license” to provide biased information, and so can actually increase bias.32 Evidence also shows that industry sponsored reports tend to be more highly cited than non-industry sponsored reports.11

Better financial disclosure might be effective at rooting out sponsorship bias if a few biased, sponsored trials distorted a body of literature composed primarily of unbiased, un-sponsored trials. On that scenario, disclosure would separate the unbiased wheat from the biased chaff. However, in the USA and elsewhere, 70% of funding for clinical trials comes from the pharmaceutical industry.5 A policy improving disclosure will merely confirm what is widely known: a large majority of trials are funded by the pharmaceutical industry. Moreover, since trials sponsored by the pharmaceutical industry are generally of equal or higher methodological quality when compared to other trials, discovering that an article has been written by a researcher with industry support is not a sign to readers that something is amiss, but rather confirmation that the article is a piece of normal science.

CONSORT

Authors of many of the studies identifying sponsorship bias recognise that financial disclosure policies do not fully address the problems they identify. They therefore frequently suggest policies to prevent pharmaceutical industry sponsorship from distorting the literature on clinical trials. Two common suggestions are the adoption of a rigorous standard for trial reporting, and a prospective registry of all clinical trials, discussed in the next section. While both of these suggestions aim to address the effects of conflict of interest, neither of them is effective at eliminating sponsorship bias.

The Consolidated Standard of Reporting Trials (CONSORT) Group explicitly recognises that poor reporting “is associated with biased estimates of treatment effect”, and aims to make trial reporting more transparent.6 For this reason, several articles on sponsorship bias specifically endorse CONSORT and others endorse similar proposals, such as the International Committee of Medical Journal Editors Uniform Requirements.2 4 16 18 33 Rigorous standards for trial reporting should to be more widely adopted and enforced. Such standards will not, however, eliminate sponsorship bias because they do not properly address the above mechanisms of bias.

  1. CONSORT encourages methodological quality in trials, an important goal. But studies of sponsorship bias report that trials funded by pharmaceutical companies appear to be of high methodological quality in the ways that CONSORT encourages. Demonstrated exceptions to quality in sponsored trials are in such aspects as the choice of appropriate comparators, but CONSORT does not require that appropriate comparator are selected: biased designs are consistent with CONSORT guidelines.

  2. CONSORT regulates the conduct of individual trials. This means that it cannot establish whether a trial simply aims to reproduce a previously established predictable outcome. A strategy that focuses on the quality of individual trials cannot address itself to the overall shape of the literature.

  3. While CONSORT might make certain kinds of fraud or misconduct more difficult, those willing to break rules are unlikely to be strongly deterred by new rules. The motive for fraud is created by conflict of interest between accurate reporting and financial gain, so the best way to eliminate fraud is to remove the conflicts of interest.

  4. In promoting transparency CONSORT puts constraints on rhetorical or interpretive effects, particularly if these are based on poor or incorrect reporting. Studies of industry-sponsored trials have shown that published results are often poorly supported by data. But while CONSORT recommends that authors should “discuss imprecisions”,6 it does not impose standards that eliminate the possibility of interpretive and rhetorical effects on conclusions. Indeed, it is hard to see how such standards could be imposed. So while CONSORT can reduce the possibility of rhetorical effects, it cannot eliminate them.

  5. Finally, CONSORT cannot address the problem of publication bias since, as with financial disclosure, it comes into effect only once studies are submitted for publication. Publication bias occurs at earlier stages, in decisions to create new analyses, or about whether and how many articles to submit.

While CONSORT or some similar reporting standard is worth adopting widely, it only partially addresses one of the five potential sources of sponsorship bias.

TRIAL REGISTRIES

Many researchers who present evidence of sponsorship bias in the clinical trial literature recommend the establishment of a public registry of all trials.1 2 4 5 810 20 24 29 3337 Investigators would be required to register a study’s protocol before initiating the study, and regulators and journal editors would refuse to consider evidence from unregistered trials.

Public trial registries, particularly those that also include the results of unpublished trials, can help ensure that the literature, and so retrospective analyses of that literature, more accurately reflects all studies. A robust trial registry would represent a marked improvement over current practices, under which negative trials never see the light of day. However, trial registries, like financial disclosure and reporting standards, do not aim to eliminate the conflicts of interest that are at the source of sponsorship bias. They aim instead to address the mechanisms by which such interests distort the scientific literature, and they are at best only partially successful. Registries can make retrospective audits and corrective meta-analyses more straightforward—though never easy—but only well after sponsorship biases have had their major effects on the medical literature. Even a rigorous trial registry will not significantly reduce sponsorship bias.

  1. Trial registries have a limited effect on the issue of design bias, as registration is unconnected with trial design. Trial registries will certainly allow for much more systematic comparisons. However, given the extent to which the pharmaceutical industry controls the conduct of clinical trials, it is quite possible that retrospective audits and meta-analyses will not reveal bias in an illuminating way.

  2. Public trial registry does not restrict the performance of multiple similar trials, if they were properly recorded. Indeed, public registration can add legitimacy to these trials.

  3. A prospective, public trial registry can help curtail some kinds of fraud and misconduct, since it can make wholesale hiding of negative results more difficult. Like other proposed solutions, however, a registry does not eliminate fraud and misconduct, since those willing to break the rules are unlikely be deterred by the introduction of new ones.

  4. A trial registry does not prevent rhetorical or interpretive effects that bias published reports. While better audits or meta-analyses might eventually reveal the extent to which the claims are overstated, a trial registry does not help the reader of any particular study uncover the extent to which its claims are the result of rhetorical or interpretive manipulation.

  5. On a more positive note, trial registries do have the potential to clearly demonstrate publication bias. If there continue to be systematic preferences, on the part of industry, authors and journals, for trials with positive results, then a public trial registry will eventually reveal these preferences.

Nevertheless, even with regard to publication bias, trial registries are limited. The earliest trials of a drug are the ones most likely to be sponsored by the drug’s manufacturer, and to set the agenda for the drug in question. Pharmaceutical companies have the resources—medical writers, statisticians, and publication planners—required to speed up the publication process and, also as a result of these resources, articles written by authors with pharmaceutical company are more likely to be published in prominent journals with higher impact factors than articles written by researchers without such sponsorship.11

Though trial registries can tell us much about the existence of sponsorship bias in general, and publication bias in particular, the power of analysis based on an extensive trial registry comes only after the drug has been approved and favourably introduced to physicians. This is not to minimise the importance of trial registries, but rather to point out that the power of such registries lies largely in what they can allow analysts to discover after the fact: they are effective tools for retrospective audits of bias, not prophylactics against bias. If the goal is to reduce the influence of the pharmaceutical industry on regulators, journal editors and practicing physicians, then trial registries, while offering an important contribution, are inadequate to the task.

CONCLUSION: WHAT IS TO BE DONE?

Since sponsorship bias is a result of conflict between science and marketing, strategies for reducing bias that do not address the underlying conflict are bound to be, at best, only partially effective.

This conclusion is not surprising: as long as the pharmaceutical industry controls the production of knowledge, the interests of the industry will influence the knowledge that is produced. Those who pay the piper call the tune and restricting the tune list does not change this fact. The conclusion is worth emphasising, however, because even authors who most strongly identify sponsorship as a problem appear to believe that modest changes in the regulation of clinical trials can eliminate bias. However, nothing short of radical reform can succeed in eliminating the distorting effect of pharmaceutical industry interests on the scientific literature.

Required is what Schaefer calls the “sequestration” of drug research and the pharmaceutical industry. Both Schaefer and Angell call for the establishment of an “Institute for Prescription Drug Trials” that would fund and oversee the conduct of all clinical trials.20 36 It alone would decide which trials are undertaken and would contract public research institutions—universities and hospitals—to carry out the research. Both the Institute and the individual trials would be paid for by a general levy on the pharmaceutical industry, but companies would have no say in the running of the Institute, in the decisions about which trials to undertaken, or in the conduct of individual trials. Medical journals would be strongly encouraged to publish only the results of trials run under the direction of the Institute and the drug approval process would only consider results from Institute-run studies. In order to complete the sequestration, public research institutions would be barred from having any financial relationships with the pharmaceutical industry.

This is a radical proposal and there are others that might have similar effects. For example, Tim Hubbard and James Love have proposed replacing the patent system for encouraging pharmaceutical research and development (R&D) with a prize system.38 The prizes would reward R&D in healthcare on the basis of its measured positive health outcomes, rather than sales. Meanwhile, all drugs would be available as generics and as such would not be the focus of such intense marketing. If properly constructed and implemented, such a system would effectively split pharmaceutical R&D from sales and reduce companies’ financial interests in medical publications. Like an Institute for Prescription Drug Trials, this proposal would require a public investment in clinical research. Given the financial clout and political sway of the pharmaceutical industry—in America, it spends more money on lobbying than any other industry20—such solutions are extremely unlikely to be adopted. But only such radical proposals have a chance at eliminating sponsorship bias. Even strict compliance with the best reformist proposals cannot fully address the problem of sponsorship bias and might even inadvertently increase that bias.32 Nothing short of a radical re-imagining of the relationship between research and industry can succeed in eliminating the distortions of the pharmaceutical industry on the scientific literature.

REFERENCES

Footnotes

  • Funding: Support: This research was supported in part by grants from the Social Sciences and Humanities Research Council of Canada.

  • Competing interests: None.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Other content recommended for you