Article Text
PDF
Abstract
Context Physicians are regularly confronted with research that is funded or presented by industry.
Objective To assess whether physicians discount for conflicts of interest when weighing evidence for prescribing a new drug.
Design and setting Participants were presented with an abstract from a single clinical trial finding positive results for a fictitious new drug. Physicians were randomly assigned one version of a hypothetical scenario, which varied on conflict of interest: ‘presenter conflict’, ‘researcher conflict’ and ‘no conflict’.
Participants 515 randomly selected Fellows in the American College of Obstetricians and Gynecologists' Collaborative Ambulatory Research Network; 253 surveys (49%) were returned.
Main object measures The self-reported likelihood that physicians would prescribe the new drug as a first-line therapy.
Results Physicians do not significantly discount for conflicts of interest in their self-reported likelihood of prescribing the new drug after reading the single abstract and scenario. However, when asked explicitly to compare conflict and no conflict, 69% report that they would discount for researcher conflict and 57% report that they would discount for presenter conflict. When asked to guess how favourable the results of this study were towards the new drug, compared with the other trials published so far, their perceptions were not significantly influenced by conflict of interest information.
Conclusion While physicians believe that they should discount the value of information from conflicted sources, they did not do so in the absence of a direct comparison between two studies. This brings into question the effectiveness of merely disclosing the funding sources of published studies.
- Conflict
- drugs and drug industry
- literature
- pharmaceutical
- physician
- prescribe
Statistics from Altmetric.com
Private industry plays an important role in generating and disseminating new scientific knowledge, but given strong profit motives, experts affiliated with private industry face significant conflicts of interest, which may consciously or unconsciously influence the way they interpret and discuss scientific evidence. Reviews have shown that industry-funded studies may be several times more likely to produce results and conclusions favourable to the sponsoring company than studies of the same agents that are funded by government or non-profit sources, even after adjusting for various measures of study quality.1–4 In addition, pharmaceutical sales representatives present doctors with information that is carefully chosen to demonstrate the benefits of the drugs that they are marketing; information that is often not representative of the larger literature on those drugs.5 6
Such conflicts of interest create challenges for physicians who are trying to interpret the scientific literature: they must find ways to judge the scientific merits of research while simultaneously taking account of how conflicts of interest have shaped the very same research. To meet this challenge, many medical journals and scientific conferences have established policies to inform people about conflicts of interest on the grounds that such information will help them discount the value of the conflicted party's message by an appropriate amount. Former Journal of the American Medical Association editor, George Lundberg, summed up this approach when he stated: ‘Our readers are not children. They're physicians, scientists, health policy experts, and medical reporters. They can figure this thing out so long as we give them the information.’7
Do physicians actually reduce the weight that they put on studies that are funded or presented by pharmaceutical companies when they make prescribing decisions based on that research? One social science experiment demonstrated that disclosing conflict of interest may not only fail to result in less biased judgements but can actually increase bias, because the act of disclosing conflict gives the conflicted parties a moral licence to exaggerate their claims.8 Two studies examine the value of disclosure in the context of medical journals, presenting British Medical Journal subscribers with a brief research report and a randomly assigned competing interest statement.9 10 They discovered that participants discounted the credibility of the research when it was authored by employees of a company that could benefit from the study, but did not discount when the authors were simply the recipients of grants from this company rather than its employees.
Whereas those previous studies examined whether doctors discount for the conflicts of interest of researchers, they did not look at how physicians consider evidence when it is presented to them by a conflicted party. The messenger, too, should be important in judging the credibility of research. For instance, when a pharmaceutical sales representative gives doctors reprints of a research report, the physicians must not only analyse the study before them, but also consider bias in the way that the study was selected for presentation from the universe of studies that have asked similar questions. We predict that this latter step will be neglected, because it requires drawing attention away from the more salient, immediately available data to consider what is not shown.
The current study aims to test whether physicians account for researcher and presenter conflict of interest when evaluating a clinical trial of a new drug and deciding whether to prescribe that drug. We present physicians with an abstract reporting the results of a hypothetical randomised, controlled trial of a fictitious new drug. Across physicians, we randomly varied the level of conflict of interest (study funding from a pharmaceutical company, study presented by a pharmaceutical company, or no conflict), the journal prestige (high or moderate), and the total number of studies published on this fictitious drug (five or one).
Methods
Subjects
In November 2006, we sent a 20-question survey to 515 obstetrician–gynaecologists who are members of the American College of Obstetricians and Gynecologists (ACOG) and who had agreed to complete four to five surveys throughout the year as part of ACOG's Collaborative Ambulatory Research Network (CARN). We randomly selected these physicians from the CARN database, which closely resembles the entire ACOG membership in distribution of gender, age, and geographical location. ACOG membership (N=53 328) comprises over 90% of board-certified, practising obstetrician–gynaecologists in the USA. We aimed to have 30 participants in each of the eight experimental groups to provide sufficient statistical power. Participation was voluntary and no compensation was offered.
Two subsequent mailings were sent to non-respondents in December and January 2007. Of the 515 surveys mailed out, 253 were completed and returned to us (49%). There was no significant difference in response rate among the experimental groups. Compared with non-respondents, respondents did not differ significantly on geographical location but were more likely to be male (57% vs 47%) and were older (mean age 49 vs 46 years). Gender and age were not associated with the level of conflict of interest or willingness to prescribe the new drug. The survey was approved by the Carnegie Mellon University Institutional Review Board.
Materials
Participants were told that they would be presented with a scenario and journal abstract of a study involving a fictitious new drug. The scenario began, ‘A new proton pump inhibitor, manaprazole, has been introduced recently by Pfizer’. It went on to describe how many randomised, controlled trials of this drug had been published (one vs five), the context in which the abstract was presented to the participant (by a Pfizer representative at a sponsored lunch vs at an academic grand rounds given by an independent physician without connections to Pfizer), and the study's funding source (Pfizer vs National Institutes of Health; NIH). In addition, a Pfizer logo was displayed at the top of the study abstract when the presenter was conflicted, whereas a University of Pittsburgh Department of Medicine Grand Rounds logo was displayed when the presenter had no conflict of interest. Above the abstract, the journal's banner was displayed (New England Journal of Medicine (high prestige) vs Alimentary Pharmacology and Therapeutics (moderate prestige)). The study's funding source also appeared below the abstract (labelled ‘CONFLICTS OF INTEREST’ in bold, all caps). The complete materials for one experimental group (moderate prestige journal/conflicted presenter/five studies) and more detailed information about the materials are presented in the appendix (online only).
Measures
Prescribing likelihood
The main outcome measure asked, ‘How likely would you be to prescribe the new drug, manaprazole, in place of omeprazole as a first-line therapy for patients like the ones in this study?’ Respondents were asked to assume that the two drugs did not differ in cost. Participants responded on an 11-point scale, with response options ranging from 0 (‘definitely would not prescribe’) to 10 (‘definitely would prescribe’), with a midpoint of 5 (‘completely unsure whether would prescribe’). The question stem repeated the major independent variables to ensure their salience (eg, ‘After reading the New England Journal abstract of this Pfizer-funded trial presented at Grand Rounds, how likely would you be to prescribe the new drug…’). Participants were also asked, using an 11-point scale, ‘how confident are you that manaprazole is superior to omeprazole, taking everything you have just read into account?’ Responses to this question tightly parallelled responses to the first question. Therefore, here we report only the results using the likelihood of prescribing.
Factors considered
Participants were then asked to report which of the following factors they took into consideration when deciding their willingness to prescribe the new drug (checking all that they had considered): proportion of patients healed on manaprazole relative to omeprazole; statistical significance of the difference between the proportion healed on manaprazole and omeprazole; sample size of the study; duration of the study; doses of the two drugs; severity of the common adverse events; prestige of the journal; number of other trials conducted on this new drug; the study's funding source; whether the authors were affiliated with Pfizer; whether the person presenting the study was affiliated with Pfizer.
Self-reported discounting
We asked participants how their likelihood of prescribing the new drug would change if only (a) the study had been presented by the other presenter—that is, the independent physician at grand rounds rather than the Pfizer representative, or vice versa; or (b) the study had been funded by the other funder—that is, NIH rather than Pfizer, or vice versa. For each of these three questions, we asked participants to imagine that only this one characteristic of the study had changed. Responses were on an 11-point scale, from −5 (‘far less likely to prescribe’) to 5 (‘far more likely to prescribe’), with a midpoint of 0 (‘neither more nor less likely to prescribe’). These measures indicate the extent to which participants think they should discount for presenter and researcher conflicts, respectively. When participants indicated that they would increase their prescribing likelihood if a conflict were removed or decrease their likelihood if a conflict were added, we classified them as reporting that they discount for this type of conflict.
Judged favourability relative to other studies
In the experimental groups in which five trials of the new drug had been published, we asked, ‘Relative to all the trials of manaprazole published so far, how favourable towards manaprazole do you think the results of this study are, given only the information with which you have been presented?’ Responses were on an 11-point scale, from −5 (‘this is the least favourable of all studies’) to 5 (‘this is the most favourable of all studies’), with a midpoint of 0 (‘this is in the middle’). Participants were also provided with a box to check if they felt that they had ‘no way of guessing how favourable towards manaprazole this study's results are relative to those of the other studies’.
Physician characteristics
We obtained primary practice setting and location, medical school graduation year and gender. We also asked participants how often they used pharmaceutical representatives when deciding whether to prescribe a new drug, given a five-point response scale ranging from ‘almost never’ to ‘almost always’ (table 1).
Additional survey questions were analysed in a separate report.12 The study was pre-tested among a group of physicians at the University of Pittsburgh to ensure the salience of the independent variables, realism of the materials and clarity of the questions.
Data analysis
We measure physicians' self-reported likelihood of prescribing the new drug after reading the scenario and abstract. Comparing this measure between the three randomly assigned conflict of interest groups reveals the extent to which participants discount for researcher and presenter conflicts. To measure the extent to which participants believe they should discount for researcher and presenter conflict, we ask physicians how their likelihood of prescribing the new drug would have been different when explicitly comparing conflict and no conflict scenarios. To examine whether participants take into account the likelihood that economically interested presenters will highlight an unrepresentative selection of publications, we compare physicians' self-reported likelihood of prescribing the new drug when the total number of studies published on this drug was five rather than one (when there is more than one study published, conflicted presenters can choose to present the one, ie, most favourable to the interests they represent).
Data were analysed using a personal computer-based version of SPSS 16.0. Descriptive and frequency statistics were computed. A paired t test was performed to compare the percentage of external versus internal factors considered when evaluating the trial. χ2 statistics were used to test associations between categorical variables. A general linear model of the likelihood of prescribing the new drug was performed, simultaneously entering the conflict of interest level and journal prestige, using the sample of participants in the five trial groups (groups 1–6 in appendix; all analyses of conflict of interest levels were restricted to these groups). Journal prestige was removed from this model when it was found to have no significant effect on prescribing likelihood. Using the sample of participants in the presenter conflict groups (groups 5–8 in appendix), a separate general linear model of the likelihood of prescribing the new drug was performed, entering the number of trials conducted as the independent variable. All tests were two-tailed, using α=0.05 to evaluate significance.
Sample characteristics (n=253)
Results
Sample characteristics are presented in table 1. The randomly assigned independent variables (journal prestige, level of conflict of interest and number of trials conducted) were not correlated with any of the measured participant characteristics.
Journal prestige did not have a significant influence on physicians' reported likelihood of prescribing the new drug (mean 6.3 when in the high prestige journal vs 6.7 in the moderate prestige journal), F(1, 247)=1.95, ns. Consequently, this variable was not included in subsequent analyses.
Taking conflict into consideration
After indicating their likelihood of prescribing the new drug, participants reported which of 11 listed factors they considered when deciding how likely they would be to prescribe the new drug. The results are displayed in figure 1. Physicians were more likely to report taking into consideration aspects of the evidence that were internal to the study (eg, the statistical significance of the main finding, the proportion of patients healed on the new drug compared with the established drug, the sample size) than those that were external to the study (eg, the presenter's affiliation with the drug company, the journal's prestige), 63% versus 41%, t(252)=11.38, p<0.001 (figure 1). Whereas 60% of participants reported considering researcher conflict of interest (research funding source or researcher affiliation with Pfizer), only 36% reported considering presenter conflict. Over a third of participants (37%) reported not considering any type of conflict of interest when deciding their likelihood of prescribing the new drug. This number did not vary significantly by randomly assigned conflict of interest level physician demographic characteristics. However, physicians who use pharmaceutical representatives ‘often’ or ‘almost always’ when deciding whether to prescribe a new drug were less likely than others to report considering conflict of interest (50% vs 69%), χ2(1, N=247)=7.56, p=0.006.
Percentage of physicians reporting consideration of internal and external aspects of the evidence when evaluating the new drug. Error bars represent 95% CI.
Discounting for conflict
Information on conflict of interest had no significant effect on prescribing likelihood; physicians receiving the no conflict scenario (mean 6.7, SD 2.2), presenter conflict scenario (mean 7.0, SD 1.9) and researcher conflict scenario (mean 6.2, SD 2.3) did not differ significantly in their reported likelihood of prescribing the new drug, F(2, 187)=2.1, ns (figure 2).
Likelihood of prescribing a new drug as first-line therapy, by level of conflict of interest. Error bars represent 95% CI. The likelihood of prescribing the new drug does not vary significantly across conflict levels.
None of the measured physician factors modified the extent to which participants discounted for conflict of interest. However, two groups did report a higher likelihood of prescribing the new drug, irrespective of conflict condition: those who are in private practice (mean 6.8 vs 5.7, F(1, 247)=13.45, p<0.001) and those who use pharmaceutical representatives ‘often’ or ‘almost always’ when deciding whether to prescribe new drugs (mean 7.5 vs 6.1, F(1, 243)=19.63, p<0.001).
Number of studies published
To test whether participants considered the possibility that conflicted presenters selectively choose to present the evidence that best serves their purposes, we compared physicians' likelihood of prescribing the new drug depending on whether the total number of studies published on this drug was five or one. (A pharmaceutical representative does not have the ability to present selectively the most positive study and conceal the negative studies if only one trial has been published.) Our findings revealed that when doctors were told that this study was one of five trials of this drug (but not told the results of the other studies), they reported being more likely to prescribe the drug (mean 7.0, SD 1.9) than physicians who were told that this was the sole trial (mean 6.1, SD 2.3), F(1, 119)=5.9, p=0.02. If physicians accounted for the conflicted presenter's selective presentation of studies, the opposite result would have been expected.
Judged favourability relative to other studies
When participants in five-trial groups were asked to estimate how favourable the results of this study were towards the new drug, relative to the other trials published so far, a minority of physicians judged this study to be more favourable than average, regardless of whether the researchers were conflicted (44%, 95% CI 36.9 to 51.1), the presenter was conflicted (45%, 95% CI 37.9 to 52.1), or neither was conflicted (34%, 95% CI 27.3 to 40.7). These proportions did not differ significantly from each other.
Self-reported discounting
We asked physicians how they believed their prescribing likelihood would have changed if conflict of interest had been removed from or added to the scenario. Most respondents indicated that adding a conflict would have made them less likely to prescribe the drug while removing a conflict would have made them more likely to prescribe it. More than two-thirds (69%, 95% CI 62.5 to 75.5) of participants reported that they would discount for researcher conflict, whereas 57% (95% CI 50 to 64) reported that they would discount for presenter conflict. These proportions did not vary significantly across experimental groups.
Discussion
In this study, we found that after reading the results of a hypothetical clinical trial of a new drug, doctors did not differ in their reported likelihood of prescribing the new drug based on the randomly assigned financial conflicts of the presenter or researchers. Yet, when scenarios with conflict and no conflict were directly compared, most participants reported that such conflicts would make them less likely to prescribe the drug. Therefore, although physicians believe that they should discount for conflict of interest, they do not do so when evaluating a single trial in isolation.
Our study is the first we know of to test whether physicians account for conflicts of interest among research presenters. Whereas researchers can directly affect the results of their research, presenters who are just messengers—distributing reprints of journal articles—might seem irrelevant to physicians. Yet conflicted presenters can potentially affect what information physicians see, by choosing to present studies that have results supporting their interests while omitting research with unfavourable results. To discount for this selective sampling, physicians must mentally remove themselves from analysing the study at hand and focus instead on how the study was sampled from the array of all similar studies that exist.
Our results are consistent with earlier research finding that medical journal readers, on average, judge a study to be no less valid or believable when the study's authors receive grants from a company that stands to benefit from the study compared with when the authors have no competing interests.10 Our study adds to this literature in several ways. Rather than limiting our sample to regular journal readers, we sampled a cross-section of obstetricians and gynaecologists. We examined whether physicians discount for conflicted presenters as well as conflicted researchers, and we compared physicians' anticipated discounting with their actual judgements.
If physicians believe that they discount the value of information from conflicted sources when making prescribing decisions, then how can we explain that they do not appear to be doing this? One possibility is that the conflict of interest manipulations were not sufficiently salient to catch the attention of the participants. This is unlikely for two reasons. First, the conflicts of interest are the most prominent information on the page: the presenter is highlighted with large graphics, and the study funding source is set apart with a bold, all caps ‘conflicts of interest’ heading. Second, in asking about the physician's likelihood of prescribing the new drug, we repeat the study funding source and presenter. Our format makes conflicts of interest far more salient than they are in actual journal articles, where disclosures are often relegated to small type at the end of the paper.
Another possibility is that physicians notice conflicts and believe they should account for them, but faced with complex information, and lacking sufficient knowledge to know how much they should discount for bias, they fail to do so at all. The data in figure 1 indicate that physicians are significantly more likely to report considering factors that are internal to the study, such as sample size and statistical significance, than external factors, such as conflict of interest. External factors may seem less immediately relevant and therefore may be easier to lose sight of in the process of integrating many elements to come to a judgement.
Furthermore, while it may be straightforward to discount the value of one piece of evidence relative to another when they are directly compared (eg, a study funded by a pharmaceutical company vs a study funded by the NIH), discounting the value of evidence presented in isolation is an inherently difficult task; there is no normative procedure defining how much to discount or how much to value the evidence in the first place.13 Furthermore, there is a substantial literature showing that people have difficulty ignoring information (such as a conflicted study) when making decisions, even if they know that this information would bias their judgement.14–16
In addition to the cognitive explanations for failing to discount conflicted evidence, there may be a motivational cause. When physicians acknowledge the biasing effects of conflict of interest while still maintaining relationships with pharmaceutical representatives, this belief and behaviour clash in the mind, resulting in cognitive dissonance.17 A study by Chimonas et al18 found that to resolve this dissonance, physicians used a variety of strategies, including trying not to think about the conflict, explaining how they could remain objective, and arguing that meetings with pharmaceutical representatives were educational. Such beliefs are not uncommon and seem to arise early in training. A large, multisite study of medical students found that 60% believed that industry-sponsored grand rounds were educationally helpful and also likely to be biased.19
The results of this study must be interpreted within the context of the design's limitations. The scenario described was a situation that the participants were asked to imagine rather than one that they experienced. This allowed us maximum consistency and control over the information presented in each experimental group, such that we could manipulate the desired variables while holding all else (eg, charisma of the presenter, information presented, emotional cues) constant. While physicians indicated that the scenarios were quite realistic in pre-tests, it is possible that people would judge the information differently after experiencing such scenarios than after imagining them.20 Another limitation is that we relied on the self-reported likelihood of prescribing rather than actual prescribing records. This was necessary because we wished to use a fictitious new drug (in a familiar drug class) so that the respondents would have no previous knowledge or preferences regarding the drug. Although participants had no reason to report their prescribing likelihood falsely, it is possible that they would incorrectly predict their behaviour or respond in a socially desirable way. Yet, given that people tend to remember messages but forget the sources of those messages over time (‘source amnesia’), it is unlikely that physicians would discount for conflicts of interest more at a later date when they are faced with a prescribing decision.15 21
These findings may not generalise to all physicians. While we randomly sampled from a nationally representative database of obstetricians and gynaecologists, half of the physicians we contacted did not return the survey. Responders and non-responders are similar in their geographical distribution, but responders are 3.5 years older than non-responders, on average, and correspondingly (in this obstetrician–gynaecologist population), more likely to be male. However, gender and age were not associated with the dependent or independent variables. Furthermore, our main concern is with detecting differences between experimental groups; random assignment should evenly distribute unmeasured variables among the groups.
This study indicates that even when conflicts of interest are disclosed repeatedly, in a highly salient manner, physicians do not discount the value of information from conflicted sources. These results bring into question the effectiveness of disclosing the funding sources of published studies as a primary strategy for dealing with conflict of interest. If physicians cannot or will not use this information, then the practice of disclosure may serve as only an illusory solution to the bias associated with industry-sponsored and presented research. This is a particularly pressing concern because up to three-quarters of trials published in the highest impact clinical journals receive industry funding,22 and these studies have been found to produce results favouring the funder by using many techniques that are not detected by the peer review process.23 24
Furthermore, it appears that most physicians do not consider that a series of similar studies can produce a range of results and that pharmaceutical representatives may be selectively presenting those studies that are the most favourable to the products they are promoting. These results point to the need for replacing the current reliance on pharmaceutical representatives for drug information with an alternative, such as giving all physicians ready access to summaries of evidence on drug safety and effectiveness compiled by unconflicted sources.
References
Supplementary materials
Web only data
Files in this Data Supplement:
Footnotes
Funding This study was supported in part by the Centers for Disease Control and Prevention and grant no R60 MC 05674 from the Maternal and Child Health Bureau, Health Resources and Services Administration, Department of Health and Human Services.
Competing interests None.
Ethics approval This study was conducted with the approval of the Carnegie Mellon University Institutional Review Board.
Provenance and peer review Not commissioned; externally peer reviewed.
Linked Articles
- The concise argument
Other content recommended for you
- Interactions of doctors with the pharmaceutical industry
- Legislative, educational, policy and other interventions targeting physicians’ interaction with pharmaceutical companies: a systematic review
- Interactions between physicians and the pharmaceutical industry generally and sales representatives specifically and their association with physicians’ attitudes and prescribing habits: a systematic review
- Medical students’ exposure to and attitudes towards product promotion and incentives from the pharmaceutical industry in 2019: a national cross-sectional study in France
- Perceptions of the provision of drug information, pharmaceutical detailing and engagement with non-personal promotion at a large physicians network: a mixed-methods study
- Ten years later: a review of the US 2009 institute of medicine report on conflicts of interest and solutions for further reform
- Conflict of interest among Italian medical oncologists: a national survey
- General practitioners and pharmaceutical sales representatives: quality improvement research
- Ethical issues raised by the introduction of payment for performance in France
- Biomedical conflicts of interest: a defence of the sequestration thesis—learning from the cases of Nancy Olivieri and David Healy