CREATING THE EVIDENCE FOR EBM

The evidence for EBM is created from scientific research. To date the majority of this research relates to testing the effectiveness of treatment interventions on research populations in clinical controlled trials. The first question we should ask is whether and to what extent disadvantaged groups participate in the production of research evidence. The production of research includes commissioning, research design, and participation in clinical trials.

Research commissioning and design

Historically, research commissioning and design has been the domain of scientists together with funders of research, either private (such as pharmaceutical companies) or public (such as government funded bodies). Almost by definition, the disadvantaged are not well represented among these groups. Information in this area is scarce. However, one report of a UK survey investigating consumer involvement in designing, conducting, and interpreting RCTs found that one third of specialised trial centres had involved consumers in some part of the process.¹ Consumers were defined as “patients and potential patients, carers, organisations representing consumers’ interest, members of the public who are targets of health promotion programmes, and groups asking for research because they believe they have been exposed to potentially harmful circumstances, products, or services” (Hanley B, et al,¹ p 520). The commonest form of consumer involvement was drafting or reviewing information for trial participants, followed by promoting recruitment, and membership of the steering committee. This work by consumers was reported in generally positive terms by survey respondents (who were researchers), who felt that consumer contributions helped to ensure that trials addressed questions relevant to consumers. The survey asked for information about the background of consumers who had been involved, but this information was not reported in the published paper. Without this information it is difficult to reach any conclusions about the presence of people from disadvantaged groups, but even without this, the type of consumer involvement reported indicates that their role is largely limited to helping the research run smoothly once the really important decisions have been finalised. There seems little opportunity in this system for substantial input—for example, in determining research topics, or interventions to be tested, or outcomes to be measured, from any consumers, let alone from people whose voices are more generally unheard.² A US review of participation of minorities in cancer research reported various barriers such as study duration, cost, time, follow up visits, and side effects, as well as cultural characteristics, and attitudes, beliefs, and knowledge about research.³ It is reasonable to think that these barriers also operate as much if not more so to exclude disadvantaged groups from participation in research commissioning or design.

Participation in research

Discussions about the kinds of populations who are generally included as trial participants have been current for some time. The gender imbalance is well known and well reported,^4,5 yet despite this, there continue to be important discrepancies in the number of women participating in—for example—cardiovascular research compared with men.^6–8 Gender is easy to define and record as a variable in trial data. Other potential markers for disadvantage are less so, but there is enough recorded information to suggest that people from ethnic minorities and from low socioeconomic groups are generally under represented. Much of the information about the research participation rates of ethnic minorities comes from the US, where study after study reports either under representation or lack of data about ethnicity. Participation rates for minority populations and the medically underserved in adult cancer treatment and prevention trials are lower than those of white non-Hispanics. The most represented group in these trials are white, middle class, highly educated men.³ Breast cancer rates in women vary with race and ethnicity; in the US white non-Hispanic women have the highest rates, closely followed by Hawaiian, African-American, Japanese, and Alaska Native.⁹ A review of trials of selective oestrogen receptor modulators, drugs that may be useful both in the prevention and treatment of breast cancer, found, however, limited ethnic variability amongst the 50 000 women who had participated in the trials, compared with the ethnicity and racial make up of the population of women who need treatment for breast cancer.¹⁰

These findings are similar to those of Heiat et al, who compared the characteristics of patients with heart failure in RCTs with those of patients with heart failure in the community, from 1985 to 1999.¹¹ They identified 59 RCTs and found that the participants in the trials were markedly different from patients in the community. In particular, trial patients with heart failure were younger, more often male, more likely to have a subnormal systolic ejection fraction, and were most commonly white. This represents only a relatively small segment of the heart failure population, with significant under representation of minorities, women, and the elderly. Perhaps most concerning, the authors did not find any marked change in the characteristics of patients in trials over time: RCTs of the 1990s continued to focus on young, white, male patients. The predominance of men of primarily European descent as research participants was also found in a review of research into the relationship between physical activity and cardiovascular disease.¹²

One study looked at sociodemographic markers in recruitment to trials run by the National Cancer Institute over a 12 month period. Sateren et al found that patients enrolled into clinical trials were significantly less likely to be uninsured and more likely to have Medicare health insurance than the patients with cancer in the community, and that geographical areas with higher socioeconomic levels had higher levels of clinical trial recruitment.¹³

For children, the situation seems to be different, as minority children with cancer have been found to be proportionately represented in clinical trials of cancer treatments.^13,14

The absence of data about ethnicity is also striking. Swanson and Bailar assessed the heterogeneity of subgroups in cancer treatment and prevention trials published in 11 journals between 1990 and 2000. They found that age and gender were reported in over 90% of the 261 published trials, but that less than 30% reported race or ethnicity.¹⁵ This lack of reporting precludes the possibility of any subgroup analysis to identify significant differences in racial or ethnic subgroups. Such subgroup analyses present their own challenges. However, appropriately rigorous analyses may be possible with some innovative application of accepted trial techniques.¹⁶

What are the implications of the absence of disadvantaged groups in trials, or the lack of identifiable data? First, there is the general observation that people in trials often fare better than people who receive treatment outside a trial. Lack of participation in trials effectively removes this benefit from disadvantaged groups. More importantly however, the lack of participation means there is a paucity of research evidence about which interventions are effective in disadvantaged groups. The generalisability of the findings is limited to people who are sufficiently similar, in relevant respects, to the trial participants. This begs the question as to which respects are relevant; physical, cultural, and structural issues may all be relevant in different ways.

There are two main ways that physical differences may be important in working out whether trial results are applicable to different populations. The first is to do with the presence or absence of comorbidities. Most RCTs exclude people with more than one disease, because the trial aims to find out specific information about the effect of a single intervention on a single disease state. Randomised controlled trials derive proof of effectiveness through strictly controlling as many variables as possible, so that any differences between the intervention and control groups may be attributed to the intervention rather than some other factor. The presence of multiple diseases and their various treatments would weaken this process, hence the exclusion of people with comorbid conditions. Yet comorbidity is an ever present fact of life in groups with socioeconomic deprivation, which means that many EBM derived guides are inapplicable to their care.¹⁷

Secondly, there is reason to believe there are some racial or ethnic and gender variations in responses to drug treatments.¹⁸ These variations, due to differences in the metabolism of some drugs, result in variable circulating concentrations of active drug, so that the same doses of particular drugs given to people of different races can have variable effects. The extent and nature of these potentially significant differences have not been fully investigated, but there are several classes of drugs known have such effects, including cardiovascular, psychotropic, and central nervous system drugs.¹⁹ The consequences of these differences are also not fully understood, but known examples include the increased sensitivity of Asian Americans to beta blockers (used to control high blood pressure), and the decreased effectiveness of ACE inhibitors in African Americans.¹⁹ Evelyn et al reviewed trial protocols and product labelling for 185 drugs approved by the Center for Drug Evaluation and Research between 1995–1999. Of these 8% (15/185) described differences related to race, but only one product label recommended a change in dosage based on racial differences.²⁰

It should be relatively easy to create the evidence about correct doses of various drugs where these are relevant for different ethnic groups and for women. More difficult is the task of unravelling cultural and structural issues to do with accepting health care, but this is a crucial part of generating good evidence. We need to know not only that an intervention works in ideal trial circumstances with a well defined population, but also that it works in the context of routine care, with heterogeneous populations. A drug may be very effective at controlling pain, but if it is culturally unacceptable to admit suffering from pain, the drug will not work for that person because they will not be able to articulate the need for it. Giuliano et al have explored a number of cultural barriers that impact upon screening participation in minority groups, and these factors may well affect the expected effectiveness of other health care interventions. Distrust of white dominated institutions is a key factor—for example, in discouraging Native American participation—as is a fatalistic approach to ill health.³ Fatalism is mentioned as a significant factor for several minority groups, but it has been suggested that fatalism may be less an intrinsic cultural belief than a protective attitude that allows people who have little power to change their material circumstances, to maintain some self respect.²¹

Some of these problems could be overcome by performing new research which targeted disadvantaged groups, in order to generate evidence about the efficacy of interventions in the trial situation, and also about effectiveness when interventions are delivered as part of normal care in the community. This is unlikely to occur, however, as research that repeats testing of an existing intervention is far less attractive, both to funders and to researchers, than research involving new interventions. Such trials are likely to be small rather than the megatrials currently in vogue, and this raises its own problems. Small trials are methodologically challenging: the smaller a trial, the larger the treatment effect necessary for the results to be significant, so that it is easy to miss small effects that may be clinically, but not statistically, significant.²² Trials that show no statistically significant benefit are less likely to be published, and so less likely to make their way into systematic reviews and thus into the accepted evidence base. Other reasons why small studies that target disadvantaged populations are unlikely to be performed concern the funding of research and the comparative nature of many studies. Pharmaceutical companies, who are major funders of research worldwide, are interested in products that will find a market, irrespective of the causes and distribution of ill health within populations. There is no incentive for these companies to fund small studies with disadvantaged groups because, even if the research identifies effective drug treatments, the market will be small and therefore unlikely to be profitable in the long run. On the other hand, pharmaceutical companies are interested in developing newer and more effective versions of existing successful drugs, which means comparative trials are an important part of their research and development programmes. Comparing new treatments with existing ones leads to clustering of research around a narrow range of interventions in research accessible populations, and again this provides no incentive to perform research with new populations.

In summary, disadvantaged groups rarely have a voice in commissioning and designing research, and have limited participation in trials of new interventions. This results in a lack of research evidence about effective interventions for this group. As the presence of evidence of effectiveness is increasingly a requirement for the provision of health care, this is a serious matter. Health care funders, both government and private, are accountable for their spending; funding interventions that are of proven effectiveness is seen as part of this. Apart from the intuitive attractiveness of funding interventions that work, no health care provider wants to be accused of wasting resources on interventions that do not work. For disadvantaged groups, this can be a vicious circle; to receive the newest and best treatments requires evidence, but exclusion from research prevents the generation of evidence and hence access to treatments. The barriers to performing research with disadvantaged groups are considerable, but unless they are overcome, the disadvantaged will remain disenfranchised from the goods of EBM. The power of EBM to mandate treatment does not function if there is no evidence.

APPLICATIONS OF EBM

Clinical care

For the reasons outlined above, there can be a lack of evidence about effectiveness for interventions in disadvantaged groups. Where there is applicable evidence, however, we might think this could be used to ensure fair distribution of evidence based interventions for all those who need them. The presence of evidence should ensure access for all, especially in situations where the delivery of care has been variable. In some cases, this has occurred; evidence based guidelines have been used to improve treatment rates in disadvantaged groups. One study found—for example—that before the introduction of evidence based guidelines, African Americans had a 60% greater likelihood of receiving inadequate haemodialysis compared with whites. In the period after the guidelines were introduced, there were dramatic improvements for African American patients, with a 92% increase in the proportion receiving adequate haemodialysis.²³ The use of evidence based guidelines helped to ensure equal treatment for all those needing dialysis, irrespective of race.

This is encouraging, however other areas of health care are not doing so well. Improved treatment of cardiovascular disease has been one of the flagships of EBM, especially the use of thrombolytics (“clot busting” drugs) in the treatment of acute myocardial infarctions. Despite widespread acceptance of the efficacy of thrombolytic drugs, it is clear that not all those who would benefit do receive these drugs. A review of 26 575 Medicare beneficiaries in the US found that, despite meeting all of the eligibility criteria, and after adjusting for differences in clinical and demographic characteristics and clinical presentation, African Americans with heart attacks were significantly less likely than whites to receive treatment.²⁴ The situation is not so different in Europe; a review of 4035 patients with acute myocardial infarctions identified a group of eligible patients who did not receive thrombolytic treatment. Statistical analysis of this untreated group found that women and the elderly were less likely to receive thrombolytic treatment compared with younger and male patients.²⁵ The tendency to undertreat heart disease in the elderly extends beyond use of thrombolytics; a Canadian review found that elderly cardiac patients receive consistently fewer prescriptions of proven cardiac therapies, including aspirin and statins, despite the fact that the effectiveness of these therapies is not affected by age.²⁶ Given the higher mortality risks from heart disease for the elderly, this is the opposite of what we might expect.

These studies show us that evidence about efficacy has not succeeded in changing patterns of treatment. In particular, disadvantaged groups are less likely than others to receive at least some efficacious treatments. These studies do not explain why this happens; we are left speculating about various reasons. Under representation in research has been suggested as a possible reason.^7,25 Clinicians might be aware of a lack of research evidence for some groups, and mistake the absence of evidence of efficacy as evidence of no efficacy—that is, they may confuse lack of proof about effectiveness with proof of ineffectiveness. Or perhaps the best evidence in the world is not enough to overcome deeply entrenched patterns of discrimination. Either way, the idea that EBM will lead to fairer treatment is challenged.

The examples discussed here concern variable access to proven treatments in eligible patients. There is less information about withholding treatments because patients do not fit the profiles of the research populations. The presence of other illnesses may have unknown effects on proven therapies, so that clinicians do not know whether evidence based therapies for one condition will work in patients with multiple illnesses. There does not seem to be a way out of this circle, for as we have already seen, the presence of other illnesses, which is more likely in disadvantaged populations, precludes participation in research and hence the generation of applicable evidence.

EBM and distribution of health care

The techniques of EBM are increasingly used to inform purchasing decisions about health care at regional or national levels, as well as informing decisions about health care for individual patients. In the UK, the National Institute for Clinical Excellence (NICE) is the body charged with assessing evidence about efficacy as part of making national recommendations about the introduction of new interventions. This was described by the then Health Secretary, Frank Dobson, in the following terms: “NICE is crucial to our plans for fair and equal treatment of patients within a truly national health service. Internal markets and postcodes were never an acceptable way to run a health service. NICE guidance will provide a common currency of effectiveness for the NHS, to inform and assist decision making about treatment and care at all levels, national, local, and individual”. The Secretary of State’s speech, made in 1999, was published on the NICE website.²⁷

As Dobson mentioned, part of the rationale for NICE was to end uneven access to various treatments across the UK, known as postcode rationing. This was widely perceived to be unfair, as people living in some areas were able to receive treatments that were unavailable in other areas. Health authorities are now obliged to fund interventions that are approved by NICE, guaranteeing nationwide access to approved interventions. Since 1999 these have included new drugs for the treatment of various cancers, obesity, Alzheimer’s, motor neurone disease, diabetes, attention deficit hyperactivity disorder, flu, and others.²⁸

There are now concerns that the aim of equitable distribution is likely to fail, as recommendations from NICE skew health care priorities and as health authorities make cuts in other areas to pay for interventions mandated by NICE.²⁹ The NICE approved interventions have to be provided by health authorities, but NICE guidance is not accompanied by dedicated funds, so that individual local authorities have to find the funds to pay for these from existing sources.³⁰ This means either cutting existing services, or using any new monies to implement NICE guidance. Each health authority makes its own decisions, so that as one set of interventions become universally provided because of NICE, others become unavailable, as they are cut to pay for the approved ones. This returns us to the situation of postcode rationing, as each authority responds to its own priorities and withdraws funding from different services.

The overall effects of this on patterns of expenditure and service provision are unknown, and to some extent unknowable: it is just not possible to know what else would have been funded if the money had not been spent on NICE approved treatments. The presence of evidence gives some interventions a spurious advantage over other interventions that are discounted because of a lack of evidence. Given the predominance of pharmaceutical interventions among those reviewed by NICE, we may be concerned about the long term extent of this pharmaceutical creep; and given the way that disadvantaged groups are under represented both in research and as recipients of evidence based care, we should be very concerned about the impact of evidence based purchasing on health care for the disadvantaged. There is the risk that services to disadvantaged groups that lack an evidence base are being sacrificed to pay for new, evidence based interventions that do not apply to the disadvantaged.

EBM AND FAIR HEALTH CARE

To be fair or just, we might think that a health care system should take into account various factors, such as need, benefit, equity (of access, of opportunities, of outcomes), or personal preferences. It is almost impossible to rank these; probably the best we can do is try to take account of all of them to some extent. Evidence based medicine addresses only one of these factors: capacity to benefit. A treatment may be effective at preventing something as important as coronary heart disease, or as trivial as premature greying of the hair. There is no logical relationship between proof of effectiveness and the urgency or importance of the condition for which the intervention is effective. Once the capacity to benefit is proved, however, this diverts attention away from more important questions such as whether this is an important health problem, or whether this should be provided, given the impact on other aspects of health care. Rather than starting with a set of priorities, and then performing the research to find out how best to achieve the agreed ends, the system is inverted by EBM. The research is performed, often for largely commercial reasons, and then the presence of this evidence is taken as some kind of imperative. Of course institutions such as NICE have criteria other than proof of effectiveness that enter their considerations, but hard evidence is very seductive, especially when used by lobby groups to argue their cause. Politically it can be very difficult to refuse people access to treatments that are evidence based, even if there are robust justice related reasons for doing so.

If we take into account the social determinants of health, this primacy accorded to capacity to benefit seems particularly invidious, as it does little to address inequalities in health. Much ill health is the result of disadvantage; although the exact mechanisms are not well understood, we can assume they are more to do with the material circumstances of people’s lives than individualistic factors. Yet EBM turns our attention away from social and cultural factors that influence health, and focuses on a narrow biomedical model of health and disease that is primarily individualistic. Instead of looking at ways to prevent ill health and ameliorate disadvantage, we are directed towards a system of health care that is very good at delivering highly sophisticated, and often expensive, individual treatments to those who are able to access them. This leaves those with the greatest burden of ill health disenfranchised, as there is little relevant research, poor access to treatments, and attention is diverted away from activities that might have a much greater impact on health.

WHO has recently published The world health report 2002, which identifies 10 major preventable risks that account for 40% of annual deaths worldwide. The list includes childhood and maternal underweight, unsafe water, and sanitation and hygiene. As we might expect, the greatest burden of health risks is borne by the poor countries, and by the disadvantaged in all societies. A recurrent theme in the report is the need for evidence based interventions. Possible interventions are listed for each risk; interventions for underweight include micronutrient supplementation and fortification.³¹ This evidence based advice almost beggars belief: if people are underweight and undernourished, surely they need food, rather than micronutrient supplementation? We are left with the uneasy suspicion that as there are no RCTs to prove that food is an effective method of reducing malnutrition, such a commonsense measure lies outside the evidence base and is therefore excluded from consideration.

CONCLUSION

So far my criticisms of EBM have been of its current processes and applications, but we still need to ask whether EBM is intrinsically inimical to the interests of disadvantaged groups, or whether it is just that its potential is not being realised. This is a difficult question.

Of course there is a place for evidence in health care: we need to know what kinds of interventions improve health outcomes, and which ones do not. And there is no necessary reason why appropriate and applicable evidence (broadly interpreted) cannot be gathered about the effectiveness of interventions for disadvantaged groups. This research will most likely be difficult, expensive, and have little commercial potential, but these are not morally valid reasons for not performing such research. Once performed, the results of this research should inform health care policies and expenditure, and lead to real improvements for the disadvantaged. Understood like this, EBM is a potentially valuable tool.

The current research climate does not, however, inspire optimism about this possible change in the direction of EBM. Medical research reflects the priorities of the rich, with 90% of research funding investigating the diseases of 10% of the world’s population. This is according to The 10/90 Report on Health Research published in Geneva, in 2002, by the Global Forum for Health Research, and cited by R Horton in his Lancet article the following year.³² This research agenda is shaped by a range of forces, but commercial forces have become dominant, so that research into interventions that are unpatentable is less and less likely.³³ Evidence based medicine has become a tool for commercial ends, narrowing the range of possible interventions and seeking ever smaller benefits with ever more expensive drugs.

It is possible that EBM could serve the interests of the disadvantaged, but this will only happen with a commitment to justice in health care at the highest possible levels, funded accordingly. Without such commitment, EBM will continue to foster an individualistic treatment oriented approach to health care, using a system that largely excludes the vulnerable and disadvantaged, and may in fact increase inequalities.