Article Text

PDF

Controversies
A case of insufficient evidence equipoise: the NICE guidance on antibiotic prophylaxis for the prevention of infective endocarditis
  1. R K Mohindra
  1. Correspondence to Dr R K Mohindra, 63 Newlands Road, High West Jesmond, Newcastle upon Tyne, NE2 3NT, UK; raj.mohindra1{at}googlemail.com

Abstract

This paper argues that the National Institute for Health and Clinical Excellence should not offer guidance in situations where there is insufficient evidence equipoise about the potential benefit of the treatment in question. This is broadly for two reasons. First, without knowing if the treatment is effective no cost-effectiveness judgement can be logically made. Second, the implementation of a population wide change in treatment where there is equipoise amounts to a de facto clinical trial that falls outside the Clinical Trials Regulations. As such there are strong ethical and possibly legal grounds for preventing such an outcome.

Guidance based upon insufficient evidence equipoise also impacts upon the clinical discretion possessed by individual medical professionals.

Statistics from Altmetric.com

Introduction

I was recently surprised to discover that Shaw and Conway1 had used Pascal's wager and examination of the ‘no-lose argument’2 in a specialist cardiology journal in order to support the current guidance from the National Institute of Health and Clinical Excellence (NICE) regarding the use of antibiotic prophylaxis for infective endocarditis (IE).3–6 The essential effect of this guidance is to withdraw the general use of antibiotic prophylaxis to prevent IE for patients undergoing dental procedures, upper and lower gastrointestinal investigations and urinary procedures. This is even for those traditionally perceived to be at high risk of IE. This was a radical departure from the pre-guidance prevailing practice.

The source of my surprise was the fact that both philosophical and theological argument had been brought to bear in order to persuade clinicians of the merits of this particular piece of NICE guidance. Guidance that should have been driven by the empirical data.7

The NICE guidance quotes an incidence of IE of 1/10 000 with an associated mortality of 20% together with significant associated morbidity. Antibiotic prophylaxis is itself associated with a mortality rate. Prendegast quotes a potential anaphylaxis related mortality of 1–3 per 100 000 uses of beta-lactam antibiotics.8

To gain some feel for the numbers let us use these figures. Assume the lower end of the antibiotic anaphylaxis mortality rate and ignore the morbidity of IE. From this we can calculate that if antibiotic prophylaxis saved one IE per 20 000 doses we would break even on mortality. Clearly this conclusion is only valid if the quoted figures accurately reflect the truth in practice.

If we do accept these figuresi the core issue becomes whether and to what extent the use of antibiotic prophylaxis actually prevents IE. In relation to this critical point NICE concludes that:

There is insufficient evidence to determine whether Or not antibiotic prophylaxis in those at risk of Developing infective endocarditis reduces the incidence of IE when given before a defined interventional procedure (both dental and non-dental)3

Definitive clinical evidence in the form of adequately powered clinical trial data is lacking. In a recent Cochrane review aimed at answering this question no randomised controlled trials, other controlled trials (quasi-randomised or historically controlled) nor any cohort studies were identified.9 Numbers can be gleaned from the available studies but the connection between such numbers and the true effect of antibiotic prophylaxis on the incidence of IE is open to debate.10 Essentially a fact neither formally proved nor formally disproved lies at the heart of the NICE guidance.11

When balancing the potential risks against the potential benefits of antibiotic prophylaxis against IE there is a requirement to consider the mortality risk associated with antibiotic prophylaxis itself. However this consideration does not logically absolve us from the need to adequately characterise the potential for antibiotic prophylaxis to reduce the incidence of IE. To cover the fact that this empirical fact is not adequately characterised by the available evidence, NICE constructs a circumstantial argument using the following observations:

  1. There is a lack of a demonstrated causal connection between interventional procedures and IE;

  2. Regular tooth brushing results in recurrent bacteraemia with oral flora3;

  3. Engagingly, it also argues that the clinical effectiveness of antibiotic prophylaxis is not proven. This is not a useful argument because it imports circularity and, in the words of that well worn phrase, ‘absence of evidence does not imply evidence of absence’.12

While it might be possible to accept such an argument, there exists a rationally defensible counter-position that we can call insufficient evidence equipoise.

Insufficient evidence equipoise versus competing evidence equipoise

Equipoise is a state of mind where the degree of justifiable uncertainty about the truth value of a particular proposition is balanced to some extent.13 14 Halpern has credibly argued that, in medical matters, the question of whether equipoise exists or not should rest upon the available high quality evidence base, not upon expert consensus opinion.15

While there is a great deal of literature about who should be uncertain, the responsible physician,16 17 the medical community18 or even the patient19 20, this is not the concern here. This reason for this is that we are not asking who should believe what or even what we can justifiably do to resolve the uncertainty.21 What we are asking how can a cost-effectiveness judgement flow from this uncertainty?

From table 1 we can see that we can be in equipoise for two reasons in this context:

  1. There is evidence that carries adequate probative force both for and against the proposition (competing evidence equipoise); or

  2. There is insufficient evidence to support the determination of either the truth or falsehood of the proposition (insufficient evidence equipoise).

Table 1

Distinguishing competing evidence equipoise from insufficient evidence equipoise

In many situations this distinction does not matter. However, here it is of critical importance because NICE has made a value judgement purportedly on the basis of a cost-effectiveness judgement.

Where there is competing evidence equipoise a choice can be made between the competing sets of evidence. Clearly this is a value judgement. Once this choice is made, a valid chain of reasoning can arrive at an objectively defensible effectiveness judgement predicated upon the evidence set chosen. This is because the chosen evidence set is necessarily of sufficient quality to establish the efficacy or inefficacy of the proposed intervention to the standard of proof required in medical practice.

Challenges to such decisions lie towards the value judgement made in selecting one evidence set over the other, rather than to the validity of the proposition that the chosen evidence set fails to adequately demonstrate the efficacy or inefficacy of the proposed intervention.

NICE judgement: cost-effectiveness and insufficient evidence equipoise

If the situation is one of insufficient evidence equipoise then the value judgement selecting one evidence set over the other must still be made. However once this selection is made and defended we find that the evidence set chosen must necessarily fail to establish the efficacy or inefficacy of the proposed intervention to the requisite standard of proof. This means that no effectiveness judgement can be validly made about the intervention in question. This does not preclude the use of the available data to develop cost effectiveness models23 but the validity of any conclusions drawn from such models is open to challenge.

In relation to this point, the NICE guidance admits the following:

The Guideline Development Group (GDG) used the decision making and conclusions of relevant national and international guidelines to help inform its own decision making. This decision-making process has been important because, for many of the key clinical questions covered in this guideline, there is no evidence base that would meet rigorous quality criteria.3

As we have seen above one such key clinical question was the empirical fact in issue here. While NICE may argue to support its value judgement that antibiotic prophylaxis should not be widely given24 it can only guess (albeit expertly) about the effectiveness or not of antibiotic prophylaxis at reducing the incidence of IE.

Two points emerge from this position. First in the absence of adequate empirical evidence the use of other expert opinion as a foundation for new expert opinion based guidance does not move the evidential ball at all. Second, and critically, NICE cannot logically reach a conclusion upon cost-effectiveness in the absence of adequate evidence of effectiveness or ineffectiveness.25 This is one powerful indictment of this guidance.

The generalisation of this last point is that, where there is insufficient evidence equipoise in relation to the clinical evidence base, cost-effectiveness judgements should not be made. Indeed in its own guidance upon social value judgements NICE states that:

Principle 1: NICE should not recommend an intervention (that is, a treatment, procedure, action or programme) if there is no evidence, or not enough evidence, on which to make a clear decision. But NICE's advisory bodies may recommend the use of the intervention within a research programme if this will provide more information about its effectiveness, safety or cost.26 27

While we may make sophist arguments about the difference between an act and an omission,28 based upon the arguments made here, the correct answer to insufficient evidence equipoise would seem to be adequate research.

A NICE trial

The advice in NICE's guidance upon social value judgements has an additional and deeper purpose. To see this we need to realise that the effect of the NICE guidance is to institute a de facto population wide prospective cohort study to test the null hypothesis that antibiotic prophylaxis does not reduce the population wide incidence of IE.

Prior guidance had issued antibiotic prophylaxis to broad population groups. The new guidance withholds antibiotic prophylaxis from virtually all these patient groups. By comparing the incidence of IE in the national population before and after the guidance was issued it should be possible to draw conclusions about the efficacy of antibiotic prophylaxis for preventing IE. Therefore by issuing and implementing national guidance NICE is effectively conducting a de facto clinical trial under the cover of a cost-effectiveness judgement.

The effect of issuing guidance on a cost-effectiveness basis is to negate any intention to undertake a clinical trial thereby side-stepping the legal framework for research.ii NICE is not a body constituted to undertake research protocols. It is a creature of statute created on the 26 February 1999 by order of the Secretary of State for Health.29 The functions of NICE are set out in directions from the Secretary of State for Health. Its core role is to operate ‘in connection with the promotion of clinical excellence and the effective use of available resources in the health service’.30 Therefore another core objection to the guidance in the context of insufficient evidence equipoise is that as a rationing body NICE should not be permitted to implement clinical trials outside the regulatory regime.

The burden of proving the empirical fact generating insufficient evidence equipoise should rest upon NICE because: (1) NICE is seeking to issue guidance that implements a change amounting to a de facto trial protocol; (2) NICE is not constituted to design or implement research; and (3) this change is not subject to oversight by the regulatory bodies charged to protect research subjects. Note how this burden of proof is more readily discharged in situations where there is conflicting evidence equipoise.

The pragmatic general principle should be that NICE should not exercise the power of the fee payer to intervene and alter the status quo where the underlying empirical evidence base is in insufficient evidence equipoise.

Expert guidance in the face of insufficient evidence clinical equipoise

Here we must first note how NICE seeks to enhance its power to implement cost-effectiveness decisions by donning the cloak of issuing expert consensus guidance. If there is truly insufficient evidence clinical equipoise then outside the context of a formal clinical trial protocol the decision whether or not to treat should fall to be discussed between the responsible clinician and the patient.31 This view is supported by the preamble of the NICE guidance itself:

This guidance represents the view of the Institute, which was arrived at after careful consideration of the evidence available. Healthcare professionals are expected to take it fully into account when exercising their clinical judgement. The guidance does not, however, override the individual responsibility of healthcare professionals to make decisions appropriate to the circumstances of the individual patient, in consultation with the patient…3

The principle is that, in the context of insufficient evidence clinical equipoise, a decision about a general patient cannot determine the decision for a particular patient. The place for expert consensus guidance here is illumination not determination. The expert consensus dimension of the NICE guidance can only validly operate through the clinical discretion of the responsible clinician in this context. The argument that the fee payer will not pay for the treatment should be kept separate from the question of whether or not the treatment might benefit this particular patient.

NICE's declared view of the evidence base cannot enhance the probative force of this evidence base merely by virtue of its perception.32 Therefore NICE must rely upon its power as the market dominant fee payer and monopsony employer to implement its guidance uniformly on the ground in this context.33

This exercise of power rather than logic to ensure implementation of the guidance means that there is a shadow cast upon clinical decision-making through the value judgements made by NICE. These shadows impact particularly upon the scope and exercise of clinical discretion. Similar shadows are found elsewhere too, for example, in relation to top up fees.34 These distortions are another important, albeit less tangible, ground for challenging NICE guidance made in this context.

Conclusion

Cost-effectiveness judgements should only be grounded in an evidence base that carries sufficient probative force to reach a justified conclusion about the proposition in question. Cost-effectiveness judgements lack purchase where there is insufficient evidence equipoise in relation to the relevant evidence base and should therefore be avoided. In this context one danger of implementing value judgements, even those based upon expert opinion, is that there is a potential for unregulated clinical trial protocols to emerge. Another danger is that such value judgements can distort the clinical discretion that tends to operate to the overall benefit of the particular patient.

The conclusion reached by NICE may or may not be correct. The argument here is not about how to interpret the available evidence. The objection is that the fact of insufficient evidence equipoise places the right to reach a cost-effectiveness decision outside the jurisdictional reach of NICE. For NICE to validly make value judgements in the context of inadequate evidence equipoise it must remove the façade of evidence-based expert consensus guidance and openly embrace the ethical debate qua ethical debate. It must also have its remit extended.

Should NICE fear legal challenge by way of judicial review? Given its own guidance upon social value judgements this is not out of the question provided a suitably interested litigant can be found. The claim here would be of unreasonableness because of the conflict of the NICE IE guidance with its own guidance on social value judgements.35 Another base for such an action might be a claim of ultra vires. Arguments here include:

  1. the fact that there was insufficient evidence upon which to make a cost-effectiveness judgement36 37; and

  2. that NICE has made an decision that lies outside its jurisdiction because the decision is one that implements a de facto clinical trial protocol outside the pre-existing legal regulatory framework.38 39

Any successful action would force NICE to contemplate the logical limits of its powers. Such arguments would carry force because of the existence of an implementation mechanism for NICE guidance within the NHS. The best remedy here might be a declaration.40

Further, and perhaps even more interestingly, Connaughton has suggested that NICE should consider offering compensation for any individual who can demonstrate that they have suffered IE as a result of the change of policy implemented by the NICE guidance.41 This view has moral force.

The final conclusion must be that whatever sense of disquiet is felt about the present situation, whatever belief is held about the use of antibiotic prophylaxis for IE, doctors in general should regard themselves as ethically bound to organise the data collection mandated by this externally imposed implicit trial protocol unless NICE or some other organisation is formally obliged to perform this important task.42

Acknowledgments

I would like to acknowledge the comments of two unnamed reviewers whose incisive and helpful comments contributed greatly to improve this article.

References

View Abstract

Footnotes

  • Competing interests None.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • i There is broader concern about a general increase in antibiotic resistance with the widespread use of antibiotic prophylaxis to prevent IE. But this effect is difficult to quantify so we will not consider it further here.

  • ii Note that Regulation 2 of The Medicines for Human Use (Clinical Trials) Regulations 2004 SI 2004/1031 requires an intention to study for the definition of a clinical trial to be met. This echoes Article 2 of Directive 2001/20/EC.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.