Insufficient evidence equipoise versus competing evidence equipoise

Equipoise is a state of mind where the degree of justifiable uncertainty about the truth value of a particular proposition is balanced to some extent.13 14 Halpern has credibly argued that, in medical matters, the question of whether equipoise exists or not should rest upon the available high quality evidence base, not upon expert consensus opinion.15

While there is a great deal of literature about who should be uncertain, the responsible physician,16 17 the medical community18 or even the patient19 20, this is not the concern here. This reason for this is that we are not asking who should believe what or even what we can justifiably do to resolve the uncertainty.21 What we are asking how can a cost-effectiveness judgement flow from this uncertainty?

From table 1 we can see that we can be in equipoise for two reasons in this context:

1. There is evidence that carries adequate probative force both for and against the proposition (competing evidence equipoise); or

2. There is insufficient evidence to support the determination of either the truth or falsehood of the proposition (insufficient evidence equipoise).

In many situations this distinction does not matter. However, here it is of critical importance because NICE has made a value judgement purportedly on the basis of a cost-effectiveness judgement.

Where there is competing evidence equipoise a choice can be made between the competing sets of evidence. Clearly this is a value judgement. Once this choice is made, a valid chain of reasoning can arrive at an objectively defensible effectiveness judgement predicated upon the evidence set chosen. This is because the chosen evidence set is necessarily of sufficient quality to establish the efficacy or inefficacy of the proposed intervention to the standard of proof required in medical practice.

Challenges to such decisions lie towards the value judgement made in selecting one evidence set over the other, rather than to the validity of the proposition that the chosen evidence set fails to adequately demonstrate the efficacy or inefficacy of the proposed intervention.

NICE judgement: cost-effectiveness and insufficient evidence equipoise

If the situation is one of insufficient evidence equipoise then the value judgement selecting one evidence set over the other must still be made. However once this selection is made and defended we find that the evidence set chosen must necessarily fail to establish the efficacy or inefficacy of the proposed intervention to the requisite standard of proof. This means that no effectiveness judgement can be validly made about the intervention in question. This does not preclude the use of the available data to develop cost effectiveness models23 but the validity of any conclusions drawn from such models is open to challenge.

In relation to this point, the NICE guidance admits the following:

The Guideline Development Group (GDG) used the decision making and conclusions of relevant national and international guidelines to help inform its own decision making. This decision-making process has been important because, for many of the key clinical questions covered in this guideline, there is no evidence base that would meet rigorous quality criteria.3

As we have seen above one such key clinical question was the empirical fact in issue here. While NICE may argue to support its value judgement that antibiotic prophylaxis should not be widely given24 it can only guess (albeit expertly) about the effectiveness or not of antibiotic prophylaxis at reducing the incidence of IE.

Two points emerge from this position. First in the absence of adequate empirical evidence the use of other expert opinion as a foundation for new expert opinion based guidance does not move the evidential ball at all. Second, and critically, NICE cannot logically reach a conclusion upon cost-effectiveness in the absence of adequate evidence of effectiveness or ineffectiveness.25 This is one powerful indictment of this guidance.

The generalisation of this last point is that, where there is insufficient evidence equipoise in relation to the clinical evidence base, cost-effectiveness judgements should not be made. Indeed in its own guidance upon social value judgements NICE states that:

Principle 1: NICE should not recommend an intervention (that is, a treatment, procedure, action or programme) if there is no evidence, or not enough evidence, on which to make a clear decision. But NICE's advisory bodies may recommend the use of the intervention within a research programme if this will provide more information about its effectiveness, safety or cost.26 27

While we may make sophist arguments about the difference between an act and an omission,28 based upon the arguments made here, the correct answer to insufficient evidence equipoise would seem to be adequate research.

A NICE trial

The advice in NICE's guidance upon social value judgements has an additional and deeper purpose. To see this we need to realise that the effect of the NICE guidance is to institute a de facto population wide prospective cohort study to test the null hypothesis that antibiotic prophylaxis does not reduce the population wide incidence of IE.

Prior guidance had issued antibiotic prophylaxis to broad population groups. The new guidance withholds antibiotic prophylaxis from virtually all these patient groups. By comparing the incidence of IE in the national population before and after the guidance was issued it should be possible to draw conclusions about the efficacy of antibiotic prophylaxis for preventing IE. Therefore by issuing and implementing national guidance NICE is effectively conducting a de facto clinical trial under the cover of a cost-effectiveness judgement.

The effect of issuing guidance on a cost-effectiveness basis is to negate any intention to undertake a clinical trial thereby side-stepping the legal framework for research.ii NICE is not a body constituted to undertake research protocols. It is a creature of statute created on the 26 February 1999 by order of the Secretary of State for Health.29 The functions of NICE are set out in directions from the Secretary of State for Health. Its core role is to operate ‘in connection with the promotion of clinical excellence and the effective use of available resources in the health service’.30 Therefore another core objection to the guidance in the context of insufficient evidence equipoise is that as a rationing body NICE should not be permitted to implement clinical trials outside the regulatory regime.

The burden of proving the empirical fact generating insufficient evidence equipoise should rest upon NICE because: (1) NICE is seeking to issue guidance that implements a change amounting to a de facto trial protocol; (2) NICE is not constituted to design or implement research; and (3) this change is not subject to oversight by the regulatory bodies charged to protect research subjects. Note how this burden of proof is more readily discharged in situations where there is conflicting evidence equipoise.

The pragmatic general principle should be that NICE should not exercise the power of the fee payer to intervene and alter the status quo where the underlying empirical evidence base is in insufficient evidence equipoise.

Expert guidance in the face of insufficient evidence clinical equipoise

Here we must first note how NICE seeks to enhance its power to implement cost-effectiveness decisions by donning the cloak of issuing expert consensus guidance. If there is truly insufficient evidence clinical equipoise then outside the context of a formal clinical trial protocol the decision whether or not to treat should fall to be discussed between the responsible clinician and the patient.31 This view is supported by the preamble of the NICE guidance itself:

This guidance represents the view of the Institute, which was arrived at after careful consideration of the evidence available. Healthcare professionals are expected to take it fully into account when exercising their clinical judgement. The guidance does not, however, override the individual responsibility of healthcare professionals to make decisions appropriate to the circumstances of the individual patient, in consultation with the patient…3

The principle is that, in the context of insufficient evidence clinical equipoise, a decision about a general patient cannot determine the decision for a particular patient. The place for expert consensus guidance here is illumination not determination. The expert consensus dimension of the NICE guidance can only validly operate through the clinical discretion of the responsible clinician in this context. The argument that the fee payer will not pay for the treatment should be kept separate from the question of whether or not the treatment might benefit this particular patient.

NICE's declared view of the evidence base cannot enhance the probative force of this evidence base merely by virtue of its perception.32 Therefore NICE must rely upon its power as the market dominant fee payer and monopsony employer to implement its guidance uniformly on the ground in this context.33

This exercise of power rather than logic to ensure implementation of the guidance means that there is a shadow cast upon clinical decision-making through the value judgements made by NICE. These shadows impact particularly upon the scope and exercise of clinical discretion. Similar shadows are found elsewhere too, for example, in relation to top up fees.34 These distortions are another important, albeit less tangible, ground for challenging NICE guidance made in this context.