From guidance to practice: Why NICE is not enough

How does NICE work?

NICE was generally welcomed on its inauguration. 1 2 Previously, a lack of capacity at national level to appraise healthcare interventions before, or indeed after, their widespread diffusion had several adverse consequences: no guidance was available when important new drugs were first marketed, local policies varied, and unproved interventions entered routine use.3 NICE filled this gap, giving guidance on interventions of uncertain value and providing clinical guidelines and clinical audit packages. NICE should be congratulated for the transparency it has shown in its processes, in the face of some opposition from the pharmaceutical industry.

NICE's decisions are based on an assessment of the technology, usually prepared by independent researchers commissioned by the Health Technology Assessment programme, and submissions from the manufacturer(s) and from patient and professional groups. These are considered by the appraisals committee, which then advises the institute on what the guidance to the NHS should be. This follows two periods of consultation, and consultees may appeal as a last step before the guidance is issued to the NHS.4 The table summarises NICE's guidance to date.

How successful has NICE been?

NICE has succeeded in executing a complex and high profile process that has changed the terms of debate about the interventions it has reviewed. There is now a broad acceptance in principle of the legitimacy of central guidance on controversial issues of service availability, even if specific pieces of guidance are not unanimously supported. Yet the real measure of NICE's success should be an improvement in the overall cost effectiveness and appropriateness of the interventions available to the NHS's users. There is as yet no published information on the implementation by the NHS of NICE's guidance, so we cannot assess success against this yardstick. Sharp criticism 5 6 indicates that NICE's honeymoon period is long since over and that there is, or will be, resistance to implementation of pieces of guidance that are particularly expensive or clinically unpersuasive. Before condemning NICE, we should examine how much of the difficulty arises from NICE itself and how much from the context in which it must work.

For NICE to achieve its goal of improving the appropriateness of healthcare interventions available in the NHS, there should be clear answers to three questions.

• How does NICE reach its conclusions? The NHS will be more likely to implement NICE's guidance with confidence if it understands the guidance's origins

• How is the NHS to respond to NICE guidance? Uncertainty about the impact of guidance will make planning and delivering clinical services more difficult

• Who monitors compliance with NICE's guidance? Without checks on compliance, there can be little certainty of NICE's impact nor feedback on the effectiveness and acceptability of its products.

How does NICE reach its conclusions?

NICE was preceded by various regional bodies, such as those in the South and West7 and Trent.8 These showed that it was feasible to evaluate healthcare interventions quickly enough to satisfy the NHS but rigorously enough to be defensible. They also showed that the NHS would, at least to some extent, act on the results. In the South and West, the Development and Evaluation Committee was governed by decision rules which tended to mandate verdicts based on the strength of available evidence of effectiveness and on the cost utility of the intervention under consideration. It might have been assumed that NICE's appraisal committee would operate similarly, and would therefore not support interventions in the absence of randomised controlled trials showing worthwhile benefit at reasonable cost.

However, this is not the case, as shown by the handling of donepezil and other anticholinergic drugs for Alzheimer's disease. NICE recommended their use,9 whereas the South and West Committee, using similar evidence, did not.10 The Trent Working Group on Acute Purchasing was also cautious about the drugs.11 NICE recognised the weakness of the evidence on cost effectiveness, and the appraisal committee noted: “The main benefit of these drugs is the improvement in patients' cognitive and other functioning, and the main potential cost-saving results from possible delayed progression to the requirement for nursing home care. Neither can be reliably or easily estimated from the existing trial evidence.” Indeed, the committee reports that the systematic review of the evidence of clinical and cost effectiveness commissioned to inform their decision “did not provide a helpful basis from which to draw a conclusion.” This was not because of the weakness of the review, but because of the severe limitations of the underlying evidence.

However, NICE has criteria for approval other than cost effectiveness12:

• The broad clinical priorities of the NHS

• The degree of clinical need of the patients with the condition under consideration

• The broad balance of benefits and costs

• Any guidance from the secretary of state and National Assembly for Wales on the resources likely to be available and on such other matters as they may think fit

• The effective use of available resources

• The encouragement of innovation.

These are broader criteria than those used by the regional development and evaluation committees and do not permit explicit rules on how decisions are made. NICE's recommendation of the anticholinergic drugs for Alzheimer's disease shows that evidence of some clinical benefit can be enough to secure approval despite a lack of adequate means of measuring that benefit, no evidence on quality of life, and uninterpretable health economics.

The wider criteria used by NICE mean that its threshold for approval will be lower than those of its regional predecessors and those used by commissioners at local level. For example, the NHS has not explicitly used its commissioning processes to encourage innovation, and indeed has tended to resist the general introduction of new interventions until they had been adequately evaluated. Conversely, the Department of Health is the sponsoring department for the British pharmaceutical industry and has a responsibility to promote its success. This may explain the difference in criteria, but it is unclear to what extent NICE's criteria should also be used by the NHS in handling interventions that NICE will not appraise. A seminal Department of Health report recommended that unevaluated new forms of health care should be provided by the NHS only “within the context of properly designed research to assess their effects.”13 Since the NICE guidance acknowledges the absence of satisfactory research on drugs for Alzheimer's disease, perhaps this would have provided a better way of controlled and evaluated innovation. Paradoxically, the guidance may jeopardise existing NHS funded, placebo controlled evaluations of the drugs such as the AD 2000 trial.14

NICE has recently been asked to produce guidance on subfertility treatments such as in vitro fertilisation. A leading objection to their inclusion in the NHS has been that the treatment of severe subfertility is not an appropriate use of NHS resources, regardless of effectiveness and cost. Cosmetic surgery and gender reassignment are often unavailable in the NHS for similar reasons. Cases such as these will test the applicability of NICE's criteria and the tolerance of the NHS to central direction on the values that determine its scope. They may also expose a tension between an approach to setting limits to NHS care based on cost effectiveness and one that is based on a wider set of values, including appropriateness.

How is the NHS to respond to NICE guidance?

The size and scope of NICE's programme mean that many clinicians will soon be making decisions affected by its guidance. What is expected of them? Until recently, NICE's recommendations had the legal status of guidance—something that the NHS was expected to consider carefully but was not obliged to follow.15 The rubric at the start of each of NICE's guidance documents tells health professionals that they are expected to “take it fully into account when exercising their clinical judgement.” This permits local discretion about implementation.

Experience, however, suggested expectations are different. When a primary care group in Devon decided against following NICE's guidance on zanamivir (Relenza) NICE's chairman, Sir Michael Rawlins, reportedly expressed his disapproval by asking, “How will they feel when one or two patients … dies of flu?”16—yet the NICE guidance states: “No reliable data are available as to the impact of the use of zanamivir on … mortality.”17 Similarly, the Department of Health stated its expectation that NICE recommendations will be followed and has specifically emphasised the importance of prompt implementation of guidance on new drugs for cancer. When Wiltshire Health Authority was reported as saying that it would be phasing in the implementation of NICE guidance,18 it quickly issued a correction.

In December 2001 the government announced that it would place statutory obligations on health authorities and primary care trusts to provide appropriate funding for treatments recommended by NICE. From January 2002, primary care trusts in England will have three months to provide funding, though this will be extended in cases where more time is required to set up the service in question. The position in Wales was not affected by the announcement and remains ambiguous.

One of the most unattractive aspects of the NHS in the 1990s was the geographical inequity of access that arose from devolved decision making on new interventions. It was disliked by clinicians, resented by patients, painful to politicians, and served nobody's interests. NICE was conceived partly to remedy this, a reason why many in the NHS welcomed it. Although the government's recent announcement removes ambiguity about its expectations, some problems remain.

It will be difficult for local health services to implement the 50 pieces of guidance expected from NICE each year, both in terms of promoting change in clinical practice and in terms of affordability. Significantly, the direction is to commissioners, not to clinicians: if the reason for a delay in implementing guidance is not financial (clinical resistance, for example), then variations in access will persist. What is the position when guidance can be implemented without new costs arising or when it would save money? Furthermore, making the rapid and universal implementation of all NICE guidance non-negotiable may distract the NHS and divert resources from other more important initiatives. Given that the health benefits of some interventions that NICE recommends are less than compelling, the release of its guidance may not end debate about their place or inspire cautious doctors to comply. In any case, as Sculpher et al point out,19 the NHS at local level must deny other services funding in order to fund what NICE recommends. This will apply both to specific interventions for which there is no NICE guidance and to whole services for which guidance is never likely (such as those for people with learning disability). The services that are not funded will vary, so inequity of access will persist and high opportunity costs may be paid.

Since this is one of few spending programmes mandated by ministerial direction, commissioners will put aside money at the start of the financial year to ensure compliance. From this year's experience, the amount put aside may be up to £1m per primary care trust. Since the content and costs of the forthcoming year's guidance are not known in advance, the amount will be speculative and is likely to be either too much, unnecessarily reducing investment in other services, or too little, requiring potentially destabilising financial shifts during the year. This problem could be solved by designated additional funding to meet costs being released with each piece of guidance, but ministers say they will not support that.

Who monitors compliance with NICE's guidance?

Several organisations claim a role in monitoring compliance with NICE's guidance, but none is ideally placed to achieve this. Speaking at NICE's conference in 2000, the health secretary, Alan Milburn, announced that the Department of Health would monitor health authorities and trusts to check on implementation of each piece of NICE guidance. His statement that monitoring would be shortly after publication of the guidance and again six months later implies that implementation is expected to be prompt. The Department of Health has already piloted monitoring by using the common information core returns that health authorities make regularly. To be sure that every piece of guidance is being followed will require clinical audit on a more widespread scale than has been usual hitherto and will yield answers more complex and equivocal than the “yes” or “no” responses sought in the pilot. The increasing distance of health authorities from responsibility for clinical governance and the organisational changes about to engulf them reduce their capacity to influence the process and supply valid information to the Department of Health.

The Commission for Health Improvement will also examine systems for ensuring compliance in its four-yearly inspections, but these are not frequent or detailed enough to give an adequate picture of progress. The Audit Commission says it, too, will have a role. At best, this will cover only a small part of the ground. Finally, it is not yet clear what sanctions will follow for NHS organisations deemed too slow to implement, particularly if they contend that the health problem is of relatively low importance locally.

Conclusion

NICE guidance in its present form may not be able to alter substantially clinical practice:

• NICE reaches its conclusions on the basis of criteria for which the NHS may not feel ownership

• The status of NICE guidance may be unclear to its recipients, with the legal obligation on primary care trusts to fund implementation being balanced by clinical freedom

• Responsibility for monitoring compliance is vague, with several agencies potentially involved, but no clear lead role, and often no objective measure of implementation.

Implementation of NICE guidance is more likely in a health service that broadly accepts the process, can appropriately place the guidance in the context of other priorities, and understands the mechanisms through which such implementation will be measured. Wider debate about the criteria, clarity on status, and more concise recommendations about clinical audit methods will all be necessary to achieve this end.