Effectiveness, efficiency, and NICE

A NICE start but evidence costs money

The National Institute for Clinical Excellence (NICE) was established in England and Wales in 1999 to “provide guidance to the NHS on the use of selected new and established technologies.”1 NICE synthesises evidence on the effectiveness and cost of treatments and reaches “a judgment as to whether, on balance, the intervention can be recommended as a cost-effective use of NHS resources.”1 How has the institute measured up to these ambitious goals, and what has been learnt about the demands of an explicit process for assessing health technology?

The institute attracted attention from the international media with its first judgment that “health professionals should not prescribe zanamivir (Relenza) during the 1999/2000 influenza season.”2 The additional cost to the NHS would have been about £10m ($15m) for the benefit of reducing episodes of flu from six days to five. Although subsequently revised,3 the decision showed that the institute has teeth and is prepared to bite even home grown drug companies like GlaxoWellcome (now GlaxoSmithKline). In some places, such as Australia4 and Ontario, Canada,5 pharmaceutical companies must prove that their products are cost effective before they can be reimbursed by the government. Although NICE operates differently in that it does not automatically assess new products and provides guidance rather than mandates, it is clear that products will need to be both effective and provide good value for money to be recommended for use in the NHS. Unusually, NICE's remit also includes medical devices and other healthcare programmes, and its activities are surely being scrutinised by other healthcare systems.

But the evidence on which the analyses of costs and benefits are based is often incomplete or inappropriate. The appraisal of hip prostheses, for example, suffered from the dearth of long term data on revision rates. Similarly, the continuing deliberations about interferon beta are likely to be constrained by data from short term trials with outcome measures that are of limited relevance to decisions about the allocation of resources. The institute, however, is pragmatic about any shortfalls in evidence: give the best advice possible using the data available today, but be prepared to revisit judgments when better data arrive. For many technologies—particularly those without a sponsoring company—the onus for generating adequate data will lie with the NHS health technology assessment programme; its budget may need to be increased, but using NHS resources to generate evidence may be money well spent.

The institute's appraisals are likely to have major implications for the drug and medical device industries because these industries supply much of the information for appraisal. The need to submit dossiers in support of their products is making companies think carefully about their research and development programmes. When products have been marketed for some time it is possible for companies routinely to accumulate data, although the data are not typically gathered within the framework of experimental studies. NICE will, however, increasingly have to consider products which have yet to reach the market, so such “real life” data will be lacking. Before launching a new product, drug companies have in the past focused on generating evidence for the drug licensing authorities. Such data are generally of limited value to NICE, so companies are likely to invest in more pragmatic clinical trials with broader population bases to collect the data on cost and health outcomes which are relevant to NICE's decision making.

The institute's interest in finding value for money puts the methods of economic evaluation under the microscope. Although analytical economic methods have developed rapidly, they have yet to make a major impact on applied economic evaluations and on the data submitted to NICE. The institute's recent publication of more detailed guidance for economic evaluation provides greater clarity about the institute's view of best practice.6 The international evidence on the quality of drug companies' economic evaluations to support reimbursement is not encouraging. Between 1994 and 1997 a total of 326 evaluations were submitted to the Australian Department of Health and Aged Care, and 218 of these had major problems detected by critical review.7 However, 62% of these problems were caused by the absence or poor quality of clinical data used in the studies. The quality of data on effectiveness will probably also be a problem for NICE.

What impact will the institute's guidance have on practice in the NHS? There will be particular interest in how the NHS reacts to appraisals that find that a technology benefits patients but introduces extra costs to the health service, such as coronary stents and taxanes. The rationale for NICE is, in part, based on the desire to end the uneven geographical distribution of particular forms of health care. However, unless funding is earmarked and made available to health authorities for these interventions, they can only be offered to patients if the provision of other services elsewhere in the system is curtailed. Hence, local variations in the availability of particular services will remain; the appraisal process will simply shift the unevenness between services. The institute's role in developing clinical practice guidelines, taken together with national service frameworks, promises to ameliorate this problem. It will be necessary to expand the appraisal process, particularly to identify widely used technologies that are not cost effective, to release resources for new interventions. In principle, NICE's role in looking at a wide range of both new and old technologies is important, although the focus of the latest group of interventions to be appraised is narrow and concentrates largely on new cancer drugs.8

The NHS and other healthcare systems that are collectively funded need transparent decision making about which types of health care offer value for money and thus can justifiably be funded; this decision making needs to use appropriate and explicit methods. There is much to commend in the early stages of the institute's appraisal process, not least the openness and transparency it has achieved through its website (http://www.nice.org.uk/). Part of the challenge for NICE and the assessment of health technology in England and Wales is economic: gathering, synthesising, and scrutinising data is a valuable exercise but it is costly. The amount and allocation of research funds should also pass the test of cost effectiveness. This will depend on whether clinicians and managers in the NHS take notice of NICE's guidance.