Intended for healthcare professionals

Education And Debate

The case against aggressive treatment of type 2 diabetes: critique of the UK prospective diabetes studyCommentary: UKPDS is well designed and clinically important

BMJ 2001; 323 doi: https://doi.org/10.1136/bmj.323.7317.854 (Published 13 October 2001) Cite this as: BMJ 2001;323:854

The case against aggressive treatment of type 2 diabetes: critique of the UK prospective diabetes study

  1. R M Ewart (rewart{at}siumed.edu), associate professor
  1. Department of Family and Community Medicine, Southern Illinois University School of Medicine, Springfield, Illinois, USA
  2. a Diabetes Trials Unit, Oxford Centre for Diabetes, Endocrinology and Metabolism, University of Oxford, Oxford OX2 6HE
  3. b Oxford Centre for Diabetes, Endocrinology and Metabolism
  4. c St Bartholomew's and Royal London School of Medicine, London EC1M 6BQ
  1. Correspondence to: Southern Illinois University Center for Family Medicine, 520 North 4th Street, Springfield, IL 62702-5238, USA
  • Accepted 12 March 2001

Summary points

During the course of the UK prospective diabetes study the length of follow up was changed, and changes seem to have been made to the end points and the groups under analysis

These changes are not in keeping with accepted scientific principles and make the results of the trial suspect

An independent review of the trial's design and analysis is needed

The results of the study do not justify aggressive treatment of type 2 diabetes

Methods

I searched Medline (Ovid's CD version) from 1976 to March 2000, using various combinations of the keywords “UKPDS” and “prospective diabetes study” and the names of the principal authors of the study's reports. I reviewed the resulting articles for any description of the design of the UKPDS. I reviewed reference lists for other references.

Changes in the end points

The authors have presented various end points over the life of the study (box). The UKPDS grew originally out of the authors' interest in the use of basal rather than postprandial glucose in monitoring diabetes. They concluded, “A prospective controlled trial of different ways of obtaining basal normoglycaemia is needed to determine whether the improved control of mild diabetes is beneficial.”3 In the first report of the study, published in 1983, the authors set out their rationale: “If, after dietary therapy, the fasting plasma glucose continues to be raised, there is little information available to determine whether one should continue with diet alone, or add a sulphonylurea, biguanide or insulin.”4 Specific end points were not laid out in this original description but were set out the following year in a letter discussing the paper. 4 5 The authors later argued (1999) that this “brief letter” had only “minor differences in wording.”6 It seems, however, that the letter was written specifically to clarify the vagueness and ambiguities of the 1983 paper: the authors wrote, “In the initial [1983] report, it was not feasible to mention all details of the study.”5

Changes in the end points and stopping points in the UK prospective diabetes study

Changes in the end points and stopping points in the UK prospective diabetes study

View this table:

In 1991, after the study had been going 14 years and after numerous interim analyses, 1 5 the authors restated their end points.7 They presented two possible interpretations of the end points, but neither matches the end points given in the final publications. 1 2 Nevertheless, the authors repeatedly refer to the 1991 paper as setting out the final end points. 1 8 9 The end points set out in 1993 are almost identical to those published in 1984. 5 8 It is not until 1995 that the end points take on a form similar to those given in the final publication in 1998. 1 10 The 1995 paper is also the first time that total mortality is set out as an end point and the first time that a clear distinction is made between microvascular and macrovascular end points (although this distinction is mentioned briefly in earlier reports). By 1996 the end points are, apart from differences in wording, identical to those seen in the final report—the only change is that “diabetes-related mortality and major clinical endpoints” is renamed “any diabetes-related endpoint.” 1 9

The final report defines, for the first time, four additional end points to be used when comparing intensive treatments (see box).1 These end points do not, as far as I could discover, appear in any other report. Every previous publication implies or states that the end points of the main comparison were to be used in the secondary comparison among the different agents—this was stated explicitly in 1984 and 1993. 5 8 In the main body of the final report the authors state that these secondary end points are to be used only when comparing different intensive regimens, not when comparing intensive and conventional regimens (although such a comparison is made in figure 4 of the report).1 Thus the authors' claim that the “intensive treatment group had a substantial, 25% reduction in the risk of microvascular endpoints” (the only one of the four additional end points that was significantly different in the retrospective comparison of conventional and intensive treatment) is not supported by their study design, and the result must be viewed simply as hypothesis generating.1

Summary of the changes

The authors made substantial changes to the end points as the study progressed. In particular, cataract extraction, vitreous haemorrhage, heart failure, and retinal photocoagulation were not included in 1984, were mentioned ambiguously (as “events,” not end points) in 1991, were not included in 1993, and then were included from 1995 on. 5 7 8 In fact, none of the publications before 1995 specifically set out the end points that were used in the final analysis. It also seems that the decisions about which of the aggregations of end points to include were based on the results of the interim analyses that were available to the authors. 1 5

Soft end points

A second point is that the study was not blinded. This is less an issue with the hard end points (death) than with the soft end points (cataract extraction and retinal photocoagulation), where the decision to perform a procedure might have been influenced by the degree of glucose control in a patient. This is important, as the decrease in these soft end points accounts for all of the significant beneficial results claimed in the study.1

Analysis of subgroups

The main purpose of the study was always to compare intensive and conventional (diet) treatment. This aim is consistent throughout the early reports of the study and was stated as the overall objective in the authors' 1998 review. 4 7 8 11 12

Over the years, however, a subtle change was made. A question based primarily on outcome (does lowering blood sugar decrease morbidity and mortality?), with a secondary question based on mechanism (does the way in which blood sugar is lowered matter?), was changed to a question based primarily on mechanism, and the question based on outcome was simply ignored. This change is reflected in the treatments that were analysed (initially conventional versus intensive treatment, then later conventional treatment versus sulphonylurea and insulin and conventional treatment versus metformin). The authors' argument, first put forth in 1996, that separate comparisons based on mechanism should be made is interesting and worthy of study, particularly with the development of the thiazolidinedione drugs, but it does not justify the decision not to publish the study as originally designed.9

Length of follow up

The study was originally planned to end in 1992, with a median follow up of seven years.5 In 1987 an interim analysis showed negative results; the study was therefore expanded in size and also in length of follow up.7 In 1990 the study was “due to report in 1995, by which time a total of 5000 patients will have been followed for a median of 8 yr.”13 In 1991 the study was “planned to finish in 1994 with a median follow-up of 9 years.”7 In 1993 it was also planned to finish the study in 1994.8 It was not until 1995 that the authors stated, “The clinical study will end in 1997 when the 4,209 patients will have had a median time since randomization of 11 years.”10

Interim analyses were planned to occur every six months to 1985 and yearly thereafter. 1 5 It seems that the authors continued the study until they obtained a result that was significant, without adjusting for repeatedly looking at the data. Although it is acceptable to extend a study for a set period and to have predetermined stopping rules, it is not acceptable to repeat interim analyses and to delay publication until a significant result is found.14

Recommendations for screening

Can the findings of this study be generalised to screening for asymptomatic diabetes? The patients in the study were aged 25-65 years, had type 2 diabetes that was newly diagnosed, and were referred by local general practitioners. No attempt was made to screen patients for diabetes, and at diagnosis 50% of the patients had evidence of diabetic tissue damage.10 The study was not designed to show a benefit from screening and would not have been powerful enough to do so even if it had been. Screening detects cases of diabetes much earlier in the course of the disease, and it is not logical to imply that beneficial treatment (if there is any) given later in the course of the disease would give the same benefit (either absolute or proportional) when given earlier. Furthermore, the ethics of screening are different in several respects from those of routine medical investigation and treatment and require that more attention be paid to issues of harm, consent, and cost.15

Are the results clinically significant?

The authors present details of their power calculations and state that the study had a good chance of detecting a 20% or 15% benefit. 1 7 9 10 12 They further state, “This reduction has been accepted as being a clinically significant gain,”7 and, “A protective effect of 15% has been judged to be clinically relevant,”12 implying that lower reductions should not be considered clinically significant.

Not one of the main results of the study even approaches these numbers. The best result is in the “any diabetes-related end point,” a reduction of 12% (95% confidence interval 1% to 21%).1 The risk of diabetes related death—clinically the most important result—was not significantly reduced.1

Conclusions

In 1999 the authors restated their position that after 1981 they made no substantive changes in the design of the study (apart from those discussed in the 1998 paper). 1 6 16 The ambiguities and contradictions in the various reports of the study cannot, however, be denied. Whether the authors' conclusions are supported by the data cannot be resolved by debate but only by an independent review of the study's design and analysis. Meanwhile it is not unreasonable to ask for the results to be published as outlined in 1984—that is, as a comparison between the intensive treatment and conventional treatment groups, using the two end points “deaths from vascular events, sudden death or renal failure” and “complication free interval, including avoidance of death from any cause, heart attack, angina, renal failure, blindness, major stroke or amputation.”5 It would not be unreasonable to add total mortality to this list, with the caveat that it was not included in the initial design.

If it is true that substantial changes, derived from ongoing reanalysis of the data, were made, and if reanalysis of the data according to the original design shows no significant benefit, then we must call into question the recommendations for screening and more aggressive treatment that have flowed from the publication of this study. 17 18 In the interim we should return to the position that, although management of the symptoms of type 2 diabetes is reasonable (that is, with the intention of keeping blood sugar concentrations below about 11-14 mmol/l (200-250 mg/dl), a recommendation in favour of screening and more aggressive care is not supported by the evidence presented in this study.

Acknowledgments

Magdi Nour helped with French translation. Robert Wesley provided criticism of earlier drafts of the manuscript. The author previously published a shorter and less detailed version of this critique, arguing only about the end points used in the 1984 and 1991 publications and not discussing the other issues (Ewart RM. The UKPDS: what was the question? [letter] Lancet 1999;353:1882).

Footnotes

  • Funding No additional funding.

  • Competing interests Both RME and his institution will lose clinical income if type 2 diabetes is treated less aggressively.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.

Commentary: UKPDS is well designed and clinically important

  1. Rury R Holman (holman{at}dtu.ox.ac.uk), UKPDS coordinatora,
  2. David R Matthews, chairman, UKPDS end point adjudication committeeb,
  3. Tom Meade, chairman, UKPDS data monitoring and ethics committeec
  1. Department of Family and Community Medicine, Southern Illinois University School of Medicine, Springfield, Illinois, USA
  2. a Diabetes Trials Unit, Oxford Centre for Diabetes, Endocrinology and Metabolism, University of Oxford, Oxford OX2 6HE
  3. b Oxford Centre for Diabetes, Endocrinology and Metabolism
  4. c St Bartholomew's and Royal London School of Medicine, London EC1M 6BQ
  1. Correspondence to: R Holman

    Ewart raises no substantive issues that haven't already been published elsewhere.1-3 The main UKPDS papers were published as a numbered sequence in which the main aims of the study and the outcomes, which were defined by protocol and were fixed from the outset, were consistent throughout. Minor differences in wording are those that would be expected between abbreviated and full descriptions of the clinical end points and between different authors' styles and editorial styles—for example, the description of end points as events. Thus, the brief letter in 1984 listed the two main outcome measures as “deaths from vascular events, sudden death or renal failure” rather than “diabetes-related deaths” and the “complication-free interval” as the converse of “any diabetes-related endpoint.”4 We agree that the UKPDS was not designed to address the question of screening, although this has not prevented other authors from seeking to make recommendations.5

    The UKPDS was planned from the outset to determine “whether improved glycaemic control of maturity-onset diabetes would diminish the morbidity and mortality of the disease.” This was stated in the first UKPDS paper, which reported data on the efficacy of the randomised treatments.6 All cause mortality, a prerequisite for all studies of clinical outcomes, was listed separately from deaths related to diabetes in the main protocol description in 1991, and deaths from hyperglycaemia or hypoglycaemia were specifically identified as being included in the aggregate outcome of “diabetes-related deaths.”7 The stopping criteria, which included only those end points that could be unequivocally verified by an end point adjudication committee that was masked to the randomised treatments, were selected by the UKPDS's data monitoring and ethics committee and have not been changed.7 Four separate non-fatal end points related to diabetes, whose determination might be influenced by clinical decisions (cataract extraction, photocoagulation, vitreous haemorrhage, and heart failure), were always to be included in the final analyses. 7 8 The 1995 paper, which described the progressive nature of type 2 diabetes over six years of follow up, did not detail the stopping criteria or the final analyses, as these were not relevant to that particular report.9 The 1998 paper clearly restates the three primary outcomes in the methods section and describes these as the major results in the summary.8 Analyses of secondary outcomes—aggregates of the predefined end points that related to the different types of vascular disease (myocardial infarction, stroke, amputation or peripheral vascular disease, microvascular complications)—showed that the 12% decrease in risk of any end point related to diabetes (P=0.029) was partly attributable to a 25% decrease in the risk of microvascular complications (P=0.0099). This result was given in context with the primary outcomes.

    The UKPDS, which started in 1977, is one of a number of seminal studies that have helped to formulate the strategies required for modern, large scale randomised controlled trials. Its design, conduct, and operational details were scrutinised by a hierarchy of committees, and the study has been the subject of regular reviews by the Medical Research Council in the United Kingdom and the National Institutes of Health in the United States.7 Although the nature of the disease and the treatments used meant that the trial could not be blinded, the data monitoring and ethics committee conducted all interim outcome analyses in complete confidence using predetermined stopping rules. It was this committee that determined that the original effect size chosen for the study was inappropriate and recommended the change to 15% as a clinically relevant value. It was also the data monitoring and ethics committee that, on the basis of the rate of event accrual, recommended that follow up be extended to 1997.

    We refute the allegation that the study is “seriously flawed.” The UKPDS was a well designed and rigorously managed intervention trial that has important implications for clinical practice.

    References

    1. 1.
    2. 2.
    3. 3.
    4. 4.
    5. 5.
    6. 6.
    7. 7.
    8. 8.
    9. 9.
    View Abstract