Discussing randomised clinical trials of cancer therapy: evaluation of a Cancer Research UK training programme
BMJ 2005; 330 doi: https://doi.org/10.1136/bmj.38366.562685.8F (Published 17 February 2005) Cite this as: BMJ 2005;330:400
- V Jenkins (val{at}sussex.ac.uk), senior research fellow in psycho-oncology1,
- L Fallowfield, professor in psycho-oncology1,
- I Solis-Trapala, statistical researcher2,
- C Langridge, data manager1,
- V Farewell, senior scientist3
- 1 Sussex Psychosocial Oncology Group, Brighton & Sussex Medical School, University of Sussex, Brighton BN1 9QG
- 2 Centro de Investigacion en Matematicas, Guanajuato, Mexico
- 3 MRC Biostatistics Unit, Institute of Public Health, Cambridge
- Correspondence to: V Jenkins
- Accepted 10 January 2005
Abstract
Objective To evaluate a training intervention aimed at improving healthcare professionals' communication with cancer patients about randomised clinical trials.
Design Before and after evaluation of training programme.
Setting Members of the National Cancer Research Network, Scottish Trials Network, and the Welsh Cancer Trials Network
Participants 101 healthcare professionals (33 clinicians and 68 research nurses).
Intervention Four modules delivered by a trained facilitator using videotapes and interactive exercises to cover general issues about discussing randomised clinical trials with patients, problems specific to adjuvant trials, trials with palliation as the goal, and trials where patients had a strong preference for one treatment arm.
Main outcome measures Before and after the intervention, participants were videotaped discussing a trial with an actor portraying a patient. These consultations were assessed for presence of information required by good clinical practice guidelines. The actor patients gave an assessment after each interview. Participants reported their self confidence about key aspects of trial discussion.
Results Analysis of the videotaped consultations showed that, after intervention, significantly more participants displayed key communication behaviours such as explaining randomisation (69 v 81, odds ratio 2.33, P = 0.033), checking patients' understanding (11 v 31, odds ratio 3.22, P = 0.002), and discussing standard treatment (73 v 88, odds ratio 4.75, P = 0.005) and side effects (69 v 85, odds ratio 3.29, P = 0.006). Participants' self confidence increased significantly (P < 0.001) across all areas. Actor patients' ratings of participants' communication showed significant improvements for 12/15 key items.
Conclusion This intensive 8 hour intervention significantly improved participants' confidence and competence when communicating about randomised clinical trials.
Introduction
Worldwide fewer than 5% of eligible patients with cancer participate in randomised clinical trials, limiting the progress of cancer research. In Britain healthcare professionals are under pressure from various bodies to recruit more patients into such trials.1 Various publications provide guidance about trial conduct and the standards expected for good clinical practice,2 but few help clinicians in their discussions with patients.3 Reference is made to the importance of communication with patients, especially with regard to informed consent, but research has shown that few clinicians feel skilled in this task. More than 52% of senior clinicians attending communication skills courses and 27% of chemotherapy nurses acknowledged that providing complex information and seeking consent to clinical trials were their main communication problems, surpassing even the breaking of bad news.4
In discussing randomised clinical trials, a healthcare professional must discuss standard treatment, the logic for a trial, the uncertainty about new drugs or procedures, and the potential benefits and harms of the different treatments, as well as explain the concept of randomisation and other terms such as double blind or placebo controlled. If communication is inadequate patients may not understand the experimental nature of the trial, be unclear about treatment options outside a trial setting, and be unable to give truly educated consent.5
One study of the content of discussions about randomised trials between clinicians and patients reported that, despite the stated requirements of trial protocols, clinicians adopted idiosyncratic methods when providing trial information.6 Although most discussed the different treatments on offer and associated side effects in great detail, they often gave minimal reasons for randomisation.
A recent survey of 190 patients enrolled in double blind placebo controlled trials showed that they did not fully understand certain aspects of trial recruitment even though they felt satisfied with the information provided.7 Understanding more about patients' reasons for trial participation or refusal and their knowledge about trials helped increase recruitment in one particular study. Researchers applied their findings to improve the recruitment rate from 30% to 70%.8 Use of more effective communication techniques may be another method of ensuring better understanding and decision making and improving recruitment to trials.
After requests for training materials, we designed a comprehensive educational programme to help health professionals when communicating about randomised trials. Many otherwise laudable communication skills courses, guidelines, and educational programmes lack any objective evaluation.9 Therefore, before launching a national training programme, we evaluated its efficacy and acceptability with UK health professionals engaged in trial recruitment.
Participants and methods
Participants
We sent a general letter of invitation to the National Cancer Research Network, Scottish Trials Network, and the Welsh Cancer Trials Network inviting healthcare professionals to attend our training course: 103 enrolled in the study, but one research nurse became ill during the course and another withdrew. Table 1 shows the characteristics of the remaining 33 clinicians and 68 research nurses or radiographers. Although nearly all participants had previously received communication skills training, many (31 clinicians and 23 nurses) had not attended a good clinical practice guideline course.
Details of participants' characteristics and previous experience of communication skills training. Values are numbers of participants
Before attending our course, participants nominated a trial in which they were currently involved which would be used in their videotaped consultations. Almost half the group (48%) chose a trial comparing standard treatment with a new treatment, 22% chose a trial that had a “no extra treatment” arm (such as surgery with or without preoperative chemotherapy), 18% chose a trial with a placebo arm, 9% chose a trial that had a “toxic” or high dose arm, and 3% chose a surgical trial.
Course contents
Our course lasted eight hours, split over two days. The course comprised interactive exercises, didactic presentations, and facilitated discussion about the videotaped scenarios in the modules. During the course, we encouraged participants to consider how they structured trial discussions with patients and how they described treatments available on and off trial and the process of randomisation, and to compare these with those depicted in the modules. At the end of the course participants generated a list of key points about trial discussion.
Course materials
We developed the training materials in close collaboration with experienced doctors, research nurses, trial managers, and patient groups. We had to conduct original research in several subjects such as patient preferences for different descriptions of randomisation as evidence based recommendations did not exist.10 The training materials comprised four video modules (see fig 1), a CD Rom, and a comprehensive facilitator's handbook.
Example of the scenarios depicted in the training videos. In module 3 Professor Peter Selby explains the Myeloma VII trial to the distressed patient “Sheila” with the research nurse present
Module 1 provides a generic introduction to randomised clinical trials and includes comments by six patients, eight clinicians, a research nurse, and a trial manager exploring the difficulties associated with discussing trials.
Module 2 deals with the discussion of adjuvant treatment trials and contains two scenarios—VICTOR, a placebo controlled colorectal cancer trial, and the intergroup exemestane study (IES), a breast cancer trial. This module also looks at handling uncertainty and dealing with uninformed and suspicious patients.
Module 3 includes two scenarios of palliative trials—Myeloma VII, a multiple myeloma trial, and the big lung trial (BLT) for non-small cell lung cancer. This module promotes discussion about handling deferential patients and their more questioning relatives and giving distressed patients complex information about highly toxic treatment.
Module 4 depicts patients who may have a strong preference for a particular treatment arm and explores how to handle patients with high information needs who have collected many internet articles and newspaper cuttings—in the MRC CLASICC trial, a surgical trial for bowel cancer, and MRC PRO7, a prostate cancer trial.
All four modules have a linking commentary and statements by clinicians and nurses.
The facilitator handbook was developed for use by both experienced and less experienced facilitators. It contains a time coded commentary about the issues shown in the accompanying videotapes and suggestions about appropriate places to stop and engage a group in exercises or discussion. It provides examples of how to structure a teaching session and a bibliography of relevant reading materials. The CD Rom contains questionnaires, handouts about the trials described in the videos, a bibliography, group exercises, and PowerPoint presentations for the didactic work showing the evidence base for suggestions made.
Videotaped consultations
Before and after the course, we videotaped participants conducting simulated discussions with patients about clinical trials. The actors who portrayed the patients were experienced in improvisation as “cancer patients” and were well briefed about their disease, its symptoms, and the trial being discussed. The actors were assigned one of four different characters; the patient who comes armed with newspaper clippings and web reports on cancer treatments (internet guru), the patient who has already decided which treatment he or she wishes to receive (preference), the patient who defers decisions about trial participation to the nurse or clinician (deferential), and the patient who is uneasy about trials (suspicious). To enhance authenticity, we used different actors in each participant's initial and post-course discussion, but the patient characteristic and the trial remained constant.
Thirty three participants conducted two videotaped patient interviews before the course, so that we could check for any practice effect in performing the interview twice.
Assessments
Before and after the training course, participants rated their self confidence on a scale from 0 (none) to 10 (very confident) about general aspects of discussing trials with patients. Items included describing randomisation, discussing different types of trials (such as placebo controlled), and handling discussions with challenging patients (such as the suspicious, demanding, or overly anxious).
We objectively assessed participants' videotaped interviews. The data manager of the unit assigned a random number to each tape, and the assessment was done by one of the authors (VJ), who was not involved with filming the interviews and was therefore blinded to whether they were before or after the training course. Analysis involved checking for the presence of key information as stated in the good clinical practice and ethical guidelines—for example, the voluntary nature of trials, explicit use of the term randomisation, and descriptions of side effects. We also assessed specific behaviours against pre-set criteria—such as checking patients' understanding and summarising information.
After each videotaped interview, the actor “patients” completed a 15 item questionnaire about whether they had understood the explanation of randomisation, were told that participation was voluntary, and other communication issues. The questionnaire had a Likert-type scale of 0-4 (0 = not at all, 1 = a little, 2 = somewhat, 3 = quite a bit, 4 = very much).
Hypotheses
Our a priori hypotheses were that
Participants' communication in key areas of discussions about clinical trials would improve after the course—that is, competence would be measurably better
Participants would feel more confident about discussing trials and about approaching different types of patients after the course—that is, self confidence would be enhanced.
Statistical methods
We analysed the data generated from the video analysis using conditional logistic regression models in order to compare the scores reported for each participant before and after the course. The analysis was based on binary (yes/no) variables derived from either a yes/no score or by defining yes to be “Very well” or “Reasonably well” and no to be “Not at all” or “Not very well” from a four level score. We estimated odds ratios, with values > 1 indicating that a “yes” score was more likely in the interview after the course and that participants' behaviour had improved after the course.
We defined scores for the actors' questionnaire responses as one for “Not at all” or “A little bit”, two for “Somewhat,” and three for “Quite a bit” or “Very much.” An odds ratio > 1 represents a shift towards upper categories after the course. The scores from the participants' confidence questionnaire were used directly, and an odds ratio > 1 indicates a shift towards higher levels of confidence after the course.
Formally, all odds ratios represent, for two observations from the same individual differing by one unit, the relative odds of the larger value being after the course compared with before the course. No distributional assumptions are required for the scores. Thus, the conditional logistic model used here provides a robust method for before and after comparisons. The key data for estimation purposes are the number of participants with different scores at the two time points. Large positive odds ratios occur when more shifts occur towards yes, or larger values, than towards no, or smaller values.
Results
We found no evidence of practice effects in the two videotaped interviews performed by 33 participants before the training course. On 108 occasions a participant scored a “yes” on a question for the second interview and a “no” for the first interview and on 112 occasions a participant scored “no” for the second interview and “yes” for the first. Thus, in the absence of evidence for practice effects, our final analysis for the whole sample was based on the initial interview before the course.
Video analysis—We examined rater reliability for 10% of the videotapes and found good agreement in rating of interviews conducted before and after the course (κ = 0.664 (SE 0.025)). Figure 2 and table 2 show the odds ratios for the video analysis. For example, with the item “Is patient's understanding of the term randomisation checked?” the odds of a “yes” score were 3.22 times higher after the course than before the course. Many important communication behaviours specific to trial discussions improved significantly. After the course, more participants described the particular type of trial “very well” (81 v 91), gave clear explanations about the treatments available on and off trial (73 v 88) and their side effects (69 v 85), and explained the term randomisation (69 v 81) but used fewer analogies to do so (36 v 22). Participants also improved in checking that the patient had understood the explanation of randomisation (11 v 31) and showed some improvement in explaining the voluntary nature of the trial (71 v 81, P = 0.08). However, some behaviours stated in the good clinical practice guidelines, such as encouraging patients to read the trial information sheet and informing them that they could withdraw at any time did not alter significantly (92 v 85 and 33 v 34, respectively).
Odds ratios (95% confidence intervals) of improved scores after the training course for the presence of key information in participants' videotaped interviews
Odds ratios and P values of improved scores after the training course for the presence of key information in participants' videotaped interviews
Actor patients' ratings—Figure 3 and table 3 show the odds ratios for the actor's responses to their questionnaire. For 12/15 statements the actors reported significant improvements after the training course; this occurred despite the relatively high positive baseline scores that influenced the lack of demonstrable change for three statements (“I was left confused,” “The healthcare professional seemed to favour one treatment over another,” “I still have unanswered questions”).
Odds ratios (95% confidence intervals) of improved scores after the training course for actor patients' assessments of participants' videotaped interviews
Odds ratios of improved scores after the training course for actor patients' assessments of healthcare professionals' videotaped interviews
For neither of these objective measures did the actor patients' sex or character type affect the pattern of participants' behaviour changes, nor was there a relation with participant type (clinician v research nurse or radiographers)
Participants' self confidence—Figure 4 and table 4 show that participants' self confidence ratings increased significantly after the training course for each statement at the P = 0.001 level. Despite these improvements, several aspects of trial recruitment remained challenging, including describing randomisation and time constraints. Explaining multi-arm and phase I trials, which were not portrayed in the training modules, remained problematic for most participants.
Odds ratios (95% confidence intervals) of improved scores after the training course for participants' self rated confidence
Participants' self confidence ratings before and after the training course and odds ratios of scores being improved (n=101)
Discussion
To our knowledge this is the first evaluation of an intervention designed specifically to help health professionals provide clear information about phase III randomised trials of cancer treatments to patients and to encourage them to approach all eligible patients for recruitment. The use of videos in health education for patients and for training health professionals is now common, but, although many are entertaining, they can encourage a passive audience, which leads to little learning. We designed our modules to stimulate and provoke constructive discussion among viewers. The course integrated different activities in order to create simultaneous rather than sequential skills development and to stimulate knowledge acquisition and awareness of how these affect patients and health professionals. This model of communication allows participants to focus on their own perceived areas of difficulty and makes the course work pertinent to their needs.11
We have used these types of “trigger” tapes successfully in our previous research with nurses and doctors working in oncology.4 11 The intervention reported here was valued highly by all participants, who rated the course as useful, interesting, informative, and enjoyable, and all stated they would recommend participation to their colleagues.
The high priority given by the UK government to increase patient recruitment into clinical trials alerted the national cancer networks in England, Scotland, and Wales to health professionals' need for training in this aspect of communication. The findings from our study confirm that healthcare professionals can benefit from structured advice and guidance about communicating about trials. Although several participants had undergone some communication skills training, none had received specific training on how to talk about trials.
The positive findings from the course included an increase in participants' reported self confidence about recruiting patients into trials, and objective analyses revealed behavioural changes in the style and content of the participants' discussions. Some researchers have suggested that behaviours changed during a training course might not be transferred into clinical practice without support or consolidation courses,12 13 but there is strong evidence that if both competence and self confidence are improved then behavioural changes often do transfer successfully into the clinical setting and endure.11 14
Our training course is now being rolled out by the national cancer research networks in England and Wales, and research to see if real patient outcomes are affected is planned. Further modules are being developed to deal with types of trials not depicted in the existing modules—namely those for paediatric and adolescent patients, screening and prevention trials, multi-arm trials, and phase I studies.
What is already known on this topic
Worldwide, few potentially eligible patients are approached about entry into clinical trials; healthcare professionals find discussing trials and obtaining truly informed consent difficult
Patients are often confused or unclear about the experimental nature of treatment in trials
What this study adds
A training course was designed specifically to help health professionals provide clear information about phase III randomised trials of cancer treatments to patients and to encourage them to approach all eligible patients for recruitment
The course increased participants' reported self confidence about recruiting patients into trials, and objective analyses revealed improvements in the style and content of the participants' discussions
Further details of the conditional logistic regression used in the study are on bmj.com
Acknowledgments
We thank members of the NCRN Consumer Liaison Group for their generous contributions and agreement to be filmed together with Dr Rob Glynne Jones, Professor Robert Leonard, Professor Peter Selby, Professor Stephen Spiro, Professor Pierre Guillou, Dr David Bloomfield, Dr Fiona McKinna, Dr Joanne Simpson, nurse lead for the WCTN Libby Batt, trials manager Nicky Gower, and Louise Leach. Mark Mansell and Roland Brinton from Visual Image Publishing produced the series with us. Finally we thank Role Call actors, all the healthcare professionals who participated in the study, and members of Sussex Psychosocial Oncology Group who assisted in the smooth running of the courses, especially Sue Catt, Karen Nicholls, Val Shilling, Lou Atkins, Leigh Johnson, Mel Price, Tracy Woodcock, and Louise Leach.
Footnotes
-
Contributors VJ and LF designed the educational training package. VJ, LF, and VF designed the evaluation project. CL was the data manager for the project. IS-T and VF conducted the statistical analysis. All authors contributed to writing the article. VJ is guarantor for the study.
-
Funding Cancer Research UK funded the project, and AstraZeneca provided an unrestricted educational grant to help video production.
-
Competing interests None declared.
-
Ethical approval None required.