The use of vignettes within a Delphi exercise: a useful approach in empirical ethics?
- 1Kingston University and St George's University of London, Faculty of Health and Social Care Sciences, Kingston, UK
- 2International Centre for Nursing Ethics, Division of Health and Social Care, University of Surrey, Guildford, UK
- Correspondence to Dr Ann Gallagher, International Centre for Nursing Ethics, Division of Health and Social Care, Faculty of Health and Medical Sciences, Duke of Kent Building, University of Surrey, Guildford, Surrey GU2 7TE, UK;
- Received 11 March 2010
- Revised 7 June 2010
- Accepted 13 June 2010
- Published Online First 16 August 2010
There has been an increase in recent years in the use of empirical methods in healthcare ethics. Appeals to empirical data cannot answer moral questions, but insights into the knowledge, attitudes, experience, preferences and practice of interested parties can play an important part in the development of healthcare ethics. In particular, while we may establish a general ethical principle to provide explanatory and normative guidance for healthcare professionals, the interpretation and application of such general principles to actual practice still requires interpretation and judgement. And many situations in healthcare practice are complex and may involve a variety of principles, each of which may conflict with the others. Simple surveys or interview studies may not be sufficient if we wish to develop a nuanced approach to ethical practice that can be set out in guidelines, codes or directives. We do not resolve moral questions by plebiscite. In this paper, the authors argue for the use of consensus methods to develop shared understanding of ethical practice, and they argue further for the combination of the Delphi method with the use of vignettes to illustrate the kind of situations that may occur in practice. They develop their argument in part by reference to their experience of using this approach in their recent research.
The use of empirical methods in healthcare ethics has attracted attention in recent years. Writing in 1999, Tony Hope1 took the view that, far from being a threat or a problem, this was ‘a healthy development in the maturing subject of medical ethics’ (p219). Since then there has been a further expansion of research in empirical ethics. As Hope observes, the question of whether something is right or wrong is ‘a philosophical, not an empirical, issue’ and such questions cannot be resolved by reference to empirical research. But, as Hope goes on to argue, philosophical argument often depends on empirical issues. And of course, while empirical data cannot resolve moral questions, it can be helpful to know what people think about such questions, how in practice they interpret moral norms, and how they approach ethical decision making when faced with complex questions in practice.
If we agree that empirical work has something to offer healthcare ethics, we then have to consider the question of what kind of empirical work and, in particular, what sort of research methods might be appropriate. Both qualitative and quantitative methods have been used in empirical ethics studies. Borry et al2 report that between 1990 and 2003 three ethics journals between them published 435 papers reporting empirical studies, of which 64% used quantitative methods. Cooper et al3 on the other hand, in a review of empirical ethics in the field of pharmacy practice, noted that over a 19-year period ‘Many of the earlier studies used a questionnaire that contained hypothetical ethical scenarios from which respondents selected options. This allowed statistical analysis of pharmacists′ ethical problems and reasoning, whereas almost all later studies adopted interview or focus group methods’ (p83). Such approaches are useful in the descriptive sense, in that they can give us snapshots of opinion among a population and insights into the experiences of individuals and groups. But such approaches do not generally lead to any kind of consensus, as opposed to simple majority views.
In this paper, we describe the use of a combination of vignettes with the Delphi method. This method was used in a study exploring conflicts of interest for general practitioners (GPs) in safeguarding children in the UK. The study has been reported elsewhere4 and we do not report detailed findings here, but we do use some examples from our data to illustrate the methodological argument.
Consensus methods in empirical ethics research
Consensus methods have been reported in the ethics literature, but they are rare, compared with other empirical methods. The two consensus approaches most often described in the general research literature are the Delphi method and the nominal group procedure; both are mentioned in ethics publications. Within these methods the use of vignettes is a popular approach to focusing the responses of participants and elicit information.
The Delphi technique has been used since the 1950s as a means to obtain a consensus of expert opinion in relation to complex
issues or problems. Sackman5 (page xi) describes it as follows:
Delphi is an attempt to elicit expert opinion in a systematic manner for useful results. It usually involves iterative questioning
administered to individual experts in a manner protecting the anonymity of their responses. Feedback of results accompanies
each iteration of the questionnaire, which continues until convergence of opinion, or a point of diminishing returns, is reached.
The end product is the consensus of experts, on each of the questionnaire items, usually organised as a report by the Delphi
The Delphi method is not without its critics (see, eg, Adler and Ziglio6). There are disagreements as to how one defines and recruits experts and concerns that the consensus process may force agreement when perhaps it is not really present. It is always possible that an important insight from one or two panel members may become overwhelmed by less perceptive but more popular points of view. The identity of individual members is normally concealed from the rest of the panel, and, this coupled with the lack of any direct contact between members, rules out the possibility of group interaction of the kind that might be expected in a focus group. However, the method is widely accepted, as can be seen from the numerous examples in the literature.
Finch7 (p105) describes vignettes as ‘short stories about hypothetical characters in specified circumstances, to whose situation
the interviewee is invited to respond’. Barter and Renold8 suggest that, in qualitative research, participants are usually asked to respond to a particular situation by stating what
they would do, or how they imagine a third person, generally a character in the story, would react to certain situations or
occurrences, which often entail some form of moral dilemma. Rapport et al9 (p39) describe the use of vignettes
to shift attention from the research participant onto an unknown other. As such, research participants are offered a choice
regarding the degree to which they ‘enter’ themselves into the unfolding research story. In this way, they can opt to remove
themselves from direct association with the vignette being presented to offer an outsider perspective.
There is comparatively little discussion in the literature of the methodological issues around the use of vignettes. Bachmann et al10 discuss in some detail the question of the number of vignettes to be employed and the number of attributes associated with each vignette, reporting that, in their review, one-quarter of studies either used vignettes with more than nine attributes or sets of more than 40 vignettes. They suggest these figures are ‘very high and could bias results’, although there is little empirical evidence as to the optimum numbers of attributes or vignettes. In our own brief review of the literature of studies using both vignettes and Delphi, one paper11 reports that ‘Two hundred eighty-eight vignettes [of non-steroidal anti-inflammatory prescribing] were evaluated for the appropriateness of each of six options’, a cognitive workload that does seem daunting.
There is little discussion also of the process of construction of vignettes or their limitations as useful representations of reality. Hughes and Huby12 argue that ‘To varying degrees, vignettes selectively simulate elements of the research topics under study… but, like any research tool, they can never mirror completely the reality and dynamism of people's lives’ (p383). They point out that the selection of material for inclusion in a vignette will inevitably privilege some aspects of a story over others. Vignettes are thus partial but somewhat static, and the reader or research participant will have to fill in a certain amount of detail from their own experience. This in turn will mean that each participant will do this slightly differently. However, Hughes and Huby also recognise that this selectivity can also be a benefit, providing a focus and helping to disentangle the complexities and conflicts of real life.
Use of the Delphi technique
There are numerous examples of Delphi studies reported in the healthcare literature. These include: as a method of choosing an allocation ratio in randomised controlled trials13; in a study of regionalisation and the treatment of disability and chronic illness14; with a panel of clinical bioethicists to investigate what might be the top 10 healthcare ethics problems faced by the public15; to develop an ethical framework for the use of cardiac report cards16; to develop consensus guidelines for analgesia and sedation in dying patients in ICU17; to explore the distinction between research and audit and the need for ethical review18; to investigate the impact of public health systems19; to develop a consultation protocol for use in cases of euthanasia20; and to explore ethical aspects of research into Alzheimer's disease.21
The use of vignettes
There is an extensive literature describing the use of vignettes in qualitative research, but rather less on their use in Delphi studies. In a systematic review of vignette studies of medical choice and judgement, Bachmann et al10 describe their search strategy as identifying 2001 records, of which 81 full papers were obtained and 30 papers included in the review. In recent general qualitative research, vignettes have been used to explore: the understanding of adjuvant therapy options for women with breast cancer22; the prescription of antibiotics for respiratory tract infections23; the use of deception in drug trials24; physicians' clinical decision making in diabetes25; drug prescribing for Japanese cedar pollinosis26; various aspects of medical education27–29; the management of hypertension30; mental health literacy31; patients' beliefs regarding medication32; psychiatric advance directives33; referral of patients to specialists by GPs34; stigma and depression35; decisions regarding the use of thrombolysis36; knowledge of tuberculosis among medical practitioners37; cost awareness of postoperative complications38; urinary incontinence following stroke39; the management of low back pain40; physician reporting41; the treatment of polymyalgia42; the management of chronic prostatitis43; perceptions of the cause of schizophrenia44; and many others.
Combining vignettes and Delphi
The use of vignettes with Delphi studies is less widely reported, but examples include the management of diabetes in adolescents45 and radiotherapy in the treatment of bone metastases.46 We found only one study in the ethics literature suggesting the combination of a Delphi approach with vignettes11; this paper gives extensive information about the vignettes used but does not describe the Delphi process employed in the study.
However, the combination of vignettes and Delphi would seem to be appropriate, given some similarities in the claims made
for their utility. Bachmann et al10 in their review argue that ‘Preferences and perceived similarities or differences between choice alternatives can be evaluated
using structured vignettes’ and go on to suggest that
These methods can be carried forward to the analysis of medical decision making, as medical decisions require judgement under
uncertainty. This uncertainty may concern a state, such as the presence of illness, the likelihood of future events, such
as those in the natural course of an illness, or the likelihood with which such events may be averted, that is, treatment
The proponents of both vignettes and Delphi would thus seem to agree that both methods are appropriate for exploring views and opinions in areas of uncertainty, particularly with regard to decision making. As Bachmann et al10 observe, much research into medical decision making has ‘explored physicians' estimations of probabilities’, but they note that there are doubts as to the consistency of their ratings of probabilities and there is research to suggest that physicians do not actually use probability ratings in the way assumed, but rather apply their own heuristics when making decisions. We would argue that, in addition to the difficulties of probability ratings even in situations where these might be appropriate, there are many aspects of healthcare in which probability in the statistical sense is of little help. GPs' decision making when faced with the possibility of child abuse or neglect is, we would argue, one such situation. The GP may be faced with a complex set of family relationships, difficult questions of inter-professional and inter-agency working, and evidence of neglect or abuse that is at best inconclusive. Any decision must of course include an assessment of risk, including estimates of probability (perhaps likelihood would be a better term here), but must also balance the conflicting demands generated by the practitioner's responsibility for the well-being of a family, both as a unit and as individuals, and in the context of relationships with other agencies and the relevant legislative framework. If decision making in areas where there is complexity and uncertainty can usefully be explored through vignettes and if Delphi is a suitable technique for exploring expert opinion and achieving consensus as to how to deal with areas of complexity and uncertainty, then using vignettes within the iterative, consensus-seeking Delphi process would seem a useful strategy.
We used four vignettes as part of our Delphi study exploring conflicts of interest experienced by GPs in the course of child protection work. Vignette 1 concerned a single mother of two preschool-age children, who attends the surgery describing symptoms of depression with suicidal ideation, and complaining that she is struggling to cope with life. In vignette 2, a 19-year-old woman, with two small children and a history of substance misuse, attends for an antenatal appointment, during which the GP notices bruising on the woman's arms. The woman is evasive about the cause, but talks about her partner believing in strong discipline and chastising her son by ‘giving him a good hiding’. Vignette 3 describes a couple, both of whom have learning difficulties. The wife has brought their 8-year-old daughter to the GP because she has an ear infection. While treating the daughter, the GP notes the child is overweight and has head lice. The final vignette described a 13-year-old girl who tells her GP that she thinks she is pregnant and wants information about an abortion. She says she has been having a sexual relationship with a family member for 2 years. She only wants an abortion and no other intervention; the GP has been her family's doctor for 20 years.
For each vignette, respondents were asked what issues arose, how the doctor should respond, why the doctor should respond in this way, and what might make a response difficult. In later rounds, they were asked to rank agreed responses in order of priority and say how quickly they should be carried out. None of these questions demand responses that deal explicitly with ethical issues, but all carry an implicit ethical dimension. All carry potential conflicts of interest, both between various characters and agencies and between short and longer term goals. In each vignette there is the possibility that a child is at risk of neglect or abuse and, as participants in our study were well aware, the UK legal position is that in such situations the child's welfare is paramount. However, determining what course of action would be appropriate in each of these cases is far from simple. Faced with a woman with mental health problems or a woman in an apparently abusive relationship, the doctor has an obvious responsibility to provide the best care for the woman. However, it is possible that, in either situation, the mother is not providing optimal care for her children and, in the latter case, the children may be at risk of harm from the woman's partner. Any of the options for management open to the doctor may have adverse consequences for the children, while any steps that might be taken to safeguard the children may have adverse consequences both for them and for their mother, in the short and the long term. And as our participants pointed out, if the doctor mishandles the situation, there is the real risk that the woman and her children may not return.
Vignette 3 raises issues of possible prejudice. Had the parents been middle class professionals, an ear infection might be a routine childhood problem, head lice are rife in many schools in all social strata, and obesity would perhaps trigger some advice about diet and exercise. The fact that both parents have learning disabilities does raise legitimate concerns, in that parents with disabilities may need additional help and support, but, as our respondents noted, it is important to maintain objectivity and a sense of proportion.
Vignette 4 is perhaps the most controversial and raises the most obvious moral concerns. We do not know who the ‘family member’ might be, but it could be a sibling or cousin close in age to the young woman, or it could be her father or other older male relative. In either case a criminal offence has, prima facie, been committed, and there are implications for future management and decision making. If a 13-year-old girl is pregnant but there is no suggestion of a family member being involved, it is quite possible that she might be referred for a termination, if she so wishes and if she is Gillick competent, with no wider involvement of her family (see, eg, ref47). If a young woman in these circumstances is adamant that she wants no other intervention or involvement of agencies other than a termination, the doctor must make a decision as to whether to respect her right to confidentiality or whether to take matters further, with all the inevitable consequences.
From the four vignettes, in response to the question ‘What should the GP do?’, a total of 58 responses reached consensus, although there was a certain amount of duplication. For example, in three of the four vignettes, the consensus was that there should be a referral to social services. Using simple qualitative thematic analysis, the 58 responses could be grouped as follows:
Professional judgement: 12
Team work: 9
Treating the patient: 8
Referring to other agency: 7
Listening carefully: 4
Explanation and information giving: 4
Seeking further information: 3
Risk management: 2
Establishing a relationship: 2
Record keeping: 2
Limiting confidentiality: 2
Assessing capacity: 2
Being non-judgemental: 1
It is interesting to note that the greatest number of responses referred to the need to exercise professional judgement in the management of the situation, while the second most frequent response category referred to the need for teamwork, involving other members of the primary healthcare team. The third category, ‘Treating the patient’, referred to treatment for the presenting condition—for example, prescribing antibiotics for the child's ear infection—while the fourth most popular category indicated the need to refer to other agencies, notably social services but also education authorities, the learning disability team, and so on. Thus the majority of responses were not concerned with technical, clinical decisions but with matters of judgement, many of which concerned ethical issues.
The standard answer to any question about the course of action to be followed in the event that neglect or abuse is suspected is that the welfare of the child is paramount, as set out in the Children Act (1989).48 In practice of course, outside of technically required interventions such as determining the correct dose of an appropriate antibiotic for a specific infection, the application of rules, principles and codes of practice requires judgement. Rules cannot usually be written so as to be so specific that they leave no room for judgement; the practitioner has to decide that a given occurrence is an exemplar of the kind of situation that requires a particular response. The responses of participants to the vignettes used in our study demonstrate the subtle and nuanced nature of practice in these complex situations. The Delphi process allowed each member of the panel to express their views, expose those views to the judgement of the other panel members, and review the responses of the other panel members. The intervention of the researchers allowed consensus to be achieved, including the modification of items that had not immediately achieved consensus in the light of comments and feedback. The result is a clearer understanding of the responses of practitioners and other experts to ethically challenging situations that would have been achieved from, for example, a simple survey or the use of vignettes alone.
There are of course limitations with our study and with the method in general. With hindsight, we used too many vignettes and too many questions in the whole Delphi exercise, which meant that we had rather more data than could be easily handled. The vignettes could have been more carefully worded. Vignettes are inevitably stylised, schematic, partial and may even be caricatures. However carefully and realistically a vignette is constructed, it is almost certain that no one else will ever experience an identical situation and thus one can rarely generalise from such findings. On the other hand, a well written vignette can be as effective as any other kind of case presentation or thought experiment, within the limitations of such approaches.
Embedding vignettes within a Delphi exercise renders the method open to all the criticisms of Delphi. One has to decide who should count as an expert in a given field of enquiry. The process of achieving consensus is contentious and may be forced. Discarding items that do not get general support from the panel means that an interesting and valuable response may be lost: the fact that only one person said something and that others disagreed does not necessarily negate the value of the comment. However, the iterative process of the Delphi approach does somewhat mitigate against this: a single response in a survey is likely to be disregarded, but circulating such a response to others does at least give it the chance of wider adoption.
The final caution is to repeat that we do not believe such processes can necessarily solve moral problems. The combination of vignettes with a consensus approach can, however, give us useful insights into how people will interpret ethical standards of practice in the context of specific situations and thus offers a useful addition to the repertoire of methods in empirical ethics research.
Professor Paul Wainwright died suddenly on 16 June 2010. He will be greatly missed by friends, colleagues and students. He made a significant contribution to applied ethics and medical humanities and to our understanding of concepts such as dignity, practice, conflicts of interest and professionalism. It has been a privilege to know and work with him and our thoughts are with his wife, Elsa, and family.
Funding The Department of Health and Department for Education and Skills.
Competing interests None.
Ethics approval This study was conducted with the approval of the Wandsworth NHS REC.
Provenance and peer review Not commissioned; externally peer reviewed.