Artificial intelligence (AI) in medicine

AI is defined as a computer system that can achieve tasks that require making observations, evaluating information and reaching decisions.1 Acting intelligently involves responding to specific circumstances, being flexible to changing environments and learning from experience (Poole, p. 1).2

AI researchers predict that, with increasing computer power, AI will become a pervasive part of our everyday lives: ‘AI will be like electricity…an essential but invisible component of all our lives’ (Walsh, p. 60).3 This, of course, includes health. Uses of AI in health are developing rapidly, including uses for AI in medicine. Recent media coverage has highlighted the emergence of AI in the clinic, with headlines like ‘A.I. versus M.D.’ and ‘Artificial intelligence: augmenting or replacing doctors?’.4 5

Diagnosis and treatment recommendations are two aspects of medical care for which AI systems are actively being developed. In relation to diagnosis, studies in the past 2 years have shown the potential power of AI in increasing diagnostic accuracy in several clinical contexts. For example, in dermatology, a computer was trained to classify skin cancers and compared with 21 expert dermatologists; the computer ‘achieved performance on par with all tested experts’.6 Treatment recommendations are another medical task being tackled by AI. IBM, Microsoft and Google are among the companies developing AI systems that generate treatment recommendations.7 IBM’s tool ‘Watson for Oncology’ is one example of this type of system. In order to generate ranked treatment options for an individual patient with cancer based on his or her clinical details, the system conducts a vast analysis of published evidence searching for relevant studies and draws on its training by expert oncologists.8 9 As described by IBM’s research manager in a recent Australian documentary, Watson for Oncology ‘brings together disparate data sources and is able to learn and reason’ and ‘generate treatment recommendations’.10

Over the last decade as these systems have been developed, they have not generated a concurrent substantial debate among bioethics scholars. There is a very limited amount of bioethics work about medical AI. The 2014 text Machine Medical Ethics is a substantial conceptual contribution and there are recent discussions in clinical journals.11–14 There are small bodies of bioethics work in technology journals, often focusing on specific technologies such as surgical robots.15 There are also recent reports from the Nuffield Council on Bioethics and the Wellcome Trust.16 17 Robots for daily caring tasks, in contrast to medical AI, is a more active area of bioethical discussion.18 An in-depth bioethical debate about AI in medical treatment is yet to develop: using the search term ‘artificial intelligence’ OR ‘machine learning’ in Google Scholar on 1 July 2018 identified only eight articles published in the top 15 bioethics journals in the last decade about the ethics of using AI in patients’ medical care (further details of this search are attached as a online supplementary file).

In this paper, I focus specifically on AI for medical treatment decision making and consider the relationship between these systems and the ethical ideal of shared decision making in healthcare. I argue that involving AI in ranking treatment options poses a significant threat to patient autonomy. Unless AI systems are carefully designed to be value-flexible and thus responsive to individual patients’ treatment goals, we risk a shift back to more paternalistic medicine in a different guise. I suggest that use of these systems also offers potential opportunities for facilitating shared decision making. Bioethicists need to contribute to discussions about AI in medicine, to ensure that these tools are designed and used in an ethically informed way that promotes shared decision making. Having helped medical culture progress beyond ‘doctor knows best’, bioethicists urgently need to engage with the emergence of AI in medical treatment in order to avoid a new era in which ‘computer knows best’.

From paternalism to shared decision making

The paradigm of ‘doctor knows best’ was a pervasive one in medicine for many decades. Much of the early work of bioethics as a discipline was changing the conceptual landscape from one of paternalism to one that recognised patients as autonomous and entitled to be involved in their own healthcare decision making.19–21 The move from paternalism to respect for autonomy is encapsulated in the 2017 changes to the World Medical Association’s Declaration of Geneva, put forward as a modern articulation of the Hippocratic Oath: the latest version now specifically refers to respecting patient autonomy by including the pledge ‘[a]s a member of the medical profession… I will respect the autonomy and dignity of my patient’.22 23 While not universally valued around the globe, the concept of patient involvement in medical decisions is fundamental to frameworks of patient-centred care, which are now widespread in Western healthcare services.24

Shared decision making is now broadly accepted as the ethically appropriate approach to medical decision making in these settings.25 26 It is seen as a well-justified combination of physician responsibility and patient autonomy, avoiding the extremes of either medical paternalism or giving patients sole decision-making power. Whitney, for example, while raising important questions about exactly what is shared in shared decision making, writes that:

[T]he general concept of shared decision making has great appeal. It recognizes our conviction that patient and physician are making common cause against illness and suffering and does not relegate the physician to the role of technician… This approach, with various refinements, is widely acknowledged as a reasonable middle ground. (Whitney, p. 96)27

Shared decision making reflects a commitment to respecting patients as individuals entitled to control their health and participate in their healthcare decision making.

There is a variety of specific models of shared decision making, alongside discussion about the disparate ways in which the term is used.28 29 However, there are two ideas that are fundamental to the concept of shared decision making and particularly relevant in the context of thinking about AI in medicine. The first is that medical decisions involve values and preferences, not just clinical information. The second is the related idea that the best treatment for one patient may not be the best treatment for another patient in the same clinical situation because treatment should be shaped ultimately by the patient’s values. For example, Charles and colleagues, in their seminal article on shared decision making, write that ‘the choice of the best treatment for a particular patient requires value judgements on the part of the patient and physician’ and that doctors need to guard against imposing their own values on patients (pp. 682, 687).25 The key idea is that different medical decisions are right for different people based on patient values. Shared decision making is most straightforward when the options involve similar risks and benefits. In other types of situations, shared decision making is much more complex: for example, when there is a lack of relevant medical evidence or when a patient declines an intervention that is clearly necessary from the clinician’s perspective.

Different writers emphasise different aspects of shared decision making. Some conceptualise shared decision making primarily as a process of information exchange in which doctors contribute their medical knowledge and patients contribute their values and situational details (eg, Frankel, p. 110).30 Others, such as Brock, emphasise joint exploration of values, including critical reflection: ‘the physician can have a responsibility to explore, together with the patient, the values by which alternatives should be evaluated’ (p. 39).26 Gillick advocates reframing shared decision making in terms of elucidating the patient’s goals of care rather than choosing a treatment.31 Despite these differences, there is consensus on the fundamental ethical idea that an individual patient’s values play a role in determining what constitutes the best medical pathway for him or her.

AI as a risk to shared decision making

AI systems that recommend treatment options present a potential threat to shared decision making, because the individual patient’s values do not drive the ranking of treatment options. In this section, I will focus on IBM’s system Watson for Oncology, as a useful concrete example for exploring the risks and opportunities of medical AI in relation to shared decision making. In terms of AI systems designed for medical treatment decision making, Watson for Oncology is the most widely discussed in the public domain.

Watson for Oncology is designed and marketed as a tool for clinicians, to assist them to ‘zero in on the most promising care options’ in an age where the available literature on cancer is huge and fast-moving.8 The system extracts clinical information from the patient’s medical record, such as gender, age, stage and type of cancer, family history, notes from previous visits, test results and comorbidities. The doctor is prompted to verify this extracted information and add additional relevant information. The information is then analysed, based on the computer’s training by oncologists at Memorial Sloan Kettering Cancer Center in New York. The system accesses over 300 medical journals and over 200 textbooks.8 According to IBM’s marketing video, Watson ‘identifies a prioritized list of treatment options based on Memorial Sloan Kettering expertise and training, and provides links to supporting evidence’.8

Watson presents a ranked list of treatment options and a synthesis of the existing published evidence relevant to that clinical situation. The treatment options are divided into three colour-coded sections: green for recommended treatments, amber for treatments to consider and red for treatments that are not recommended. There may be multiple options in each section. The options are ranked based on outcome statistics presented in terms of ‘disease-free survival’.

For each treatment option, there are two literature tabs available to the clinician. One tab gives links to literature that supports that treatment option, identified by Memorial Sloan Kettering clinicians. The other tab gives links to Watson-identified literature relevant to that clinical situation. Information on toxicities associated with the treatment (such as vomiting, anaemia, diarrhoea and so on) is also available to clinicians. The system also has the capacity to be customised to the specific geographic context where it is being used. For example, local clinical guidelines and availability of drugs are included for clinicians.

Currently, Watson for Oncology ranks treatment options based on a particular value: maximising lifespan. The value set driving Watson’s rankings is not determined by the individual patient being treated. Rather, it is generic and essentially covert. So there are two main reasons to see these types of AI systems as a potential threat to shared decision making. First, the values driving the treatment rankings are not specific to the individual patient. We know that patients’ values differ. Not all patients aim exclusively for longevity in their treatment choices. As Liu and colleagues highlight, ‘[o]ne patient with a terminal disease may choose palliation; another will opt for further chemotherapy’ (p. 115).14 Second, these types of AI systems currently do not encourage doctors and patients to recognise treatment decision making as value-laden at all. There is a danger that the computer is seen as figuring out the right answer (Goodman, p. 132)32 rather than suggesting a treatment based on a particular set of generic parameters that may not in fact best reflect the goals, values and preferences of this specific patient.

It could be objected that patients’ values still play a role in treatment decision making using Watson for Oncology. Once the system has generated the ranked list of treatment options, the patient’s values are added into the decision making as a way of thinking about the ranked list. A rhetoric of individualised treatment and sharing decision making is certainly used in marketing Watson for Oncology. The system is presented as ‘provid[ing] new opportunities for cancer care providers to… confidently engage with patients on a more personalized basis, and make better, more informed care choices together’ (italics added).8 Potential users are told that ‘ after reviewing all the patient information and going through the treatment plans, the doctor now can easily share any of this information with her patient’.8 Watson makes information available to the clinician to support a conversation with the patient about the options: evidence about each option’s outcomes, side effects and timeframes. Potentially, at this point in the decision-making process, the patient receives full information about the different options and the clinician and patient discuss which approach would be best for that individual.

However, this type of process is fundamentally different to shared decision making. Respect for patient autonomy means that patients’ values should drive the ranking process. The patient’s own values should be overtly shaping treatment decision making as a primary parameter, not a secondary consideration. Patient values should not be discussed as a reaction to an already ranked list. Such an approach diminishes the patient’s role and represents a backward step in respecting patient autonomy. While ranking treatment options on the basis of a generic value set is problematic, the possibility of an AI system that made a singular recommendation would be even more worrying from an ethical perspective. Of course, in current clinical practice, treatment decision making by humans does not always meet the ideal of shared decision making. However, as new technologies are developed, we should be aiming to facilitate shared decision making to the greatest possible extent.

Towards value-flexible AI in medicine

In discussions of AI more broadly, it is widely recognised that AI systems make decisions that involve values. As well as industry-based initiatives to explore value-based questions around AI,33–35 there is an increasing body of academic literature arguing that the values built into AI systems in all contexts should be thoughtfully chosen and explicit.36 37

An important concept that has emerged from these discussions is ‘value sensitive design’.38 Value sensitive design is an ‘approach to the design of technology that accounts for human values in a principled and comprehensive manner throughout the design process’ (Friedman, p. 69).³⁸ The word ‘value’, in this context, ‘refers to what a person or group of people consider important in life’ (Friedman, p. 69)38 This approach involves integrating three types of investigation: conceptual, empirical and technical. Designers consider conceptual questions about values, such as:

Who are the direct and indirect stakeholders affected by the design at hand?… What values are implicated? How should we engage in trade-offs among competing values in the design, implementation, and use of information systems (eg, autonomy vs. security, or anonymity vs. trust)? (Friedman, p. 72)38

This analysis is integrated with findings from empirical investigations of the specific human social context in which the technology will be situated. Technical investigations aim to design systems that support the values identified in answering the conceptual questions (Friedman, p.72–73).38

However, the values included in value sensitive design discussions to date are primarily shared values. Discussions of value sensitive design sometimes highlight value diversity among individuals, but the primary focus is on shared values. For example, Dignum writes that ‘[e]ach individual and socio-cultural environment prioritizes different moral and societal values’ (p. 4702)36 but then focuses only on how to identify prevailing ‘societal values’ or ‘community values’ (p. 4702).36 In her ethical investigation of healthcare robots, van Wynsberghe puts forward a ‘care-centred value sensitive design approach’, defined as ‘a prospective methodology for designing a care robot in a manner that incorporates the core care values of attentiveness, responsibility, competence and reciprocity’ (p. 102).18 The values are those that relate in general to a particular domain of activity, rather than variable with the specific individual engaging with the system.

For AI systems in medical treatment, we need more than design that is attentive to shared or community values. We need value-flexible design: AI systems that allow for diversity among the values of individual users and can incorporate different values into decision making based on the specific user. The challenge is not merely to articulate the social values that should be incorporated into an AI system that ranks treatment options, but rather to design AI systems that are sufficiently flexible to enable the individual patient’s own values to drive decision making in his or her own case. To respect patient autonomy and facilitate shared decision making, we need AI systems that acknowledge and accommodate the diversity of values among the patient population. To avoid a return to more paternalistic medical care, the value basis of the system’s decision making must be sufficiently flexible to be individualised.

Adopting this approach to design could turn AI into an opportunity to enhance and embed shared decision making in practice. There is of course a substantial gap between the wide acknowledgement of shared decision making as the ethically appropriate approach in policy documents and bioethics literature, and the universal adoption of this approach in medical practice. With or without AI, current treatment decisions are not always shared. As research has highlighted, there are many challenges to shared decision making in practice; barriers include patient factors, clinician factors and system-related issues.31 Couët and colleagues have highlighted that ‘few health care providers consistently attempt to facilitate patient involvement, and even fewer adjust care to patient preferences’ (p. 543).39 Models for systematically integrating patient preferences into medical decision making are being explored.40 41 Well-designed AI systems could be used as a tool to prompt doctors and patients to discuss treatment goals and articulate the patient’s values relevant to the decision at hand. So while AI systems pose a significant threat to shared decision making, value-flexible design could enable such systems ultimately to promote shared decision making. Value-flexible AI in the clinic could become an ally in facilitating shared decision making and embedding respect for patient autonomy into clinical practice.

While value sensitive design case studies usually focus on shared values, there are case discussions that indicate that value flexibility is possible. Borning and colleagues describe designing a system for an urban development context, involving conflict between stakeholders: ‘a dispute-filled environment’ where there are ‘multiple stakeholders with strongly held, divergent values’ (p. 2, 18).42 This discussion provides some important resources for thinking about situations like medical treatment, where individual values differ. Borning and colleagues write that:

[I]n the present project, we made a sharp distinction between explicitly supported values (ie, ones that we explicitly want to support in the simulation) and stakeholder values (ie, ones that are important to some but not necessarily all of the stakeholders)…[W]e decided that the system should not a priori favor or rule out any given set of stakeholder values, but instead should allow different stakeholders to articulate the values that are most important to them, and evaluate the alternatives in light of these values. (Borning, p. 5)42

This approach acknowledges value diversity and attempts to accommodate differences among individuals. Translated into the medical clinic context, this flexible individualised approach would potentially enable AI systems to support shared decision making.

Thinking in terms of value-flexible AI in medicine invites many new questions. How exactly would a valuable-flexible system function in the clinic? Could such a system be designed to address the known barriers to shared decision making? Could an AI system potentially adapt to changes in a patient’s priorities along their illness trajectory? Could it assist with navigating treatment decision making when doctors and patients disagree? Overall, the fundamental challenge is to incorporate our best conceptual and empirical knowledge about shared decision making in the clinic into the design of these systems.