Changing modes of interaction and their entanglement with different normative notions
| Normative notions | Interaction modes | ||
| Clinician–AI-DSS | Clinician–patient | Patient–AI-DSS | |
| Agency |
Description: introduction of AI-DSS competence and authority alongside clinician competence. Challenge: spectrum ranges from augmentation to replacement. Consequence: need to prioritise and to mediate between clinician and AI-DSS judgements in case they diverge. |
Description: introduction of additional evidence at the centre of shared decision-making. Challenge: finding context-sensitive modes of participation in collaborative evaluation of outputs. Consequence: need for enhanced and standardised sensitivity to clinician and patient attitudes and expectations about roles in decision-making informed by AI-DSS. |
Description: interplay between AI-DSS ideally ameliorating individual patient care and patient data learning and refining AI-DSS. Challenge: sharing of large amounts of health data as a necessary condition for using medical AI. Consequence: need for mechanisms to exercise informational self-determination through awareness of available options, data flow controllability and privacy by design. |
| Trustworthiness |
Description: room for errors across data collection, model training and implementation; significance of AI-DSS reliability, validity and attunement to task. Challenge: user-friendliness, safety and efficacy, potential burden of proof for clinician who deviates from AI-DSS recommendations. Consequence: need for clinician training to evaluate reliability, validity and attunement. |
Description: perceived transformations to clinical decision-making processes. Challenge: shifts and extensions of conceptions of clinician competence and communication, expectation that human clinician remains in the loop. Consequence: need for new forms of counselling and communication pathways. |
Description: emerging conditions of trustworthiness for deployment of AI-driven technology from the patient perspective. Challenge: varying degrees of background knowledge and openness for AI-driven clinical tools, possibility that contact to human clinicians becomes a luxury. Consequence: need for provision of clear information on opportunities, challenges, data protection, procedures and addressees for damage claims, and humans in the loop. |
| Transparency |
Description: AI uncovering correlations without necessarily eliciting underlying causal relations. Challenge: lack of evidence on how and why AI-DSS arrives at a given output. Consequence: need for safety and efficacy validation and certification markers. |
Description: black box issues coupled with information asymmetries between clinician and patient. Challenge: asymmetries in ability to assess and reflect on AI-DSS outputs, challenges in attributability of recommendations. Consequence: providing information and education about AI-driven clinical decision-making, clarifying the role and competence of the clinician relative to the AI-DSS. |
Description: black box issues aggravated by lack of clinical and technical background. Challenge: making the meaning, quality and limitations of AI-DSS outputs transparent to patients. Consequence: need for provision of clear information on the nature of AI-DSS outputs, for example, through visualisations and innovative patient interfaces. |
| Responsibility |
Description: partial shift of responsibility for diagnosis, recommendations and decision-making from clinician towards AI-DSS. Challenge: problem of many hands in AI development and implementation, complicating attributions of responsibility for malfunctions. Consequence: need for clear and context-sensitive principles for how ethical and legal responsibility distributes across multiagential structures. |
Description: changing nature of clinician responsibility in view of informational asymmetries between clinician and patient. Challenge: collaboratively assessing risk-benefit ratios. Consequences: need for training and education for clinicians and patients towards responsible utilisation of AI. |
Description: consideration of and reliance on AI-DSS outputs when patients make decisions on their own authority,5 paternalism versus new forms of individual choices. Challenge: mediating between self-interest, solidaric participation and backlashes at the level of justice. Consequence: need for education and counselling. |
AI-DSS, artificial intelligence-driven decision support system.