Trading off accuracy and explainability in AI decision-making: findings from 2 citizens' juries.

OBJECTIVE: To investigate how the general public trades off explainability versus accuracy of artificial intelligence (AI) systems and whether this differs between healthcare and non-healthcare scenarios. MATERIALS AND METHODS: Citizens' juries are a form of deliberative democracy eliciting informed judgment from a representative sample of the general public around policy questions. We organized two 5-day citizens' juries in the UK with 18 jurors each. Jurors considered 3 AI systems with different levels of accuracy and explainability in 2 healthcare and 2 non-healthcare scenarios. Per scenario, jurors voted for their preferred system; votes were analyzed descriptively. Qualitative data on considerations behind their preferences included transcribed audio-recordings of plenary sessions, observational field notes, outputs from small group work and free-text comments accompanying jurors' votes; qualitative data were analyzed thematically by scenario, per and across AI systems. RESULTS: In healthcare scenarios, jurors favored accuracy over explainability, whereas in non-healthcare contexts they either valued explainability equally to, or more than, accuracy. Jurors' considerations in favor of accuracy regarded the impact of decisions on individuals and society, and the potential to increase efficiency of services. Reasons for emphasizing explainability included increased opportunities for individuals and society to learn and improve future prospects and enhanced ability for humans to identify and resolve system biases. CONCLUSION: Citizens may value explainability of AI systems in healthcare less than in non-healthcare domains and less than often assumed by professionals, especially when weighed against system accuracy. The public should therefore be actively consulted when developing policy on AI explainability.

Should free-text data in electronic medical records be shared for research? A citizens' jury study in the UK.

BACKGROUND: Use of routinely collected patient data for research and service planning is an explicit policy of the UK National Health Service and UK government. Much clinical information is recorded in free-text letters, reports and notes. These text data are generally lost to research, due to the increased privacy risk compared with structured data. We conducted a citizens' jury which asked members of the public whether their medical free-text data should be shared for research for public benefit, to inform an ethical policy. METHODS: Eighteen citizens took part over 3 days. Jurors heard a range of expert presentations as well as arguments for and against sharing free text, and then questioned presenters and deliberated together. They answered a questionnaire on whether and how free text should be shared for research, gave reasons for and against sharing and suggestions for alleviating their concerns. RESULTS: Jurors were in favour of sharing medical data and agreed this would benefit health research, but were more cautious about sharing free-text than structured data. They preferred processing of free text where a computer extracted information at scale. Their concerns were lack of transparency in uses of data, and privacy risks. They suggested keeping patients informed about uses of their data, and giving clear pathways to opt out of data sharing. CONCLUSIONS: Informed citizens suggested a transparent culture of research for the public benefit, and continuous improvement of technology to protect patient privacy, to mitigate their concerns regarding privacy risks of using patient text data.

Investigating the Extent to Which Patients Should Control Access to Patient Records for Research: A Deliberative Process Using Citizens' Juries.

BACKGROUND: The secondary use of health data for research raises complex questions of privacy and governance. Such questions are ill-suited to opinion polling where citizens must choose quickly between multiple-choice answers based on little information. OBJECTIVE: The aim of this project was to extend knowledge about what control informed citizens would seek over the use of health records for research after participating in a deliberative process using citizens' juries. METHODS: Two 3-day citizens' juries, of 17 citizens each, were convened to reflect UK national demographics from 355 eligible applicants. Each jury addressed the mission "To what extent should patients control access to patient records for secondary use?" Jurors heard from and questioned 5 expert witnesses (chosen either to inform the jury, or to argue for and against the secondary use of data), interspersed with structured opportunities to deliberate among themselves, including discussion and role-play. Jurors voted on a series of questions associated with the jury mission, giving their rationale. Individual views were polled using questionnaires at the beginning and at end of the process. RESULTS: At the end of the process, 33 out of 34 jurors voted in support of the secondary use of data for research, with 24 wanting individuals to be able to opt out, 6 favoring opt in, and 3 voting that all records should be available without any consent process. When considering who should get access to data, both juries had very similar rationales. Both thought that public benefit was a key justification for access. Jury 1 was more strongly supportive of sharing patient records for public benefit, whereas jury 2 was more cautious and sought to give patients more control. Many jurors changed their opinion about who should get access to health records: 17 people became more willing to support wider information sharing of health data for public benefit, whereas 2 moved toward more patient control over patient records. CONCLUSIONS: The findings highlight that, when informed of both risks and opportunities associated with data sharing, citizens believe an individual's right to privacy should not prevent research that can benefit the general public. The juries also concluded that patients should be notified of any such scheme and have the right to opt out if they so choose. Many jurors changed their minds about this complex policy question when they became more informed. Many, but not all, jurors became less skeptical about health data sharing, as they became better informed of its benefits and risks.