Trading off accuracy and explainability in AI decision-making: findings from 2 citizens' juries.

OBJECTIVE: To investigate how the general public trades off explainability versus accuracy of artificial intelligence (AI) systems and whether this differs between healthcare and non-healthcare scenarios. MATERIALS AND METHODS: Citizens' juries are a form of deliberative democracy eliciting informed judgment from a representative sample of the general public around policy questions. We organized two 5-day citizens' juries in the UK with 18 jurors each. Jurors considered 3 AI systems with different levels of accuracy and explainability in 2 healthcare and 2 non-healthcare scenarios. Per scenario, jurors voted for their preferred system; votes were analyzed descriptively. Qualitative data on considerations behind their preferences included transcribed audio-recordings of plenary sessions, observational field notes, outputs from small group work and free-text comments accompanying jurors' votes; qualitative data were analyzed thematically by scenario, per and across AI systems. RESULTS: In healthcare scenarios, jurors favored accuracy over explainability, whereas in non-healthcare contexts they either valued explainability equally to, or more than, accuracy. Jurors' considerations in favor of accuracy regarded the impact of decisions on individuals and society, and the potential to increase efficiency of services. Reasons for emphasizing explainability included increased opportunities for individuals and society to learn and improve future prospects and enhanced ability for humans to identify and resolve system biases. CONCLUSION: Citizens may value explainability of AI systems in healthcare less than in non-healthcare domains and less than often assumed by professionals, especially when weighed against system accuracy. The public should therefore be actively consulted when developing policy on AI explainability.

Should free-text data in electronic medical records be shared for research? A citizens' jury study in the UK.

BACKGROUND: Use of routinely collected patient data for research and service planning is an explicit policy of the UK National Health Service and UK government. Much clinical information is recorded in free-text letters, reports and notes. These text data are generally lost to research, due to the increased privacy risk compared with structured data. We conducted a citizens' jury which asked members of the public whether their medical free-text data should be shared for research for public benefit, to inform an ethical policy. METHODS: Eighteen citizens took part over 3 days. Jurors heard a range of expert presentations as well as arguments for and against sharing free text, and then questioned presenters and deliberated together. They answered a questionnaire on whether and how free text should be shared for research, gave reasons for and against sharing and suggestions for alleviating their concerns. RESULTS: Jurors were in favour of sharing medical data and agreed this would benefit health research, but were more cautious about sharing free-text than structured data. They preferred processing of free text where a computer extracted information at scale. Their concerns were lack of transparency in uses of data, and privacy risks. They suggested keeping patients informed about uses of their data, and giving clear pathways to opt out of data sharing. CONCLUSIONS: Informed citizens suggested a transparent culture of research for the public benefit, and continuous improvement of technology to protect patient privacy, to mitigate their concerns regarding privacy risks of using patient text data.

Commercial use of health data-A public "trial" by citizens' jury.

INTRODUCTION: Surveys suggest a dichotomy in how citizens view research for public benefit and research for commercial gain. Therefore, a research initiative, such as a learning health system, which works for both public and commercial benefit, may be controversial and lower public trust. METHODS: This study aimed to investigate what informed citizens considered to be appropriate uses of health data in a learning health system and why they made those decisions. Two-paired 4-day juries were run, with different jurors but the same purpose, expert witnesses, and facilitators. Overall, 694 people applied; 36 jurors were selected to match criteria based on demographics and privacy views. Jurors considered whether and why eight exemplars of anonymised patient data were acceptable. The exemplars were either planned initiatives to improve care pathways (Planned Examples) or possible commercial data uses (Potential Examples). RESULTS: These citizens' juries found that all Planned and two of the Potential Examples were considered appropriate by most, but not all, jurors because they could deliver public benefit. In general, positive health outcomes for patients were more acceptable than improved efficiency of services for the NHS, although they recognised that the latter also improved health. Jurors had concerns about whether improving efficiency would lead to inequitable distribution or closure of services, based on their existing understanding from media reports. Commercial gain that accrued secondary to this benefit was acceptable, with some jurors becoming more accepting of commercial uses as they understood them better. Prioritising profit, however, was unacceptable, regardless of any governance arrangements. CONCLUSIONS: Jurors tended to be more accepting of data sharing to both private and public sectors after the jury process. Many jurors accept commercial gain if public benefit is achieved. Some were suspicious of data sharing for efficiency gains. Juries elicited more informed and nuanced judgement from citizens than surveys.

Investigating the Extent to Which Patients Should Control Access to Patient Records for Research: A Deliberative Process Using Citizens' Juries.

BACKGROUND: The secondary use of health data for research raises complex questions of privacy and governance. Such questions are ill-suited to opinion polling where citizens must choose quickly between multiple-choice answers based on little information. OBJECTIVE: The aim of this project was to extend knowledge about what control informed citizens would seek over the use of health records for research after participating in a deliberative process using citizens' juries. METHODS: Two 3-day citizens' juries, of 17 citizens each, were convened to reflect UK national demographics from 355 eligible applicants. Each jury addressed the mission "To what extent should patients control access to patient records for secondary use?" Jurors heard from and questioned 5 expert witnesses (chosen either to inform the jury, or to argue for and against the secondary use of data), interspersed with structured opportunities to deliberate among themselves, including discussion and role-play. Jurors voted on a series of questions associated with the jury mission, giving their rationale. Individual views were polled using questionnaires at the beginning and at end of the process. RESULTS: At the end of the process, 33 out of 34 jurors voted in support of the secondary use of data for research, with 24 wanting individuals to be able to opt out, 6 favoring opt in, and 3 voting that all records should be available without any consent process. When considering who should get access to data, both juries had very similar rationales. Both thought that public benefit was a key justification for access. Jury 1 was more strongly supportive of sharing patient records for public benefit, whereas jury 2 was more cautious and sought to give patients more control. Many jurors changed their opinion about who should get access to health records: 17 people became more willing to support wider information sharing of health data for public benefit, whereas 2 moved toward more patient control over patient records. CONCLUSIONS: The findings highlight that, when informed of both risks and opportunities associated with data sharing, citizens believe an individual's right to privacy should not prevent research that can benefit the general public. The juries also concluded that patients should be notified of any such scheme and have the right to opt out if they so choose. Many jurors changed their minds about this complex policy question when they became more informed. Many, but not all, jurors became less skeptical about health data sharing, as they became better informed of its benefits and risks.

How can one be both a philosophical ethicist and a democrat?

How can one be both a philosophical ethicist and a democrat? In this article I conclude that it can be difficult to reconcile the two roles. One involves understanding, and reconciling, the conflicting views of citizens, and the other requires the pursuit of truth through reason. Nevertheless, an important function of philosophy and ethics is to inform and improve policy. If done effectively, we could expect better, and more just, laws and policies, thereby benefiting many lives. So applying philosophical thinking to policy is an important job. However, it comes with substantial difficulties, not least in reconciling, or choosing between, competing philosophical theories. Despite the importance of the task, and the apparent obstacles, there is relatively little literature on how to apply ethics to real-world policy-making. Democracies need ethicists who can engage in democratic debate and bridge the gap between philosophy and public policy. I offer some tactics here.

Should policy ethics come in two colours: green or white?

When writing about policy, do you think in green or white? If not, I recommend that you do. I suggest that writers and journal editors should explicitly label every policy ethics paper either 'green' or 'white'. A green paper is an unconstrained exploration of a policy question. The controversial 'After-birth abortion' paper is an example. Had it been labelled as 'green', readers could have understood what Giubilini and Minerva explained later: that it was a discussion of philosophical ideas, and not a policy proposal advocating infanticide. A serious policy proposal should be labelled by writer(s) and editor(s) as 'white'. Its purpose should be to influence policy. In order to influence policy, I suggest three essential, and two desirable, characteristics of any white paper. Most importantly, a white paper should be set in the context in which the policy is to be made and applied.