What the public hopes and fears about the use of AI in health care

There has been increasing interest in using health “big data” for artificial intelligence (AI) research. As such, it is important to understand which uses of health data are supported by the public and which are not.

Previous studies have shown that members of the public see health data as an asset that should be used for research provided there is a public benefit and concerns about privacy, commercial motives and other risks are addressed.

However, this general support may not extend to health AI research because of concerns about the potential for AI-related job losses and other negative impacts.

Our research team conducted six focus groups in Ontario in October 2019 to learn more about how members of the general public perceive the use of health data for AI research. We found that members of the public supported using health data in three realistic health AI research scenarios, but their approval had conditions and limits.

Robot fears

Each of our focus groups began with a discussion of participants’ views about AI in general. Consistent with the findings from other studies, people had mixed — but mostly negative — views about AI. There were multiple references to malicious robots, like the Terminator in the 1984 James Cameron film.

“You can create a Terminator, literally, something that’s artificially intelligent, or the matrix … it goes awry, it tries to take over the world and humans got to fight this. Or it can go in the absolute opposite where it helps … androids … implants.… Like I said, it’s unlimited to go either way.” (Mississauga focus group participant)

Additionally, several people shared their belief that there is already AI surveillance of their own behaviour, referencing targeted ads that they have received for products they had only spoken privately about.

Some participants commented on how AI could have positive impacts, as in the case of autonomous vehicles. However, most of the people who said positive things about AI also expressed concern about how AI will affect society.

“It’s portrayed as friendly and helpful, but it’s always watching and listening.… So I’m excited about the possibilities, but concerned about the implications and reaching into personal privacy.” (Sudbury focus group participant)

Supporting scenarios

In contrast, focus group participants reacted positively to three realistic health AI research scenarios. In one of the scenarios, some perceived that health data and AI research could actually save lives, and most people were also supportive of two other scenarios which didn’t include potential lifesaving benefits.

They commented favourably about the potential for health data and AI research to generate knowledge that would otherwise be impossible to obtain. For example, they reacted very positively to the potential for an AI-based test to save lives by identifying origin of cancers so that treatment can be tailored. Participants also noted practical advantages of AI including the ability to sift through large amounts of data, perform real-time analyses and provide recommendations to health care providers and patients.

When you can reach out and have a sample size of a group of ten million people and to be able to extract data from that, you can’t do that with the human brain. A group, a team of researchers can’t do that. You need AI. (Mississauga focus group participant)

A CBC report on the future of AI in health care.

Protecting privacy

The focus group participants were not positively disposed towards all possible uses of health data in AI research.

They were concerned that the health data provided for one health AI purpose might be sold or used for other purposes that they do not agree with. Participants also worried about the negative impacts if AI research creates products that lead to lack of human touch, job losses and a decrease in human skills over time because people become overly reliant on computers.

The focus group participants also suggested ways to address their concerns. Foremost, they spoke about how important it is to have assurance that privacy will be protected and transparency about how data are used in health AI research. Several people stated the condition that health AI research should create tools that function in support of humans, rather than autonomous decision-making systems.

“As long as it’s a tool, like the doctor uses the tool and the doctor makes the call…it’s not a computer telling the doctor what to do.” (Sudbury focus group participant)

Involving members of the public in decisions about health AI

Engaging with members of the public took time and effort. In particular, considerable work was required to develop, test and refine realistic, plain language health AI scenarios that deliberately included potentially contentious points. But there was a large return on investment.

The focus group participants — none of whom were AI experts — had some important insights and concrete suggestions about how to make health AI research more responsible and acceptable to members of the public.

Studies like ours can be important inputs into policies and practice guides for health data and AI research. Consistent with the Montréal Declaration for Responsible Development of AI, we believe that researchers, scientists and policy makers need to work with members of the public to take the science of health AI in directions that members of the public support.

By understanding and addressing public concerns, we can establish trustworthy and socially beneficial ways of using health data in AI research.