For Patients to Trust Medical AI, They Need to Understand It
AI holds great promise to increase the quality and reduce the cost of health care in developed and developing countries. But one obstacle to using it is patients don’t trust it. One key reason is they perceive medical AI to be a black box and they think they know more about physicians’ decision-making process than they actually do, the authors research found. A remedy: Provide patients with an explanation of how both types of care providers make decisions.
Artificial intelligence-enabled health applications for diagnostic care are becoming widely available to consumers; some can even be accessed via smartphones. Google, for instance, recently announced its entry into this market with an AI-based tool that helps people identify skin, hair, and nail conditions. A major barrier to the adoption of these technologies, however, is that consumers tend to trust medical AI less than human health care providers. They believe that medical AI fails to cater to their unique needs and performs worse than comparable human providers, and they feel that they cannot hold AI accountable for mistakes in the same way they could a human.
This resistance to AI in the medical domain poses a challenge to policymakers who wish to improve health care and to companies selling innovative health services. Our research provides insights that could be used to overcome this resistance.
In a paper recently published in Nature Human Behaviour, we show that consumer adoption of medical AI has as much to do with their negative perceptions of AI care providers as with their unrealistically positive views of human care providers. Consumers are reluctant to rely on AI care providers because they do not believe they understand or objectively understand how AI makes medical decisions; they view its decision-making as a black box. Consumers are also reluctant to utilize medical AI because they erroneously believe they better understand how humans make medical decisions.
Our research — consisting of five online experiments with nationally representative and convenience samples of 2,699 people and an online field study on Google Ads — shows how little consumers understand about how medical AI arrives at its conclusions. For instance, we tested how much nationally representative samples of Americans knew about how AI care providers make medical decisions such as whether a skin mole is malignant or benign. Participants performed no better than they would have if they had guessed; they would have done just as well if they picked answers at random. But participants recognized their ignorance: They rated their understanding of how AI care providers make medical decisions as low.
By contrast, participants overestimated how well they understood how human doctors make medical decisions. Even though participants in our experiments possessed similarly little factual understanding of decisions made by AI and human care providers, they claimed to better understand how human decision-making worked.
In one experiment, we asked a nationally representative online sample of 297 U.S. residents to report how much they understood about how a doctor or an algorithm would examine images of their skin to identify cancerous skin lesions. Then we asked them to explain the human or the algorithmic provider’s decision-making processes.
