Effects of XAI on perception, trust and acceptance

05 Oct 23 14:00 UTC

Large opaque models like neural networks require dedicated methods to study and interpret their behavior. In this series we review recent developments, analysing their relevance for business applications.

Maternus has a background in Mathematics and Computer Science. After working as an AI Engineer for some time, he decided to dig deeper into the field of Uncertainty Quantification, and is now pursuing a PhD on Inverse UQ with a focus on applications in the fields of crashworthiness and passenger safety in cars.

Besides classical Bayesian methods to obtain a distribution on the parameters of interest, he currently explores the applicability of Simulation-Based Inference to his problem.

This talk delves into the influence of Explainable Artificial Intelligence (XAI) on human cognition, trust, and acceptance of AI-driven systems. Through a review of empirical studies, this presentation illuminates how the provision of intelligible explanations shapes individuals’ perception of AI-generated outputs. By synthesizing findings from diverse contexts, we uncover critical insights into the mechanisms underlying the cognitive processing of explanations, shedding light on the factors that modulate trust and acceptance levels. This discourse aims to inspire a broader conversation on designing XAI systems that not only excel in performance but also empower users through comprehensible and trust-building explanations.

References

[Buc20P]

Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems, Zana Buçinca, Phoebe Lin, Krzysztof Z. Gajos, Elena L. Glassman.

Mar 2020

Explainable artificially intelligent (XAI) systems form part of sociotechnical systems, e.g., human+AI teams tasked with making decisions. Yet, current XAI systems are rarely evaluated by measuring the performance of human+AI teams on actual decision-making tasks. We conducted two online experiments and one in-person think-aloud study to evaluate two currently common techniques for evaluating XAI …

Publication

[Her23I]

Impact of explainable AI on cognitive load: Insights from an empirical study, Lukas-Valentin Herm.

May 2023

While the emerging research field of explainable artificial intelligence (XAI) claims to address the lack of explainability in high-performance machine learning models, in practice XAI research targets developers rather than actual end-users. Unsurprisingly, end-users are unwilling to use XAI-based decision support systems. Similarly, there is scarce interdisciplinary research on end-users’ …

Publication

[Lei23E]

Effects of Explainable Artificial Intelligence on trust and human behavior in a high-risk decision task, Benedikt Leichtmann, Christina Humer, Andreas Hinterreiter, Marc Streit, Martina Mara.

Feb 2023

Understanding the recommendations of an artificial intelligence (AI) based assistant for decision-making is especially important in high-risk tasks, such as deciding whether a mushroom is edible or poisonous. To foster user understanding and appropriate trust in such systems, we assessed the effects of explainable artificial intelligence (XAI) methods and an educational intervention on AI-assisted …

[Shi21E]

The effects of explainability and causability on perception, trust, and acceptance: Implications for explainable AI, Donghee Shin.

Feb 2021

Artificial intelligence and algorithmic decision-making processes are increasingly criticized for their black-box nature. Explainable AI approaches to trace human-interpretable decision processes from algorithms have been explored. Yet, little is known about algorithmic explainability from a human factors’ perspective. From the perspective of user interpretability and understandability, this study …

Publication

[Van21E]

Evaluating XAI: A comparison of rule-based and example-based explanations, Jasper van der Waa, Elisabeth Nieuwburg, Anita Cremers, Mark Neerincx.

Feb 2021

Current developments in Artificial Intelligence (AI) led to a resurgence of Explainable AI (XAI). New methods are being researched to obtain information from AI systems in order to generate explanations for their output. However, there is an overall lack of valid and reliable evaluations of the effects on users' experience of, and behavior in response to explanations. New XAI methods are often …

Publication

References

In this series →