TY - GEN
T1 - Integrating Policy Summaries with Reward Decomposition for Explaining Reinforcement Learning Agents
AU - Septon, Yael
AU - Huber, Tobias
AU - André, Elisabeth
AU - Amir, Ofra
N1 - Publisher Copyright:
© 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.
PY - 2023
Y1 - 2023
N2 - Explainable reinforcement learning methods can roughly be divided into local explanations that analyze specific decisions of the agents and global explanations that convey the general strategy of the agents. In this work, we study a novel combination of local and global explanations for reinforcement learning agents. Specifically, we combine reward decomposition, a local explanation method that exposes which components of the reward function influenced a specific decision, and HIGHLIGHTS, a global explanation method that shows a summary of the agent’s behavior in decisive states. Results from two user studies show significant benefits for both methods. We found that the local reward decomposition was more useful for identifying the agents’ priorities. However, when there was only a minor difference between the agents’ preferences, the global information provided by HIGHLIGHTS additionally improved participants’ understanding.
AB - Explainable reinforcement learning methods can roughly be divided into local explanations that analyze specific decisions of the agents and global explanations that convey the general strategy of the agents. In this work, we study a novel combination of local and global explanations for reinforcement learning agents. Specifically, we combine reward decomposition, a local explanation method that exposes which components of the reward function influenced a specific decision, and HIGHLIGHTS, a global explanation method that shows a summary of the agent’s behavior in decisive states. Results from two user studies show significant benefits for both methods. We found that the local reward decomposition was more useful for identifying the agents’ priorities. However, when there was only a minor difference between the agents’ preferences, the global information provided by HIGHLIGHTS additionally improved participants’ understanding.
KW - Explainable AI
KW - Neural Networks
KW - Reinforcement Learning
UR - http://www.scopus.com/inward/record.url?scp=85169004199&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-37616-0_27
DO - 10.1007/978-3-031-37616-0_27
M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???
AN - SCOPUS:85169004199
SN - 9783031376153
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 320
EP - 332
BT - Advances in Practical Applications of Agents, Multi-Agent Systems, and Cognitive Mimetics. The PAAMS Collection - 21st International Conference, PAAMS 2023, Proceedings
A2 - Mathieu, Philippe
A2 - Dignum, Frank
A2 - Novais, Paulo
A2 - De la Prieta, Fernando
T2 - 21st International Conference on Practical Applications of Agents and Multi-Agent Systems, PAAMS 2023
Y2 - 12 July 2023 through 14 July 2023
ER -