Towards reconciling usability and usefulness of policy explanations for sequential decision-making systems.

ABSTRACT: Safefy-critical domains often employ autonomous agents which follow a sequential decision-making setup, whereby the agent follows a policy to dictate the appropriate action at each step. AI-practitioners often employ reinforcement learning algorithms to allow an agent to find the best policy. However, sequential systems often lack clear and immediate signs of wrong actions, with consequences visible only in hindsight, making it difficult to humans to understand system failure. In reinforcement learning, this is referred to as the credit assignment problem. To effectively collaborate with an autonomous system, particularly in a safety-critical setting, explanations should enable a user to better understand the policy of the agent and predict system behavior so that users are cognizant of po

SUBMITTER: Tambwekar P 

PROVIDER: S-EPMC11298694 | biostudies-literature | 2024

REPOSITORIES: biostudies-literature