Realistic Actor-Critic: A framework for balance between value overestimation and underestimation.

ABSTRACT:

Introduction

The value approximation bias is known to lead to suboptimal policies or catastrophic overestimation bias accumulation that prevent the agent from making the right decisions between exploration and exploitation. Algorithms have been proposed to mitigate the above contradiction. However, we still lack an understanding of how the value bias impact performance and a method for efficient exploration while keeping stable updates. This study aims to clarify the effect of the value bias and improve the reinforcement learning algorithms to enhance sample efficiency.

Methods

This study designs a simple episodic tabular MDP to research value underestimation and overestimation in actor-critic methods. This study proposes a unified framework called Realistic Actor-Critic (RA

SUBMITTER: Li S 

PROVIDER: S-EPMC9868235 | biostudies-literature | 2022

REPOSITORIES: biostudies-literature