Relative importance sampling for off-policy actor-critic in deep reinforcement learning.

ABSTRACT: Off-policy learning exhibits greater instability when compared to on-policy learning in reinforcement learning (RL). The difference in probability distribution between the target policy (π) and the behavior policy (b) is a major cause of instability. High variance also originates from distributional mismatch. The variation between the target policy's distribution and the behavior policy's distribution can be reduced using importance sampling (IS). However, importance sampling has high variance, which is exacerbated in sequential scenarios. We propose a smooth form of importance sampling, specifically relative importance sampling (RIS), which mitigates variance and stabilizes learning. To control variance, we alter the value of the smoothness parameter [Formula: see text] in RIS. We develop

SUBMITTER: Humayoo M 

PROVIDER: S-EPMC12022357 | biostudies-literature | 2025 Apr

REPOSITORIES: biostudies-literature