ADAPTIVE REWARD WEIGHTING FOR STABLE IMITATION LEARNING IN TD3-SAC-GAIL

Abstract

Imitation learning enables reinforcement learning agents to acquire complex behaviors from expert demonstrations, but its performance remains strongly influenced by the quality of expert data and the balance between imitation and exploration during policy optimization. In particular, TD3-SAC-GAIL improves the exploration capability of GAIL by combining deterministic policy smoothing from TD3 with entropy-driven exploration from SAC; however, the use of fixed reward or exploration weighting can lead to an inappropriate exploration–exploitation balance across different training stages and environments. This paper proposes Adaptive Reward Weighting for Stable Imitation Learning in TD3-SAC-GAIL, an extension of the TD3-SAC-GAIL framework that dynamically adjusts the contribution of exploration and imitation signals according to the current learning condition. The proposed mechanism is designed to encourage exploration when learning progress is limited while maintaining greater emphasis on policy quality as the training process becomes stable. The framework is evaluated in four continuous-control environments, namely HalfCheetah, Walker2d, Hopper, and LunarLanderContinuous. Experimental evaluation considers expert-surpassing performance, training behavior, reward stability, and the effect of adaptive weighting. The proposed approach provides a systematic mechanism for controlling the exploration–imitation trade-off and aims to improve the stability and robustness of GAIL-based policy learning while retaining the exploration advantages of the TD3-SAC hybrid framework.