Combining evolutionary and curiosity-driven algorithms to enhance and adapt exploration efficiency in Q-Learning

Details

Supervisors

Nijssen, Siegfried

Faculty

Ecole polytechnique de Louvain

Degree label

Master [120] : ingénieur civil en informatique, à finalité spécialisée

Abstract

Reinforcement Learning (RL) performance is highly sensitive to the hyper-parameter values used during training. However, tuning these parameters often relies on experimental methods guided by intuition or computationally expensive techniques such as grid search and random search, which can lead to suboptimal performance. To address this issue, Automated Reinforcement Learning (AutoRL) has gained significant attention in recent years. This master’s thesis explores the development of a novel exploration strategy that combines AutoRL with a curiosity-driven approach, aiming to generalize the well-known challenge of the exploration-exploitation trade-off. Our approach integrates the Never Give Up (NGU) method developed by Badia et al with online tuning mechanisms inspired by Evolutionary Approachs, and is applied to the widely-used Q-learning algorithm. This work contributes to the field of Automated Reinforcement Learning by introducing a general exploration strategy that eliminates the need for hyper-parameter tuning, thereby enhancing the efficiency and applicability of reinforcement learning algorithms. Our approach surpasses simpler implementations of evolutionary algorithms applied to Q-learning. These advancements have the potential to make reinforcement learning more accessible to practitioners who are not experts in the field, thereby broadening the scope of its real-world applications.