Reinforcement learning for the optimal control of hybrid systems

Details

Supervisors

Legat, Benoît ; Jungers, Raphaël

Faculty

Ecole polytechnique de Louvain

Degree label

Master [120] : ingénieur civil en mathématiques appliquées

Abstract

Hybrid systems, because they comprise of both continuous dynamics and discrete events, allow the modeling of complex systems for which traditional dynamical systems are inadequate. However, despite considerable efforts from, notably, the control theory and computer science communities, their optimal control remains a challenging topic because of their inherent complexity. This work proposes to address this issue by designing an algorithm, called hybrid dual dynamic programming algorithm, capable of finding near-optimal trajectories for time-invariant discrete-time affine hybrid optimal control problems by iteratively learning their value function through the reciprocal succession of the exploration of the state-space by means of a model predictive controller, and the approximation of the value function in the explored regions via dual dynamic programming. Its application to discrete-time piecewise affine control systems and discrete-time linear control systems under linear temporal logic constraints is presented, which illustrates its capabilities and offers an opportunity to discuss the main challenges it faces, consisting in the presence of a duality gap due to the non-convexity of the problem and in its susceptibility to under-explore parts of the state-space, and potential solutions to overcome them.