Séminaire hybride à l'Université McGill ou Zoom.

The first algorithm for the Linear Quadratic (LQ) control problem with an unknown system model, featuring a regret of O(√T ), was introduced by Abbasi-Yadkori and Szepesvári (2011). Recognizing the computational complexity of this algorithm, subsequent efforts have been dedicated to proposing algorithms that are computationally tractable while preserving this order of regret. Although successful, the existing works in the literature lack a fully adaptive exploration- exploitation trade-off adjustment and require a user-defined value, which can lead to overall regret bound growth with some factors. 

In this work, noticing this gap, we propose the first fully adaptive algorithm that controls the number of policy updates (i.e., tunes the exploration-exploitation trade-off) and optimizes the upper-bound of regret adaptively. Our proposed algorithm builds on the SDP-based approach of Cohen et al. (2019) and relaxes its need for a horizon-dependent warm-up phase by appropriately tuning the regularization parameter and adding an adaptive input perturbation. We further show that through careful exploration-exploitation trade-off adjustment there is no need to commit to the widely-used notion of strong sequential stability, which is restrictive and can introduce complexities in initialization.

Bio: Jafar Abbaszadeh Chekan is a Ph.D. candidate in the Department of Aerospace Engineering at the University of Illinois at Urbana-Champaign (UIUC), where he is currently working on safety-aware learning-based control systems toward his doctoral degree. He is also affiliated with the Decision & Control Group at the Coordinated Science Lab (CSL). Prior joining UIUC, he received his master's degree in Engineering Mechanics from Virginia Tech in 2018. His research interest generally lies in controls, learning, game theory and their intersections.