Single Timescale Actor Critic: A Small-Gain Analysis: Bahman Gharesifard

Speaker: Bahman Gharesifard, Professor, Electrical & Computer Engineering, University of California, Los Angeles
Talk Title: Single timescale actor critic: a small-gain analysis
Abstract: We consider the used-in-practice setting of actor-critic where proportional step-sizes are used for both the actor and the critic, with only one critic update with a single sample from the stationary distribution per actor step. Using a small-gain analysis, we prove convergence to a stationary point, with a sample complexity that improves the state of the art. The key technical challenge is in connecting the actor-critic to a perturbed gradient descent, which is often obtained by allowing for infinitely many critic steps and is not possible in single-time scale settings. This is a joint work with Alex Olshevsky at Boston University.
Bio: Bahman Gharesifard is currently a Professor and Area Director for Signals and Systems at the Electrical & Computer Engineering Department, University of California, Los Angeles. He was an Associate Professor, from 2019 to 2021, and an Assistant Professor, from 2013 to 2019, with the Department of Mathematics and Statistics at Queen’s University. He was an Alexander von Humboldt research fellow with the Institute for Systems Theory and Automatic Control at the University of Stuttgart in 2019-2020. He held postdoctoral positions with the Department of Mechanical and Aerospace Engineering at University of California, San Diego 2009-2012 and with the Coordinated Science Laboratory at the University of Illinois at Urbana-Champaign from 2012- 2013. He received the 2019 CAIMS-PIMS Early Career Award, a Humboldt research fellowship for experienced researchers from the Alexander von Humboldt Foundation in 2019, an NSERC Discovery Accelerator Supplement in 2019, and the SIAG/CST Best SICON Paper Prize 2021, and the Canadian Society for Information Theory Best Paper Award in 2022. He has served on the Conference Editorial Board of the IEEE Control Systems Society and IEEE Control System Letters, and is currently an Associate Editor for the IEEE Transactions on Network Control Systems. His research interests include systems and control, distributed control, distributed optimization, machine learning, social and economic networks, game theory, geometric control theory, geometric mechanics, and applied Riemannian geometry.
Slides: bu.edu/hic/files/2024/05/Bahm...