Actor-Critic Model Predictive Control (Talk ICRA 2024)

An open research question in robotics is how to combine the benefits of model-free reinforcement learning (RL) - known for its strong task performance and flexibility in optimizing general reward formulations - with the robustness and online replanning capabilities of model predictive control (MPC). This paper provides an answer by introducing a new framework called Actor-Critic Model Predictive Control. The key idea is to embed a differentiable MPC within an actor-critic RL framework. The proposed approach leverages the short-term predictive optimization capabilities of MPC with the exploratory and end-to-end training properties of RL. The resulting policy effectively manages both short-term decisions through the MPC-based actor and long-term prediction via the critic network, unifying the benefits of both model-based control and end-to-end learning. We validate our method in both simulation and the real world with a quadcopter platform across various high-level tasks. We show that the proposed architecture can achieve real-time control performance, learn complex behaviors via trial and error, and retain the predictive properties of the MPC to better handle out of distribution behaviour.
Reference:
A. Romero, Y. Song, D. Scaramuzza,
"Actor-Critic Model Predictive Control",
IEEE International Conference on Robotics and Automation, 2024
PDF: rpg.ifi.uzh.ch...
For more info about our research on:
Agile Drone Flight: rpg.ifi.uzh.ch/...
Drone Racing: rpg.ifi.uzh.ch/...
Machine Learning: rpg.ifi.uzh.ch/...
Affiliations:
A. Romero, Y. Song, and D. Scaramuzza are with the Robotics and Perception Group, Dep. of Informatics, University of Zurich, and Dep. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland
rpg.ifi.uzh.ch/

[@윤우승학생물리·천문]

So great

[@j_de_ese]

Great work!!🎉

[@alexgraziose717]

Inspiring. Well done!

[@HanamyWarrior]

great!

[@kesavank7607]

can share the source code of the simulation passed by the gate?
,

[@user-lt5no1xt1z]

How do you train the reinforcement learning model? Is it through simulation?