Control of a Quadrotor with Reinforcement Learning
Vložit
- čas přidán 2. 06. 2024
- In this video, we demonstrate a method to control a quadrotor with a neural network trained using reinforcement learning techniques. With reinforcement learning, a common network can be trained to directly map state to actuator command making any predefined control structure obsolete for training.
More detail regarding the paper can be found from arxiv.org/abs/1707.05110
and implementation is available from bitbucket.org/leggedrobotics/rai - Věda a technologie
You guys are amazing, thank you for showing the direction
Sir!
Which simulator have u used for training?
Thank you very much, that is really awesome to see
Hello, What a great video. I'm new to the subject of Reinforced Learning and hoped I could ask a question. Here goes: Does the ability of the drone to recover depend on input from the various sensors e.g. gyro. So say the drone flew from shade to sunlight, would there need to be a temperature sensor on board to allow it to cope with any sudden movement associated with moving from hot to cold. Or does the reinforced learning model not care about any of that and just learn to deal with whatever it encounters (so you could throw away the temperature sensor) and it would stabilize just fine with any sudden movement associated with hot to cold. Thanks
The performance of the quadrotor is excellent.
But i have a question about the reinforcement learning. I want to know what a role the RL algorithm play in the control system? navigation or attitude control?
RL algorithm just provide where to move, and PID controler to finish attitude control? Or both attitude control and location control are all RL algorithm`s duty?
Thanks for your response.
Random question here: I'm doing something similar using PPO on a quadrotor (simple simulation using OpenAI gym). I'm trying to get the cable suspended load case now, but still struggling with the reward function. Terminal states are way more difficult to handle than the standard quad.
These 4 PWM actions can be perform at the same time in one iteration loop?
This is great indeed
What simulation engine did you used? Was that Unity? How did you dealed with reality-gap - what was your approach here. Did the copter flight on policy trained only in simulation?
Please have a look our repo and corresponding paper mentioned in the video description. A short answer for "Did the copter flight on policy trained only in simulation?" is yes, simulation only. Thanks for your interests.
Cool work! I just completed a similar project, but I used DDPG with SNNs, nice to see implementation on real quadcopter, have you tried different tasks?
can you tell me how did you simulate your project. can you provide me the simulation code and model in matlab?
I've read the paper however I could not understand how do you define the 4 actions? Each action is each rotor's velocity, right? But then how do they choose the velocity in the beginning?
Hello Anush. Have you made any progress with implementing the models from the paper?
As explained in the paper , there are two networks, one of them is a policy network. The policy network gives the PWM or rpm needed for each motor.
Hello good job, excuse me I'm a beginner and I don't understand why the value network is essential for learning. With only one value of distance between the quad and the point this could not be enough.
Amazing work! How did you guys handle the noise in sensors (Gyro/Accelerometer) ?
kalman filtering based techniques are used to achieve this i guess...also as far as the localization problem is concerned, the drone has reflective markers so im assuming a camera capturing mechanism is present for 3d localization.
@@harshavardhankulkarni6599 yep! Makes sense
what drone did you use? i planning to buy a drone for my research to do similar things. can some suggest me a drone
The answer is in the V.Experiments section of the paper.
Build your own
This looks amazing!
I have experience with ROS and tensorflow (very basic). I'm highly interested in applying your findings to a quadcopter I'm working on.
Can you please point me in the right direction?
What can I start with and what must I learn?
Thanks!
If you are starting from scratch, start with David Silver's lectures on youtube. Then download RLLAB and try simple learning tasks by yourself. You also have to read the papers related to the algorithms they implemented.
Jemin Hwangbo thanks!
Good job guys. Can you share details of the platform you're using - is the machine learn platform tensorflow/python-based, and is the quadrotor system running on ROS?
We use Tensorflow and a custom quadrotor simulator. During experiments, we compute the MLP using Eigen library and the rest of the software is on ROS. We will share the code as soon as the paper is accepted ;)
Jemin Hwangbo
Awesome, I'm looking forward to your paper
Jemin Hwangbo can you please tell me more about the quadcopter simulator? How you customized it and what did you use basically?
Thanks :)
Any updates on the code you are using? Is latency from the computer to the ESC output command a problem?
I built my own simulator for speed. It's basically the Newton-Euler equation without any air drag and gyroscopic effect. Probably the simplest simulator you can imagine. I opensourced the code so anyone with c++ skills can use it. I do not know much about the latency from the computer to the ESC. It seems like it was not a problem for me
mY GOSH!
what kind of drone do you use?
Hummingbird quadrotor from Ascending Technologies! it is written in section 5 of their paper.
where is fucking source code