I must say that in more than 6 months, this is by far the best lecture/ material I have come across that was able to make me understand what policy gradient method actually is. I really praise this work. :) Thank you.
Very good lecture about policy gradient method. I have looked through a lot of articles and was understanding almost everything, but your derivation explanation is really the best. It just opened my eyes and showed the whole picture. Thank you very much!!
I might be missing a simple concept here but how are we increasing/decreasing the grad log probability of the actions using the gradient of U(theta)? I get that positive return for a trajectory will make the gradient of U positive and so theta will be increased in favour of those trajectories but how is it increasing grad log prob?
could you train a robot for 2 weeks in the real world then use those trained parameters to optimize a virtual environment? You know.. making the virtual environment very close to the real world?
Real world is very complicated with model uncertainties, friction, wear and tear and what have you... Simulators can come close , but we cannot expect them to fully mimic real world phenomena.
1:11 why are DQNs and friends Dynamic Programming Methods? I mean the neural network works as functions approximator to satisfy Bellmans eqn. , but still Backprop is the workhorse. In my opinion DQNs are much more similar to PG methods than to Bellman Updates??! And another issue with RL Landscape slide is where the heck are model based RL algos?? This slide should be renamed to model free RL landscape.
After 5 years still, the lecture is a great video, thank you a lot
Every time I forget how policy gradients work exactly, I just come back here and watch starting at 9:30
And every time you visited this video, you forgot where to start watching. Thats why you posted this comment. Smart guy!
This is the best lecture I have seen on policy gradient methods. Thanks a lot.
Great Lecture!!!! Pieter's explanations are just a gem!
I must say that in more than 6 months, this is by far the best lecture/ material I have come across that was able to make me understand what policy gradient method actually is. I really praise this work. :) Thank you.
Amazing lecture! Love how Pieter explains the math. super easy to understand.
Honestly I can’t keep up without seeing what he’s pointing at. Gotta pause and search around the screen each time he says “this over here”
Exactly. "This over here" has got to be the most uttered phrase in this lecture. So frustrating.
It's such a good lecture, I'm stopping to ask myself why it was so easy to cover such significant information with full understanding
Best lecture on Policy Gradients hands down. Has covered some worth noting superficial details of many papers as well.
Awesome instructor.
I got a lot out of this lecture in particular. Thank you.
Very good lecture about policy gradient method. I have looked through a lot of articles and was understanding almost everything, but your derivation explanation is really the best. It just opened my eyes and showed the whole picture. Thank you very much!!
amazing work. super understandable, concise and information dense.
Wow, this is beautiful!
excellent lecture. Thank you for sharing.
wow damn this is so well explained and the last video is very entertaining.
The explanation of the derivation of policy gradient is really nice and understandable here
best lecturer in series
Great lecture, thanks
I might be missing a simple concept here but how are we increasing/decreasing the grad log probability of the actions using the gradient of U(theta)? I get that positive return for a trajectory will make the gradient of U positive and so theta will be increased in favour of those trajectories but how is it increasing grad log prob?
Awesome!! Thanks
great talk!
Where can i buy his T-shirt?
Instead of having a baseline, why not make your reward function be negative for undesired scenarios and positive for good ones? Great lecture!
why a good "R" would increase the probability of path??? Please help me
nice but hard to follow without knowing what "this" refers to. I hope my guesses were right :)
It would be nice to see what laboratories they had (what exercises)
dont know if ur still interested, but the labs are on the bootcamp website
PG is awesome!!!
Doesn't depend on environment Dynamics really?? Wow
All the pain and stress just goes away when we see our algorithms working😇😇
23:20- Gradient of expectation is expectation of gradient "under mild assumptions". What are those assumptions?
math.stackexchange.com/questions/12909/will-moving-differentiation-from-inside-to-outside-an-integral-change-the-resu
very nice :)
The guy in the background at 51:30
could you train a robot for 2 weeks in the real world then use those trained parameters to optimize a virtual environment? You know.. making the virtual environment very close to the real world?
Of course you could. It's the opposite of what OpenAI does when they train a model in a virtual environment and deploy it in reality.
Real world is very complicated with model uncertainties, friction, wear and tear and what have you...
Simulators can come close , but we cannot expect them to fully mimic real world phenomena.
1:11 why are DQNs and friends Dynamic Programming Methods? I mean the neural network works as functions approximator to satisfy Bellmans eqn. , but still Backprop is the workhorse. In my opinion DQNs are much more similar to PG methods than to Bellman Updates??! And another issue with RL Landscape slide is where the heck are model based RL algos?? This slide should be renamed to model free RL landscape.
If you can't explain it like Pieter Abbeel or Andrew NG then you don't understand it well enough.
Can you share ppt??
You can find all the slides and the other lectures here:
sites.google.com/view/deep-rl-bootcamp/lectures
... you can use "a", and the math will be the same. :)