Understanding scipy.minimize part 1: The BFGS algorithm

A description of how quasi Newton algorithms in general, and in special the BFGS algorithm work. Animations are made with the manimce library.
Sources:
* Nocedal & Wright: Numerical Optimization Ch. 6 (which also presents a SR1 trust region method)
* Dennis & More: Quasi-Newton Methods, Motivation, and Theory, SIAM Review, Vol. 19, No. 1, 1977 (describing the PSB method. The video is based mostly on the derivation in section 7 of this paper.)
The actual update formulas of BFGS are not included in the video. These can be found in both sources as well as e.g. Wikipedia.

[@deepbhadja5730]

Ridiculously underrated channel, high quality visuals and great explanation.

[@andresyesidmorenovilla7888]

Awesome video! Incredible content quality for such a small channel!

[@Frans_Rodenburg]

Nice video! There are plenty of videos on Newton-Raphson methods but I could not find any visual explanations of quasi-Newton methods until I found yours.

[@FolkerHoffmann]

Thank you!

[@m.khanmohamadi9815]

awesome visualization. thanks

[@mallahsara4565]

Awesome content! Keep up the good work!

[@philipcasserstal9260]

Great video! Nice visuals and easy to understand

[@owaischunawala4030]

Thank you for making this video!

[@SS-yb1qd]

Ubelivably excellent.

[@KNO476]

Amazing content

[@oldudot6940]

thank you, it was very helpful !

[@rasm7266]

Thanks a lot for making this video. :-)

[@prodbyryshy]

beautiful

[@jakovbilic4556]

Really helpful!

[@a.renegeist2800]

Really nice video, thank you! Just a really minor correction to 04:15... due to the symmetry you need to compute slightly more than half of the values, not exactly half of the values.

[@FolkerHoffmann]

Thanks! You are of course completely right with your correction!

[@cvanaret]

Thanks for the video! One comment: even though you discuss the importance of convexity later on, your claim "in higher dimensions, finding the minimum of a quadratic function is very easy" is misleading ;)

[@wildreams]

"Sci-py" as in "Sci" in Science ;)

[@FolkerHoffmann]

Thanks! I knew the meaning of the "Sci" but somehow never connected this to the prononciation :-/ (And the prononciation is literally on the first page of the documentation :D)

[@wildreams]

@@FolkerHoffmann lol, all good. Thanks for the great content!