What is LoRA? Low-Rank Adaptation for finetuning LLMs EXPLAINED

How does LoRA work? Low-Rank Adaptation for Parameter-Efficient LLM Finetuning explained. Works for any other neural network as well, not just for LLMs.
➡️ AI Coffee Break Merch! 🛍️ aicoffeebreak.creator-spring....
📜 „Lora: Low-rank adaptation of large language models“ Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L. and Chen, W., 2021. arxiv.org/abs/2106.09685
📚 sebastianraschka.com/blog/202...
📽️ LoRA implementation: • Low-rank Adaption of L...
Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏
Dres. Trost GbR, Siltax, Vignesh Valliappan, Mutual Information, Kshitij
Outline:
00:00 LoRA explained
00:59 Why finetuning LLMs is costly
01:44 How LoRA works
03:45 Low-rank adaptation
06:14 LoRA vs other approaches
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕
Patreon: / aicoffeebreak
Ko-fi: ko-fi.com/aicoffeebreak
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔗 Links:
AICoffeeBreakQuiz: / aicoffeebreak
Twitter: / aicoffeebreak
Reddit: / aicoffeebreak
CZcams: / aicoffeebreak
#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research​
Music 🎵 : Meadows - Ramzoid
Video editing: Nils Trost

[@rockapedra1130]

Perfect. This exactly what I wanted to know. "Bite-sized" is right!

[@MikeTon]

Insightful : Especially the comparison from LORA to prefix tuning and adapters at the end!

[@AICoffeeBreak]

Thank you! Glad you liked it.

[@DerPylz]

Yay, thanks!

[@SoulessGinge]

Very clear and straightforward. The explanation of matrix rank was especially helpful. Thank you for the video.

[@AICoffeeBreak]

Thank You for the visit! hope to see you again soon!

[@pranav_tushar_sg]

thanks!

[@AICoffeeBreak]

You're welcome!

[@minkijung3]

Thanks Letitia. Your explanation was very clear and helpful to understand the paper.

[@AICoffeeBreak]

I'm so glad it's helpful to you!

[@deviprasadkhatua]

Excellent explaination. Thanks!

[@AICoffeeBreak]

Glad you enjoyed it!

[@dineth9d]

Thanks!

[@AICoffeeBreak]

Welcome! :)

[@outliier]

What a great topic!

[@keshavsingh489]

So simple explanation, thank you soo much!!

[@wholenutsanddonuts5741]

I’ve been using LoRAs for a while now but didn’t have a great understanding of how they work. Thank you for the explainer!

[@wholenutsanddonuts5741]

I assume this works the same for diffusion models like stable diffusion?

[@AICoffeeBreak]

For any neural network. You just need to figure out based on your application which matrices you should reduce and which not.

[@wholenutsanddonuts5741]

@@AICoffeeBreak so super easy then! 😂 seriously though that’s awesome to know!

[@jarj5313]

THANKS THAT WAS GREAT EXPLANATION

[@michelcusteau3184]

By far the clearest explanation on youtube

[@AICoffeeBreak]

Thank you very much for the visit and for leaving this heartwarming comment!

[@elinetshaaf75]

true!

[@butterkaffee910]

I love lora ❤ even for vit's

[@amelieschreiber6502]

LoRA is awesome! It also helps with overfitting in protein language models as well. Cool video!

[@thecodest2498]

Thank you sooooo much for this video. I started reading the paper, was very terrified by it, then I thought I should watch some CZcams video, watch one video, was asleep half-way through the video. Woke up again and stumbled across your video, your coffee woke me up and now I got the LoRA. Thanks for your efforts.

[@AICoffeeBreak]

Wow, this warms my coffee heart, thanks!

[@m.rr.c.1570]

Thank you for clearing my concepts regarding LoRA

[@AICoffeeBreak]

[@kindoblue]

Loved the explanation. Thanks

[@user-ig3rp7fk9c]

Firstly thanks for the amazing video. Can you also make a video about QLoRA.

[@ambivalentrecord]

Great explanation Letitia

[@AICoffeeBreak]

Glad you think so! 😄

[@karndeepsingh]

Thanks again for amazing video. I would also request a detailed video on Flash Attention. Thanks

[@AICoffeeBreak]

Noted. It's on The List.
Thanks! 😄

[@soulfuljourney22]

Concept of rank of a matrix,tauught in such an effective way

[@AICoffeeBreak]

Cheers!

[@Lanc840930]

Very comprehensive explanation! Thank you

[@Lanc840930]

Thanks a lot. And I have a question for “linear dependence ”, is this mention in original paper?

[@AICoffeeBreak]

The paper talks about the rank of a matrix, so about linear dependency between rows / columns.

[@Lanc840930]

oh, I see! Thank you 😊

[@deepak_kori]

You are just amazing >>> so beautiful so elegant just wow😇😇

[@bdennyw1]

Fantastic video as always. QLora is even better if you are GPU poor like me.

[@alirezafarzaneh2539]

Thanks for the simple and educating video!
If I'm not mistaken, prefix tuning is pretty much the same as embedding vectors in diffusion models! How cool is that? 😀

[@ArunkumarMTamil]

how is Lora fine-tuning track changes from creating two decomposition matrix? How the ΔW is determined?

[@Micetticat]

LoRA: how can it be so simple? 🤯

[@AICoffeeBreak]

Kind of tells us that fine-tuning all parameters in an LM is overkill.

[@dr.mikeybee]

If we knew what abstractions were handled layer by layer, we could make sure that the individual layers were trained to completely learn those abstractions. Let's hope Max Tegmark's work on introspection get us there.

[@floriankowarsch8682]

As always amazing content! 😌
It's perfect to refresh knowledge & learn something new.
I think interesting about LoRA is how strong it actually regularizes fine-tuning: Is it possible it overfit when using a very small matrix in LoRA? Can LoRA also harm optimization?

[@TheRyulord]

Still possible to overfit but more resistant to overfitting compared to a full finetune. All the work I've seen on LoRAs say that it's just as good as a full finetune in terms of task performance as long as your rank is high enough for the task. What's interesting is that the necessary rank is usually quite low (around 2) even for relatively big models (llama 7B) and reasonable complex tasks. At least that's all the case for language modelling. Might be different for other domains.

[@kunalnikam9112]

In LoRA, Wupdated = Wo + BA, where B and A are decomposed matrices with low ranks, so i wanted to ask you that what does the parameters of B and A represent like are they both the parameters of pre trained model, or both are the parameters of target dataset, or else one (B) represents pre-trained model parameters and the other (A) represents target dataset parameters, please answer as soon as possible

[@yacinegaci2831]

Great explanation, thanks for the video!
I have a lingering question about LoRA: Is it necessary to approximate the low-rank matrices of the difference weights (the Delta W in the video). Or can we reduce the size of the original weight matrices? If I understood the video correctly, at the end of LoRA training, I have the full parameters of the roginal model + the difference weights (in reduced size). My question is why can't I learn low rank matrices for the original weights as well?

[@AICoffeeBreak]

Hi, in principle you can, even though I would expect you could lose some model performance. The idea of finetuning with LoRA is that the small finetuning updates should have low rank. matrices. BUT there is work using LoRA for pretraining, called ReLoRA. Here is the paper 👉 arxiv.org/pdf/2307.05695.pdf
There is also this discussion on Reddit going on: 👉 www.reddit.com/r/MachineLearning/comments/13upogz/d_lora_weight_merge_every_n_step_for_pretraining/

[@yacinegaci2831]

@@AICoffeeBreak Oh, that's amazing. Thanks for the answer, for the links, and for your great videos :)

[@terjeoseberg990]

I thought this was long range wide band radio communications.

[@AICoffeeBreak]

🤣🤣

[@ayyship]

Why use weight matrixes to start with if you can use lora representation? Assuming you gain space, the only downside I can think of is the additional compute to get back the weight matrix. But that should be smaller then the gain of the speed up of backward propagation.

[@AICoffeeBreak]

Thanks for this question. You do not actually start with the weight matrices, you learn A and B directly from which you reconstruct the delta W matrix. Sorry this was not clear enough in the video.

[@alislounge]

Which one is the most and which one is the least 'compute efficient'? Adapters, Prefix Tuning or LORA?

[@onomatopeia891]

Thanks! But how do we determine the correct rank? Is it just trial and error with the value of R?

[@AICoffeeBreak]

Exactly. At least so far. Maybe some theoretical understanding will come up in time.

[@davidromero1373]

Hi a question, can we use lora to just reduce the size of a model and run inference, or we have to train it always?

[@AICoffeeBreak]

LoRA just reduces the size of the trainable parameters for fine-tuning. But the number of parameters of the original model stays the same.

[@ryanhewitt9902]

Aren't we effectively using the same kind of trick when we train the transformer encoder / self-attention block? Assuming row vectors, we can use the form W_v⋅v.T⋅k⋅W_k.T⋅W_q⋅q.T. Ignoring the *application* of attention and focusing its calculation, we get the form k⋅W_k.T⋅W_q⋅q.T . Since W_k and W_q are projection matrices from embedding length to dimension D_k, we have the same sort of low rank decomposition where D_k corresponds to "r" in your video. Is that right?

[@mesochild]

what do i have to learn to understand this help please

[@mkamp]

Absolutely awesome explanation. Would like to get your take on LoRA vs (IA)**3 as well. It seems that people still prefer LoRA over (IA)**3 even though the latter has a slightly higher performance?

[@AnthonyGarland]

Thanks!

[@AICoffeeBreak]

Wow, thanks a lot! 😁