Why aren't the padding tokens appended during data preprocessing, before the inputs are turned by the feedfoward layer into the key, query, value, vectors?
Adding padding before forming K, Q, and V vectors would insert extra tokens into the input sequences, altering their lengths and potentially distorting the underlying data structure. As a result, the subsequent computation of K, Q, and V vectors would incorporate these padding tokens, affecting the model's ability to accurately represent the original data. During the attention calculation, these padding tokens would influence the attention scores, potentially diluting the focus on the actual content of the input sequences. This could lead to less effective attention patterns and hinder the model's ability to learn meaningful representations from the data. Furthermore, applying padding after forming K, Q, and V vectors allows for the efficient use of masking techniques to exclude padding tokens from the attention mechanism. By setting the attention scores corresponding to padding positions to negative infinity before the softmax operation, the model effectively ignores these tokens during attention calculation. This approach preserves the integrity of the input sequences, ensures accurate attention computations, and maintains the model's focus on relevant information within the data. P.S. I used ChatGPT to format my answer because it can do this thing better.
@@slayer_dan how could it possibly save computing power to pad the matrices with multiple, 512-element vectors, rather than simply appending tokens to the initial sequence of tokens?
Take it with a grain if salt but I think if you hardcore the mask to not be paid attention to, you don't need learn that extra behavior for the [pad] token so it's more stable.
@@neetpride5919 Multiplying matrices on the GPU can be optimized by efficiently sizing the matrices such that they fit more cleanly in GPU cache. For example, suppose you have two skinny tall matrices that you want to multiply together. Sometimes, the kernel operations that occur across one (tall) axis can be the bottleneck in compute time. There are instances in optimization where padding matrices with 0s to uniform square shapes or multiples of the cache block size can speed up the kernel operations on the GPU.
I would love if the quizzes had answers in the comments eventually. I know this is a fresh video, but I want to check my work, not just have a discussion 😅
Amazing work, can't wait for next episode !
Thank you for your amazing content. Where can I acess the drawio file?
Why aren't the padding tokens appended during data preprocessing, before the inputs are turned by the feedfoward layer into the key, query, value, vectors?
Adding padding before forming K, Q, and V vectors would insert extra tokens into the input sequences, altering their lengths and potentially distorting the underlying data structure. As a result, the subsequent computation of K, Q, and V vectors would incorporate these padding tokens, affecting the model's ability to accurately represent the original data. During the attention calculation, these padding tokens would influence the attention scores, potentially diluting the focus on the actual content of the input sequences. This could lead to less effective attention patterns and hinder the model's ability to learn meaningful representations from the data.
Furthermore, applying padding after forming K, Q, and V vectors allows for the efficient use of masking techniques to exclude padding tokens from the attention mechanism. By setting the attention scores corresponding to padding positions to negative infinity before the softmax operation, the model effectively ignores these tokens during attention calculation. This approach preserves the integrity of the input sequences, ensures accurate attention computations, and maintains the model's focus on relevant information within the data.
P.S. I used ChatGPT to format my answer because it can do this thing better.
@@slayer_dan how could it possibly save computing power to pad the matrices with multiple, 512-element vectors, rather than simply appending tokens to the initial sequence of tokens?
Take it with a grain if salt but I think if you hardcore the mask to not be paid attention to, you don't need learn that extra behavior for the [pad] token so it's more stable.
@@neetpride5919 Multiplying matrices on the GPU can be optimized by efficiently sizing the matrices such that they fit more cleanly in GPU cache. For example, suppose you have two skinny tall matrices that you want to multiply together. Sometimes, the kernel operations that occur across one (tall) axis can be the bottleneck in compute time. There are instances in optimization where padding matrices with 0s to uniform square shapes or multiples of the cache block size can speed up the kernel operations on the GPU.
great video thanks
question ... in the third question ... how do sample subset of keys, queries "depending on importance"
can i find the flow chart graphic of the informer model on github? and is draw io for free?
Love from kashmir india bro!❤❤❤
With regard to the quiz I think it is B D B. Not sure how this is going to launch a discussion though. You present things very well.
Ding ding ding! Good work on the quiz! While this may or may not spark a discussion, just wanted to say thanks for participating :)
Also as always great video, hoping in future you deal with encoder only and decoder only transformers...
Yep! For sure. Thank you so much!
Can u tell an interactive model of AI neural network for school project.. And ur videos are nice and I understand easily.. Pls tell
I would love if the quizzes had answers in the comments eventually. I know this is a fresh video, but I want to check my work, not just have a discussion 😅
is this same as the wjat timesfm uses
In answer to your question, I can either:
A) mono-task
or
B) screw up several things at once
cool!
Thank you 🙏
Not sure if just me, but starting at about 4:50 your graphics are so dark...
maybe go to a white background or light gray, like your original png...
Yea. Let me try brightening them up for future videos if I can. Thanks for the heads up