Transfer learning and Transformer models (ML Tech Talks)
Vložit
- čas přidán 24. 07. 2024
- In this session of Machine Learning Tech Talks, Software Engineer from Google Research, Iulia Turc, will walk us through the recent history of natural language processing, including the current state of the art architecture, the Transformer.
0:00 - Intro
1:07 - Encoding text
8:21 - Language modeling & transformers
29:46 - Transfer learning & BERT
43:55 - Conclusion
Catch more ML Tech Talks → goo.gle/ml-tech-talks
Subscribe to TensorFlow → goo.gle/TensorFlow
product: TensorFlow - General; re_ty: Publish; - Věda a technologie
This was one of the best videos I’ve seen explaining transformers and NL models….well done and look forward to the other videos in the series!🍻
The comparison between attentions heads and CNN filters made so much sense!
Please @Tensorflow Team continue with this lecture series ML Tech series
After reading about language models, word embeddings, transformers, etc. for a month, this video put everything in order for me. Thanks!
I didn't understand a single word 😕
Thank you for the awesome talk on all the main NLP models, in particular, the great explanation of the Transformer model!
Great overview and explanation of the Transformer network. I am just starting my exploration into NLP and this talk has saved me lots of time. I now know that this where I need to be focussing my attention. Thank you 👍🙏😍
Great Video. The first Transformer explanation that (correctly) does not use the Encoder/Decoder diagram from the Transformer paper, well done!
Additionally talking about the exact outputs (using only one output for predictions) was very helpful.
best youtube NLP walk through without cutting corners. best delivery as well.
A cristal clear explanation of Transformers. Papers in many cases are very difficult to follow. Pointing out the important omited details which are critical for the model, even if not explained, is very useful. Many out there try to explain transformers without having a clue of what it is. Clearly, this is not the case. Thanks in its deepest tokenized meaning for sharing your knowledge. BTW, the last programming tip is really helpful. A small hands on demo of using BERT(or any flavor of BERT) with a classifier for a particular application would be amazing for another video.
I believe this video provides the most comprehensive explanation of transformers.
This is the best video on youtube that introduces transformer models
One of the best explanation i have come across on transformers. Thanks
So clear. This is one of the best videos explaining transformer architecture.
After a year returning to that video finally I fully (or at least saw the entire video in a row) understand what is going on!! Maybe one more time to fix and go to the next part!! Thanxx
Best explanations so far of the attention or QKV concept... I was searching for a good way to visualize it.. Thanks a ton!!
Very well explained! Thank you very much. Especially loved the comparison between CV kernels and multiple QKV parameters.
One of the best explanations of transformers that I've seen!
This video initialised my attention weights for transformers,
She's so good! I've watched a few videos attempting to explain these self-attention version of transformers and this one is by far the best in so many aspects with actual deep understanding of the architecture at the top followed closely by coherently communicating concepts, good script, presentation and graphics! I hope she narrates more videos like this... I'm about to search and find out lol! 🧐🤞 🤓
Mapping to geometry is pro. I have thought since my education about 40 years ago that current mathematics is taught incorrectly. Here is a pro example of how math should be taught!
Wow that explanation actually dissipated many of my questions. Thanks a lot Julia!
Best Video on Transfer Learning. So much clarity
44:40: Thanks for your attention 😁
Caught that too
Hands down the best explanation, this after watching so many videos, terrific, Looking forward to some videos on understanding on BARD and its fine tuning
awesome; the very BEST explanation on self-attention and trasformers
Great video, it would be nice to have a video of reinforcement learning in future ml tech talks.
The explanation is so clear, thank you.
Thank you very much ! Great video and very well explained. Yes a video about sentiment analysisfine tuning would be Amazing !
Very straightforward.
Thank you so much
Great presentation! Really easy to understand exaplanations of some hard topics, thank you.
Great presentation. Really well structured.
My favorite explanation so far. Great job.
super professional explanation of the topic! Excellent work!
It's very important library in nlp great work
Very clear explanation. Thank youuu
at minute 35 the video describes transfer learning, and it is said that during the fine tuning phase ALL the parameters are adjusted, not only the classifier parameters. Is that right? In contrast, when using a pre-trained deep network for a specific image calssification, I froze all parameters belonging to the CNN and just allowed the classifier parameters to vary
Excellent presentation of complex NLP topic.
She has done an incredible job.
This is very beautifully explained!
Agreed with all. This person should take the lead for other Google educational videos.
Thanks a ton for the explantion! Just wanted to ask how do we arrive at the values for matrices K, V and Q?
Great explanations, thank you so much for this video!
Glad it was helpful!
That was super interesting. Very clear explanation
I expected some wishy-washy feel-good "explanation", but I'm pleasantly surprised. So far the best explanation. Goes after the relevant distinguishing key features of the transformers without getting bogged down in unnecessary details.
Awesome.. great explanation. Thanks.
Great Presentation
Very nice topic discussion! Thank you 🙂
Amazing talk! very informative. Thank you :)
Excellent teaching !
Crystal clear .. Tnx
amazing video !
Great talk, really clear, thanks!
Also I see what you did "Thanks for your attention" 🤣
Great Explanation
Thank you
very good presentation!
well done video!
Simply great! 👏👏👏
Simply awesome
Can someone explain the inputs dict shown in the code at 42:15.
Nice, thank you ❤
Great presentation! Are the slides available for download? This would be fantastic. Thank you.
Thank you for shraing !
Where can I get the slides for this talk? Great talk
Great slides.
I came here from tutorials sections of tensorflow official webpage, but i get caught by her beauty
real help, thanks.
very nice
تحياتي الخالصة شكرا جزيلا
nice video
This is a fairly good presentation. There are some areas where it summarizes to the point where it becomes almost misleading, and at least very questionable: 1. Several other sources that I read claim that the Bert layers will have to be frozen during fine tuning, so I think it is still open for debate what the right thing to do is there? 2. This presentation glosses over the outputs of the pretraining phase. I think the output corresponding to the CLS token is pretrained with the “next sentence prediction task”. So, is this output layer dropped entirely in the fine tuning task? Otherwise I don’t see how the CLS token output would be a good input for sentiment classification. 3. The presentation suggest that the initial non contextual token step is also trainable and fine tunable. Isn’t it just fixed byte pair encodings? I know that these depend on frequencies of letters in the language but can these be trained in process with Bert? 4. This presentation equals transformers very silently to transformer encoders, and thus drops the fact that transformers can also be decoders. I think all initial transformers were trained on sequence to sequence transformation, and then the decoders were trained on next token prediction giving rise to things like GPT, whereas the encoders were trained on a combination of masked token prediction and next sentence prediction giving rise to the BERT like models.
Julia,
Your presentation has triggered a Eureka moment in me . What makes a great training video? Can AI help answer that. Here is a suggestion. Get a collection of videos and rank them by review comments. Using a large language model, find patterns and features and see whether there are correlations between the features and the views and review rankings. The model should be unsupervised. Some of the features can be extracted from comments
Interesting
Thanks for your ATTENTION 🤗🤗.. Pun intended!44:39
Please provide the slides
i love it
Where can I get the GitHub file
This is a positive comment. CZcams should let it past it’s sentiment filter.
Immaculate!
👍🏻👍🏻👌
okay what you are saying is completely vague . like for the query matrice you mentioned ( some other representation [why do we need another representation at all ?])
Thanks for not calling Sentiment Classification as Sentiment Analysis!
Sorry to say, but this was not very good. Key information is missing mostly the WHYs ? why is there a need for Query and Key Matrices? what is the main function of these matrices? How does the Attention function alter the Feedforward NNs?
I always find the face of presenter distracting when it is on the slides … can you just talk over slides instead of covering them with presenter’s face??