Thank you, sir, for sharing valuable information through your CZcams channel. Once again, I have a request: please create a series on how to apply Transformers to time series tasks such as anomaly detection, forecasting, or classification. Working on just one of these tasks would be sufficient for us. I have followed numerous articles, short notes, and videos regarding the application of Transformers to time series data, but it is still not clear to me. I am a beginner on this Transformer journey, and there are no useful videos available on CZcams overall.
Great video, but I think the "Likert scale" part around 31:00 is not correct. They scale the ratings for each rater separately, and they use those questions to detect sensitive topics and filter them.
Awesome video. A few comments. Is the architecture for the rewards model OpenAI used actually publicly documented? Can we be sure it is a GPT model? (I believe you mention it). I would love a contrastive deeper look at BERT vs GPT following what you mention. You mention it when you talk about stacking them up but it could make sense to talk about the what’s and whys a bit more. At 43:41 you say that the table is generated anew as being the reason for different generations even for the same prompt, but wouldn’t the table be the same, just the sampling strategy would result in the table being processed resulting in a different token? Overall: Maybe as a video title “making of the tasty ChatGPT sausage” would have been better :) The title may set wrong expectations for folks who casually discover your great content. From the title alone many people may expect to become heroes in using ChatGPT.
Is there ANY open-source repo that includes a tool for manually rating the results of your own ChatGPT like in steps 2 and 3 of this video? I want to *actually* train a TNN from scratch and I have the time to do it. Why do so few people talk about this aspect of ChatGPT?
Your channel deserves more. Thanks a lot
Thank you, sir, for sharing valuable information through your CZcams channel. Once again, I have a request: please create a series on how to apply Transformers to time series tasks such as anomaly detection, forecasting, or classification. Working on just one of these tasks would be sufficient for us. I have followed numerous articles, short notes, and videos regarding the application of Transformers to time series data, but it is still not clear to me. I am a beginner on this Transformer journey, and there are no useful videos available on CZcams overall.
Thank you for sharing this valuable Content, Great Channel
Great picture of how the GPTs work and what they are. Awesome 👍
Mind-blowing that all of this good stuff is free. Great video!
Can’t beat that price :)
Thks mate. I love the summary around 2:30!
Great video, but I think the "Likert scale" part around 31:00 is not correct. They scale the ratings for each rater separately, and they use those questions to detect sensitive topics and filter them.
Watching this while finetuning llama 2 :D
I think this may be the first gpt tutorial featuring RLHF
Awesome video. A few comments.
Is the architecture for the rewards model OpenAI used actually publicly documented? Can we be sure it is a GPT model? (I believe you mention it).
I would love a contrastive deeper look at BERT vs GPT following what you mention. You mention it when you talk about stacking them up but it could make sense to talk about the what’s and whys a bit more.
At 43:41 you say that the table is generated anew as being the reason for different generations even for the same prompt, but wouldn’t the table be the same, just the sampling strategy would result in the table being processed resulting in a different token?
Overall:
Maybe as a video title “making of the tasty ChatGPT sausage” would have been better :)
The title may set wrong expectations for folks who casually discover your great content.
From the title alone many people may expect to become heroes in using ChatGPT.
I have a question:
How does the GPT architecture provide outputs without an encoder? Doesnt the cross attention module need KV pairs from the encoder?
but why sft i.e. first step needed, we can do with it ofcourse it helps but not necessary right ? lets do 2 and 3 only .
Is there ANY open-source repo that includes a tool for manually rating the results of your own ChatGPT like in steps 2 and 3 of this video? I want to *actually* train a TNN from scratch and I have the time to do it.
Why do so few people talk about this aspect of ChatGPT?
can you make video about deepspeed coding implementation
Bahi is on Steroids after 100k.