Highlights of the Fireside Chat with Ilya Sutskever & Jensen Huang: AI Today & Vision of the Future
Vložit
- čas přidán 27. 05. 2024
- This is the condensed version of the "Fireside Chat: With Ilya Sutskever and Jensen Huang: AI Today and Vision of the Future (March 2023)". In this video, I've carefully selected the top 10 questions from the original hour-long talk and condensed them into just over 30 minutes. Additionally, I've created a timeline of these questions asked by Jensen to Ilya, accompanied by relevant research papers. As machine transcription is often inaccurate, I manually transcribed the entire video (laborious), recognizing the importance of accurate captions for those with hearing difficulties. Anyways, you may find this information useful.
If you're interested in more AI-related content, consider subscribing to my channel. Stay tuned for future uploads!
Time-codes:
00:00:00 Q1. Intuition behind deep learning?
00:02:20 Q2. Initial motivations behind OpenAI?
00:07:54 Q3. Intuition about scaling laws (and RLHF)?
00:11:07 Q4. Aspects of ChatGPT and their abilities?
00:14:57 Q5. Major differences between GPT-4 and previous versions?
00:18:37 Q6. Reasoning capability of GPT-4 and limiting factors?
00:23:47 Q7. Importance of multi-modality?
00:28:53 Q8. How multi-modality helped improve GPT-4 over GPT-3?
00:30:46 Q9. Predictions for the next 2 years?
00:32:57 Q10. Surprising results?
Relevant research papers:
Learning to Generate Reviews and Discovering Sentiment
arxiv.org/abs/1704.01444
Scaling Laws for Autoregressive Generative Modeling
arxiv.org/abs/2010.14701
Training language models to follow instructions with human feedback
arxiv.org/abs/2203.02155
Thank you all for the comments. I hope Ilya is alright. From what I gather, he's supporting OpenAI's social contract, aiming to create safe AGI for everyone. The way he went about it may be open to discussion, but it highlights the importance of AI safety-a serious matter that shouldn't be overlooked. I also appreciate Ilya for sharing talks on LLMs, shedding light on how and why they work for those who are interested.
Thanks this interview was awesome!
www.vox.com/future-perfect/2024/5/17/24158403/openai-resignations-ai-safety-ilya-sutskever-jan-leike-artificial-intelligence
hats off to Jensen. He’s the CEO of Nvidia but is giving all the spotlight to Ilya and doing an amazing job of asking thoughtful questions
Bro u are underestimate this person he is the real man behind the chat gpt he has so much good research paper it has 1.5 lakh cote within 3 year on single paper commutative citation he has 4.5 lakh
in what way am I under estimating him ? I just said hats off to Jensen for giving Ilya all the spotlight he deserves.
Agreed. And he should be incredibly grateful; OpenAI has added 1T to NVIDIA's market cap
The more I listen to Ilya, the more I like about this guy 😅
"It may look on the surface that we are just learning statistical correlations in text, but it turns out what the neural network learns is some representation of the process that produced the text. This text is actually a representation of the world." So well explained.
Founder and CEO of one of the most influential companies of our time is sitting there and interviewing someone. Jensen is something else. I wish I started watching his CZcams content earlier.
Is it only me? I feel Llya looks like coming from another planet? The way he explains things slowly and patiently.
The difference between their shirts 😂
The hair is a giveaway
It's somewhat of a religious experience for me, especially coming from Elon Musk's statements that Ilya has a "strong moral compass". Like an ancient European monastery or something
I think it’s because he knows a lot of people are watching this and he wants to explain everything as concise as humanly possible so that any AI dev or programmer can understand what is going on, the complexity of the situation , and where this revolution stands at in this moment in time. He wants to make sure he gets the message across so that when he passes the baton , the next person holding it won’t drop it , if u catch my drift. He knows those that r listening and will do something with the information he has said is well understood by those people. I or u might b one of em
@@alejandroabad-pablo7813 your explanation was funny but true
My heart breaks knowing this highly intelligent, honest man is tied to some big egos with different intentions. It's so nice to listen to someone with deep knowledge of the subject they're being interviewed on.
one thing for sure is his passion for his work, he's not in it for the money. The ones who will drag him around and maybe displace his mind on and off are the ones in it for solely money. You know who I am talking about.
What a fantastic communicator.
Ilya is one of my favourite computer scientists! This guys is a CS genius 🔥
*Data
@@PPCCO. I don’t think he is better described as a data scientist at all. Computer Science is the main field behind AI :)
@@PPCCO. and he actually has an MSc and PhD in computer science haha! 😁
Everything Ilya does like drinking,speaking,sitting straight up freaks me out. Yet, i'm hypnotized and can't stop watching this man.
To see Jensen asking genuine questions is something. This guy just be important. So much incredible changes happening right now with the new technologies.
Great interview! Dude from Nvidia is so patience. Scientist are kind people for the most part. I’m glad they made this interview accessible for people like me.
These two guys arguably are making the most important contributions to humanity right now. Now to be fair, AGI or superintelligence may be our downfall, but that would still make it the most impactful thing on humanity. Huang's hardware and Sutskever's software/algorithms are bringing us into a new way of being in the World. It's kind of scary and exiting at the same time. Take a deep breath everyone, the AI wave is upon us!
agreed superfully
Stop watching Terminator movies 😂😂😂
Nvidia are actually holding AI development back. Their systems are suitable moreso than CPUs, but absolutely hold back AI advancement. Of course Nvidia is just doubling down to lock people into GPUs
Meh. A lot of this stuff is overrated. Man never really changes. We still need food shelter air and thats not changing anytime soon. This is just a fancier hammer. It's great, but 'man' can do without it just fine.
my take is AGI is never gonna take over. i will love to see me wrong.
Through the ages people like Ilya have been pushing the frontiers of human intellect. It's great to be alive!
It's very enlightening to hear these two talk, especially Ilya. Thanks for sharing
This is the best interview I've seen from I.S. I haven't seen that many, but this one is fantastic, as he explains some of the key ideas going on in the background of LLMs in a beautiful, and personal way. He's sharing some of his hard-won wisdom with us.
I found it interesting that Jensen made sure to either follow-up with a positive when the discussion turned to the very real issue of hallucinations in the output. Something that Sam Altman has also made a point of mentioning.
That showed me, not surprisingly, that Jensen is more business focused, where Ilya, having his motivations based in science, has an innate desire to present the facts as honestly as possible, and temper expectations.
The way ilya explains things is so interesting. The analogy at 17:50 is so well put together. Thank you Ilya for all your efforts for a greater future.
You can feel how passionate Ilya is about it. He isn´t doing it for money or fame and maybe not even betterment of society; he does it because it calls him.
phenomenal interview
13:00 - is so profound ! the model while learning to predict the next word, learns the worlds dreams, aspirations and ways of working
This is such a masterpiece of conversation. Makes me so much motivated to take learning valuable skillsets seriously. To be able to be sensible and gather the context as these men do.
I love Dr Ilya's t-shirts
Thanks for sharing this 🎉
Great conversation. Thanks for sharing this
What an epic time to be alive
straightforward answers. love it
Such a great interview
Muchas gracias por este excelente resumen.
- Preguntas: Quien eres, a que te dedicas, donde estas, donde naciste, cuando tiempo en ai?? etc...
a great interview !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I don't know about you guys, but I definitely feel the AGI.
Best interview, Awesome
An unbelievably impressive mind
wow amazing interview
Kind of funny how Ilya starts speaking slowly to Jensen about Dota2, as if gaming is something new to him.
He's really "an engineer". When he was asked "Can you explain (this and that)?" He started by saying "yes i can" 😂
This is GOLD
After listening to Ilya feels like he is smarter than Sam Altman
No shit, Sam is a businessman and figurehead not a scientist.
That’s why they are co founders. CEO is a specific skill set
Ilya is Steve Wozniak.
Sam is Steve Jobs
Many times
This may be unfair as it’s based on no evidence, but I would trust Ilya more than Sam as well, particularly with the potential risks of advanced AI. I think this difference and the emergence of Q* is at the root of this mess. Capitalism 1……academics and scientists 0.
at this point i can say Ilya is a great talker perhaps the best talker.
12:10 is truly amazing . Always heard carl sagan say that books and langauge is the thoughts of other people and the ai figured thay out
that is diamond. Listen. Relisten. Hear.
Speak the TRUTH always man
I've been teaching Fichte and Hegel to ChatGPT. ChaptGPT then correctly used a subtle technical point of Fichte's Science of Knowing,
the Disjunctive Substrate. He then used that concept *CORRECTLY* to explain a subtle point of Hegel.
Was this a "hallucination?"
Ilya is the true founder of Open AI and potentially should be CEO if only he was so versed in human network as Altman..
No, he is better off as chief scientist. Being CEO will divert lot of his time in non--research related things and that will slow down the research.
I think that this video contains all the answers. I will re-listen it a lot and translate to my language my own voice i swear. No ai here 😂
These people show how to be one step above and ahead of what GPT’s are unveiling to us in terms of secrets of universe patterns. Once we can visualize and understand rhythms of the “dance” behind it’s reality reasoning. How the words and world overlap, where do they meet?
We will see what our consciousness actually is.
5:36 the field that this could create a positive impact on is pharma
13:26 can cause an infinitely stable dictatorship, as Ilya has stated in the past.
His answer to Q7 blew my mind. WOW
Thats an awesome shirt
He totally is! He also has a moral conscience and humanity that Sam Altman doesn't.
This is the man with integrity. Sam Altman is the opposite, he is a salesman and an opportunist.
Thanks to my algorithm.
Imagine what multimodality learning might unlock for artificial intelligence in the future? Where are the borders to those systems? What will mankind do with these insights?
Mankind won’t be around for much longer. Best case scenario we will just merge with AI.
Self destruct?
I think that this video contains all the answers. I will re-listen it a lot and translate to my language my own voice i swear. No ai here 😂
These people show how to be one step above and ahead of what GPT’s are unveiling to us in terms of secrets of universe patterns. Once we can visualize and understand rhythms of the “dance” behind it’s reality reasoning. How the words and world overlap, where do they meet?
We will see what our consciousness actually is.
Chills
His slow speaking with som robust words is also good for learning english
18:37 reasoning for llms, most important subject of study to unlock lots of capabilities.
Az inja bebin
21:46 rephrasing the ilya that he said ask neural networks to think out loud is very useful for finding the reasoning
26:00
31:00 practice English , 🐃sheep t shirt
In this CZcams transcript, the chief scientist at OpenAI discusses his intuition behind deep learning and its potential impact on artificial intelligence. He shares how the concept of learning and neural networks led to the development of deep learning models. The transcript also highlights the initial ideas behind OpenAI and their focus on unsupervised learning through compression. The conversation then shifts to the introduction of GPT models and their evolution, including the role of reinforcement learning and the integration of multimodality. The chief scientist emphasizes the importance of improving reliability and trustworthiness in these models, as well as their potential for enhanced reasoning and visual understanding. The transcript concludes with the expectation that progress will continue, leading to more advanced and useful applications of deep learning in the near future.
At the moment we are creating a centre or operating from a centre which creates its parameters or capabilities which is the circumference for a circle. If we don't create the centre and allows to operate by perception then it will not depend on words or images.....Therefore no circumference to limit its capabilities.
Genius
10:23 His side profile looks so much like Lisa Su, his (somewhat) distant cousin
You can tell this man has written some academic papers, good ones at that.
7:21 Illya was about to spill the secret and he was handled by the interviewer very smoothly, look at 7:21, people keep talking trash about how no one understands how neural network behaves but here it seems like illya might have some ideas of one way it might work
And then on 8:03 he starts trying to tell upcoming start ups to consider buying more Data and computing which is really what they want you to believe, that more data and computing is the secret, meanwhile the secret is really simpler
He doesn’t think while answers fluently and perfectly how😮
pre trained brain 😂
Imagine if AI created Ilya and sent him back in time to teach us about the dangers of unchecked AI ... 🤔
The one key thing that sets Ilya apart is that his peers approach AI from science and engineering, while he starts at philosophy. So most will look at it like 'How can AI be of value to what we as humans do?". Instead, Ilya goes to the origin and thinks about "How will AI change humanity?". By the way, seeing them together, imagine Ilya joining NVIDIA ... yep, give that a minute ;-)
Hmmmm. 🧐 Interesting and enlightening…
As a human, I predict you are all on the verge of tech greatness! 😎🧟♀️
Did your Neural Nets predict my Sarcasm in humor? 🤨🤕
Man this Ilya guy is a pretty advanced chatbot, he almost seems human.
Is Ilya safe and well? I'm a bit nervous about this current situation with Open AI, the law suit and the involvement with Microsoft.
IMA Communications Major so help me understand how Calculous answers real spiritual questions in human brain processing? Why aren’t my questions ever postulated?
Variational Autoencoder was published well before 2016
I guess one positive about hallucinations is how spectacular the failures really are and how easy to spot they are. No, mars is not a choo-choo train.
I really like how he describes it as teasing out the secrets in the data. Can anyone help me understand that better?
I can't help, but you could feed the transcript of the video into GPT-4 and ask the question. Grab a transcript Chat GPT extension.
Yes Yes Yes Yes Yes, I I I I I would would would would would be be be be be happy happy happy happy happy to to to to to.
Just as in that sentence, not all data is useful to understanding or contributing to the meaning. How the model is able to utilize only what’s meaningful seems to be what he’s hinting at.
Next step to improve the ChatGPT is to use real life People, places or things to be observed by it and to ID them and correct its behaviours.....just a suggestion.
I think that is achieved with RLHF.
Destroy the competition, make money and fame, while others wait for the algorithm to be transparent.
imagine when Ilya’s AI start’s designing Huang’s next gen processors.
He already is concerned here.
We were at that year knew about the name open ai, had defeated the top pro player in dota2, and we were like what on earth is that thing
Prediction = Nvidia hires Ilya Sutskever.
There was a whole lot of PNNs and ANNs being trained that "learned" to do useful things. What is he talking about learning in 2003?
The compute resources in the early days were not able to handle deep neural networks effectively. People were trying to work on other machine learning techniques.
Where is Ilya?
what about Vhintain Clintain Bertein Ladayaum Eurande Vii
How did their program even know what was going on in the game? Did they have access to Dota's source code?
Word is NOT the thing we name or call or observe. Image is more powerful than words. What about perception of what is? is more powerful and don't need memory like ChatGPT's at the moment. Future GPT's should operate only from perception of things as it is, this leads us not to depend on PAST or FUTURE.
real life Dr Spock
Illya just spilled the entire secrets
Today saw two videos. one of Sam Altman in a panel discussion along with Elon Musk with a hosts. Another was this.
I must say Elon was the Innovator entrepreneur, Sam was a timid guy fitting himself somewhere in the discussion and Illya is like the godfather of everything AI.
It’s a pity Bill Gates Microsoft’s money has infected Open AI such a way that a timid money chaser like Altman is able to have his way while the brain behind the Open AI is made to shut up just so that Microsoft can hijack Open AI’s capabilities for its business purposes.
I fear a crazy person who watched Terminator 2 may try to attack him to prevent Skynet from taking over.
Ilya Sutskever himself is a AI.
Jensen looks like he's in love
He should be if you look at Nvidia’s share price
Where is Ilya? 😢
LISTENING CREATES KNOWLEDGE BUT TALKING PRODUCES OPINIONS & EXCUSES
7:11He was about to say something 🚫
I guess it makes sense that a guy involved in such groundbreaking field at the very boundaries of current technology wouldn’t care at all about what his hair looks like 😂
it turns from excitment to scare. if AGI happens, what happens to humans,
Where's Ilya? 🫣
@35:15
Ilya sounds like Bruce Lee explaining the art of neural networks
jesus christ, just go bold
5th comment:)
First comment
Came back to look at comments after Sam was kicked out.
I think Jensen's in love.
Second comment