Deep Learning Basics: Introduction and Overview
Vložit
- čas přidán 1. 06. 2024
- An introductory lecture for MIT course 6.S094 on the basics of deep learning including a few key ideas, subfields, and the big picture of why neural networks have inspired and energized an entire new generation of researchers. For more lecture videos on deep learning, reinforcement learning (RL), artificial intelligence (AI & AGI), and podcast conversations, visit our website or follow TensorFlow code tutorials on our GitHub repo.
INFO:
Website: deeplearning.mit.edu
GitHub: github.com/lexfridman/mit-dee...
Slides: bit.ly/deep-learning-basics-sl...
Playlist: bit.ly/deep-learning-playlist
Blog post: / tke476jw2t
OUTLINE:
0:00 - Introduction
0:53 - Deep learning in one slide
4:55 - History of ideas and tools
9:43 - Simple example in TensorFlow
11:36 - TensorFlow in one slide
13:32 - Deep learning is representation learning
16:02 - Why deep learning (and why not)
22:00 - Challenges for supervised learning
38:27 - Key low-level concepts
46:15 - Higher-level methods
1:06:00 - Toward artificial general intelligence
CONNECT:
- If you enjoyed this video, please subscribe to this channel.
- Twitter: / lexfridman
- LinkedIn: / lexfridman
- Facebook: / lexfridman
- Instagram: / lexfridman - Věda a technologie
First lecture in the 2019 deep learning series! It's humbling to have the opportunity to teach at MIT and exciting to be part of the AI community. Thank you all for the support and great discussions over the past few years. It's been an amazing ride.
You are Awesome, sir!
I was waiting for it 😬🤗
Go Go Lex!!! This is Awesome! Best way to start this year
This is the best AI talk I have seen, I’m looking forward to developing my skills. I have so many ideas to tackle some of the harder questions and some issues I’ve noticed in training models and data gathering which I think are currently flawed.
@Mohit Sharma We're releasing tutorials on our GitHub repo: github.com/lexfridman/mit-deep-learning
3 years later..he never would have guessed he would be best buds with Joe Rogan, David Goggins and interview Ye and others. Crazy
shows you if you're disciplined, a real human with a heart, and grind will get you to your goals. I am too dumb for this video.
For reals!
Can AI be used in predicting lottery pick3. I have a whole unique method that needs deep learning aid.
@@dyfrigshandy Thanks though don't konw what it is.I have been researched it for a decade.I wanna share with people with the same hobby.
@@sandigoletic7204 And still scared to post interview with Andrew Tate : d
When she says “go deeper” but you’re all out of PowerPoint slides
So true
You Made my Day!
@@Adriano70911 stfu
@@Adriano70911 stfu
😜
0:48 Deep Learning Basics Summary
5:00 Visualization of 3% of the neurons and 0.001% of the synapses in the brain
6:26 History of Deep Learning Ideas and Milestones
9:13 History of DL Tools
11:36 TensorFlow in One Slide
13:32 Deep Learning is Representation Learning
16:05 Why Deep Learning? Scalable Machine Learning
17:10 Gartner Hype Cycle
18:18 Why Not Deep Learning?
21:59 Challenges of Deep Learning
29:20 Deep Learning from Human and Machine
30:00 Data Augmentation
31:36 Deep Learning: Training and Testing
32:10 How Neural Network Learn: Backpropagation
32:28 Regression vs Classification
32:54 Multi Class vs. Multi Label
33:13 What can we do with Deep Learning?
33:45 Neuron: Biological Inspiration for computation
34:14 Biological and Artificial Neural Networks + Biological Inspiration for Computation
35:55 Neuron: Forward Pass
36:40
Combining Neurons in Hidden Layers: The "Emergent" Power to Approximate
37:37 Neural Networks are Parallelism
38:00 Compute Hardware
38:27 Activation Functions
39:00 Backpropogation
40:07 Learning is an Optimization Problem
41:34 Overfitting and Regularization
42:58 Regularization: Early Stoppage
44:04 Normalization
44:32 Convolutional Neural Networks: Image Classification
47:52 Object Detection/ Localization
50:03 Semantic Segmentation
51:27 Transfer Learning
52:27 Autoencoders
55:05 Generative Adversarial Networks (GANs)
57:03 Word Embeddings (Word2Vec)
58:58 Recurrent Neural Networks
59:49 Long Short-Term Memory (LSTM) Networks: Pick what to forget and what to remember
1:00:15 Bidirectional RNN
1:00:50 Encoder Decoder Architechture
1:01:38 Attention
1:02:10 AutoML and Neural Architecture Search (NASNet)
1:04:40 Deep Reinforcement Learning
1:06:00: Toward Artificial General Intelligence
Thank you
You're doing gods work
You are awesome... May many great things go into your life.
This was quite nice to take time so we could save some :). A selfless creature, indeed!
Oliver Woods no, his friend is, however he is allowed to read his slides and present the lecture as he holds a degree in the liberal arts
I slept listening to you this morning and saw my mom reading deep learning books in my dream.
Lmfaoooo 😂😂
Your unconscious is telling you to learn
TMI. Abit TooTMI
Whoa
This means the genes on your mother's side are pushing you to learn, improve, overcome. She is saying "you, my son, are the future of intelligence in the universe... for good... or for ill" [ominous music intensifies]
This might be 4 years old but it is still incredibly helpful in understanding the current state of ML and ANN. Thank you Lex.
I really admire the work that Lex is doing both at MIT and his podcast!
then you are a dummy
then you are a clown@@mkballer4502
Thank you so much Lex. This will help us a lot. This will help the students, who cant afford paid online courses and none in the neighbourhood can teach.
Watching this on 2023, after the advancements of generative pretrained models, is mind-blowing. Things advanced so much in 4 years.
Superb lecture. The guy speaks as if he sell dreams.Great confidence and knowledge
This lecture is awesome and really inspiring. I've been a fan now for years now Lex, and I'm really happy to see your success. I just wanted to point out that I believe your analysis of "One Shot Learning" re: human bipedal locomotion might be a little off base. The learning and development process that leads to bipedalism is characterized by a list of precursors like crawling, sitting up, and standing up. This process takes usually between 1 and 2 years. This time (and the hundreds if not thousands of reps that come with it) is needed to build from the ground up both the requisite muscular strength and the requisite neural pathways for these coordinations to be possible. The process can be accelerated through coordination-specific training on the part of the parents (which occurs quite often). Errors that occur in this process lead to hardcore biomechanical problems down the road (e.g. requiring knee replacement at 55) Bipedalism is pretty complex, and is way harder than quadrupedalism, which would fall more in the scope of your one shot learning claim.
Loved your post.
Let your child crawl to build their core strength before you worry that they aren’t standing yet.
Putting diapers/nappies on a crawling child is similar to hobbling a horse.
Think about it.
The longer they crawl the better they will be able to walk.
Obviously to let them crawl longer and without a massive chunk of material forcing misaligned muscular development is a huge inconvenience to the care giver.
Prioritise your goals.
Amazing talk! Thank you, Lex! What an exciting time to be alive...
Thank you for your honesty, Dr
Fridman. Brilliant and thought -provoking to those who can ask questions to answer.
Lex,you are amazing as a lecturer and a finer example of a loving human. Your voice is so deep assertive and clear to the audience
You're handsome with good attitude ,body language and can easily connect with people. I pray God bless you and family with blessings because we need you.Congrats man.
I don't exactly know why, but I am so proud of him.
Both as a human and as a person who still puts efforts to not let knowledge become the source of cynicism. There's something about not giving up on love and other intellectually ridiculed concepts such as kindness. There's something pure about it.
And for that purity, I am so proud of him.
Electrical and computer engineering student here who's doing Jiu Jitsu as well. You can imagine how big a fan I am of Lex. So cool to see him actually going into the technicalities of his work.
Same here lol
He’s just like me fr! Headass
Real
@@wrestlingscience deadass no cap
@@wrestlingsciencelol
I have attempted to meditate many times in my life and prior to this CD the only success I've experienced is with live guided meditation. czcams.com/users/postUgkxzpa8CIfZcihW4Z0F_ja0QF3W9KIatrsq This is the first CD I've used that cuts through my unmedicated ADHD and enables me to truly relax and experience a quiet and energizing interval. The instructors voice is very soothing and pleasant to listen to. I am easily able to sit successfully through the entire CD, and for quite some time after. I cannot adequately express how tremendously helpful this CD has been on my spiritual journey!! Two thumbs up and 10 stars!
You know what lex will revolutionize the world..a great scientist and a fluent speaker,it always a pleasure to listen lex😍😍
It’s super helpful to know how AI systems work, even though I don’t work in tech. It also helps me feel relieved to know that AI is still very far from becoming sentient. I didn’t realise just how amazing the human brain is in comparison.
I have never heard a technical course so poetic!
Lex is a really admirable professor applying academy to solve real world problems through engineering ways. Kudos!
like pretty much all engineers? LOL
He can’t speak properly
@@Dadaadad268 I think we can
He isn't a professor
He is not a professor.
Thank you so much, Mr. Lex Fridman, for contribution and sharing your lectures!
This is a great rundown of the general DL basics. Really good lecture
Dope lecture. Good coverage. I love the hidden point that performance depends on smaller batch sizes, which means higher sample rates (to me), Data is capital.
So talented, this guy should make his own podcast
Thank you for being such an amazing source of information and learning.
Been a fan of your podcast for a while. Really puts you in a while new light to see you yeah. You really seem in your element teaching!
Thank you Lex for all your contribution and for sharing so much on CZcams. My life would not be the same with our you podcast series
I have school exams to read for.. but this video is more exciting to watch
This is an extremely useful resource; thank you for sharing this!
This is amazing Lex! Superb FREE content so the cat and let curiosity kill it, over and over again, loving every secret
Great explanation. This is the first lecture in which I am able to understand very easily. The way of explaining is mesmerizing.
Great resource Lex. Thank you for sharing. Keep them coming :)
The best part was the honesty, on possible secundary effects that Deep Learning might do... none the less, we should definetely go ahead with Artificial Intelegence, never forgeting that C language is always there if we need to take a step back :)
Thank you for sharing this on CZcams. This is what gives me hope in todays world. The walls that surround knowledge are coming down. Go team PEOPLE.
100%
Like the berlin wall fell, the wall for knowledge must fall..
every content you put is a gem Lex!
Thanks lex for your sharing. So I can follow this training from Turkey. I wish you success. Good Work.
Me too
Adamsın reis
Great video, thanks for this Lex
thank you so much Lex! We from all over the world who can't afford to go to MIT, can get the same what your students learn!
Thanks for sharing. My daughter is the frenchwoman in MIT, majoring Computational and System Biology.
What's a clear explanation! That is a real professor!
Lex Fridman is absolutely fucking winner. Winner doing winner things. Is there a human being on earth who doesn’t like the guy? What an awesome blessing of a human being. We need more
facts
Sam Hyde dont like Fridman and he somehow right.
This guy should starts a podcast. I am sure it would be popular.
Good to see you teach..a teacher who is a continuous learner
I am very fond ot the interviews in your podcast. Born 1961 I started my academic career studying computer science.I was one of those guys who chose the subject, because I could perform well above average with little effort. Now I am a lowly catholic priest still interested in all kind of science. Had I stayed in the computing business, I would have specialized in data modeling, data mining and data visualization. The lesson raised some philosophical questions with with practical consequences what I would like to research in your line of work.
1. The philosophical issue raises with the very definition of an information bit representing an yes/no answer to a given question. The most important thing in the whole computation/data business is to select the right questions and a well enough working way to answer them. I think it weren't Biil Gates abilities as a programmer which made him successful, but the set of questions he wanted to provide solutions.
This is where I draw the line separating human intelligence from artificial intelligence. Human intelligence is about selecting the right questions. Once that is done and there is some relation to computable empirical data, I think AI will outperform human efforts as it develops. I always expected AI to become superior in games like chess or poker, because those games are inherently digital, based on a restricted set of predefined questions. (i.e. Is there a white queen on square e1?')
Training an AI somehow expands the limits I assumed as given The training of an AI creates a layer of abstraction, something I previosly saw as purely human.
2. If I would research AI I would try to visualize that abstractions. I would implement functions like: 'Draw many different cats' if the AI is trained to recognize cats or 'Draw many different pictures similar to cats and equally similar to dogs.
Then I would try to understand what the AI perceives as cat-like and if there is a better recognition when repeating the learning with the AI-generated examples.
Has someone already tried this strategy?
Did it work?
3. I am a fan of Gregory Batesons theory of 'binocular learning'.
So, when researching autonomous driving I would experimentally use two cams with two AIs interacting like the two sides of a human brain and try evaluate if I implemented something to generate the added knowledge Bateson describes as result of comparing different descriptions of the same.
If successfuly generating Bateson's additional value I would try to understand, if there is a general difference between humans and AIs when generating deeper understanding by using that method probably based on Bateson's levels of learning.
Has anyone done research like this?
What were the results?
I guess, using multiple input-devices (i.e stereoscopic cameras or combining different electro-manetic wavelength cameras) will greatly improve the reliability of AI's results, while using multiple interconnected AIs will mainly improve the researcher's theoretical understanding by 'listening' to AIs 'discussing' their abstractions.
If answering my questions don't go far beyond 'yes' or 'no' because I never invested any time to understand neuronal networks or AI.
Thank you so much, Mr. Lex Fridman, for contribution and sharing your lectures!
Great resource Lex. Thank you for sharing. Keep them coming :)
Hi Lex, Thank you for this amazing video!
A great introduction lecture! Full of “fruit”, I learned a lot in just a hour. Thanks a lot for sharing!
Loyalty's
Business
@@dezziepierce4769 ??
Curious to know what you learned from this
@@coop4476 And how. That's what we want to find out.
Thanks a lot, Lex, for sharing this series video
Clean, clear and realistic lecture!
This lecturer has a good voice. He should start a podcast or something
One of the most precise lectures since my Engineering school times. Would love to hear more from you.
wow!
I've been listening to Lex's podcast for a while, this is the first time I have audited one of his courses. I think he is starting to remind me of the Carl Sagan of our age.
I'm so excited to join this class!
Lex Friedman has become an inspiration to me greatly
Such a dense intro. Love this
Greetings from Australia. Amazing class. Take care and thank you.
I would pay this man $$$$ just to keep pumping out lectures weekly
this is a very interesting lecture, thank you so much for making it available to a wider audience. are the other lectures of the series also available online?
It's interesting watching this lecture at the end of 2022, and seeing just how many problems deep learning has solved since this video was released. At 27:43, we've already reached Art and Book Writing, and are well on our way to a few others. And yet self driving hasn't advanced much at all.
Well, don't go by Tesla. Self driving has advanced a lot even since this lecture. Veritasium has a great video on this.
The mark of a master is that he/she makes the complicated simple ... not simplistic ... but simple enough for the uneducated to be able to appreciate the major points. Thank you, Lex.
Also, someone who I assume is not Lex drew me into a strange WhatsApp conversation that I terminated because the language was cryptic and not at all characteristic of Lex. You might change your CZcams password ... me recommending an MIT faculty to change his password. Just trying to help preserve your brand equity and the trust we place in you.
A tour de force in the selection, organization, and presentation of an overview of Deep Learning. I really enjoyed it - thanks for doing this and making it freely available to everyone!
It's pretty amazing to see how excited "nervous'' he is to do this lecture, just as much as most of us are to learn this topic. :D
Thank you Lex for making the outstanding lecture. Hats off!
English is not my first language but your voice is clear and pronunciation easy to understand. Keep up the good work.
Piggyback on Mr. Sherma's comment: "English is my first language,But your voice is clearAnd pronunciation isn't too understand.Keep up the good work"
"English is my first language,But your voice is clearAnd pronunciation easy to understand.Keep up the good work"
it's awsome to know the evolution and new skills of deep learning in this course!
I had no idea that you are/were a professor, and a great one at that.. thanks for sharing this video
Awesome presentation! Thank you very much for sharing it.
I love your teaching style, @Alex Fridman
Thanks Mr. Alex for your piece
imho the best lecture to watch in january 2019
Agreed
lex is the best sad i never had a teacher like him
...Lex seems to be an angel, amazing person!
Awesome, Thanks! Can't get enough.
It's cool to see you in your element Lex!
"welcome everyone to 2019, it's really good to see everybody here"
Time travellers?
HAHAHA good point.
I'm going to say "Welcome everyone to 2021... you survived Covid-19 and Trumps' incompetency."
@@LadyCoyKoi
Trump saved America.
God bless Trump
We're all time travelers. I've never met anyone stranded to one moment in time.
@@c1dv1c1ous You've never been to one of my lectures then
Thanks a lot for sharing such a good and well done material
Honestly, thanks a lot for this professor.
"All kinds of problems are now in digital form" man, that was deep!
Waiting for the lecture held today. :)
Love your work Lex.
Thank you! This is gonna be great.
Lex, thank you. You are a legend.
The way you said course 6.S094 make you sound like a awesome robot professor Lex!
Thank you for sharing knowledge
imho the best lecture to watch in january 2019
Very intelligent man bjj player as well
Beginner > Hazard > Expert Love it!
You sir rock, great lecture. Thanks for sharing!
Great overview ! Thanks Lex !
Amazing lecture. Lex, you are a legend. Thank you .This runs at x1.25 really well too (for the busy minds out there....)
Thanks for the tip. Worked well :)
Lex you are so old school it's great
Lex, Great Structure and coverage.
You are breathtaking Lex
Wow, what a guy. Thank you for sharing this video. Very well put together and engading lecture.
Hey man, love your videos. Saw you first on Joe Rogan and been watching ever since :) Keep it up.
Great lecture! Thank you. 👍
Nice! Realy Lex is doing a great job. Lex's podcasts are very nice I listen to them every week. I suggest you should also watch ..... you will get an amazing experience with Lex ........ :)
I get weird feeling when I hear lex talk. There's something that binds deep learning, media programming, and overall take over of a free thinking society. The way they collect data will not change. The population will change to make it easier for them to collect data and have a control.
I wish I could watch an entire course by Lex :)
was wondering about the same. but i guess thats not available online
Thank you so much for contribution and sharing your lectures
Great introduction. Thank you very much for sharing it.
The timing was perfect! I am writing a paper on deep learning.