Moore's Law is Not Dead (Jim Keller) | AI Podcast Clips
Vložit
- čas přidán 8. 02. 2020
- Full episode with Jim Keller (Feb 2020): • Jim Keller: Moore's La...
Clips channel (Lex Clips): / lexclips
Main channel (Lex Fridman): / lexfridman
(more links below)
Podcast full episodes playlist:
• Lex Fridman Podcast
Podcasts clips playlist:
• Lex Fridman Podcast Clips
Podcast website:
lexfridman.com/ai
Podcast on Apple Podcasts (iTunes):
apple.co/2lwqZIr
Podcast on Spotify:
spoti.fi/2nEwCF8
Podcast RSS:
lexfridman.com/category/ai/feed/
Jim Keller is a legendary microprocessor engineer, having worked at AMD, Apple, Tesla, and now Intel. He's known for his work on the AMD K7, K8, K12 and Zen microarchitectures, Apple A4, A5 processors, and co-author of the specifications for the x86-64 instruction set and HyperTransport interconnect.
Subscribe to this CZcams channel or connect on:
- Twitter: / lexfridman
- LinkedIn: / lexfridman
- Facebook: / lexfridman
- Instagram: / lexfridman
- Medium: / lexfridman
- Support on Patreon: / lexfridman - Věda a technologie
![[실시간] 전철에서 찍힌 기생생물 감염 장면 | 기생수: 더 그레이](http://i.ytimg.com/vi/4nkFvwxGAfA/mqdefault.jpg)
Btw. For people who don't know. This is the guy who designed AMD's Ryzen architecture.
Mike Clark?
and?
not really designed, but he was a main lead overseeing a bunch of teams. He himself doesn't like to be credited as a creator of this architecture.
@@gickygackers My god is that not enough?
A bigger FYI, it wasn't just Jim Keller, there was also Mike Clark, Suzanne Plummer and a slew of other people. Jim gets too much recognition imo.
"The number of people predicting the death of Moore's law doubles every two years."
-Peter Lee, Microsoft
I heard that the number of Elvis impersonators is growing exponentially. One day we'll all be Elvis impersonators even if we don't like his music.
@@jeremywvarietyofviewpoints3104 There are plenty of professional cosplayers that aren't familiar with their characters. But that's mainly because they cosplay many characters.
😂😂😂😂👌👌 good one
I am a bit annoyed that a lot of really amazing silicon was wasted on phones that don't at all need it.
Whenever I begin to get confident in my own intelligence I like to listen to guys like Jim Keller talk so I can be pounded back to reality.
One can always feel confident in your own intelligence but once you get comfortable with how much you think you know that's just a recipe for disaster...
He's obviously the best at designing CPUs , but Expertise and intelligence are different things.
I'm pretty sure experts in other fields would disagree with some of his opinions.
@@kaba_me No. He doesn't merely have knowledge, he is clearly highly intelligent as far as I can tell. Using abstract concepts to solve complex problems is not merely a matter of having knowledge
@@museitup4741 I don't doubt his intelligence... I doubt his expertise in other fields.
@@kaba_me You are correct, there is a huge difference.
Jim is like the Rambo of integrated circuits.
Imagine if Joe Rogan interviewed Jim Keller? Their gap in knowledge would be so great that they would form a black hole in between them.
I thought the same. We need to get Jim on Joe Rogen
I'm imagining Eddie Bravo poking his head out of that hole like a loony toons closing animation saying "space isn't real"
Best
It would probably be a great interview, he is a master interviewer. His forte
Renier Van Rensburg And Lex is a master debater! 😂
Meanwhile, the JavaScript developers are making sure that any hardware progress is nullified by adding more NPM dependencies
Cesar Canassa leave us alone
npm: **feels bad man**
LMAO! XD
Node is pretty fast, the problem is the Golden Hammer law, Node was born as a server-side tool for building non-blocking custom services.
Today is pretty much shoved everywhere, transpilers, syntax checkers, desktop apps (electron), game-engines, which is usually what happens when a thing becomes popular.
LOL!
@@rewrite1239 I tried building rust for Android once, regretted wasting 10 days on it.
This guy does not break eye contact and continues conversation without skipping a beat. I don't think I could do this in a dream.
It's a learned skill. You'll get it eventually
@@xzcsdf9574 Like many things, simply practicing and not getting in your own way by telling yourself you can't do something leads to progress.
He also sounds like he is recovering from a hangover.
@@beachboy_boobybuilder idk if you’re Albert Einstein’s son, if you sound like this recovering from a hangover.
He's used to being the smartest man in the room
Lol, this got confrontational, but in a totally academic kind of way, then got instantly resolved. Good chat.
I don't know if confrontation is a thing on Lex's part of the spectrum.
@@poneill65 He's a grappler bro.
@@SwaggySolidarity So? Doesn't require eye contact. He's just running a grappling algorithm,.. "I opponent leg leg goes there, put right leg there,..." etc, etc. The chap clearly has a completely "spectrum" typical aversion to eye contact, with forced token engagement with his interlocutors.
Lex was right though, a Convolutional Neural Network (not to be confused with just convolution) is a search. The CNN, like all searches, is trying to to find a given pattern with within the confines of a grouping of data. The only difference between it and CNN search is that it breaks the main search up into small functions that look for traits and features and each search returns a probability of discovery. This result is then used in the next search to hone in on the least inaccurate guess as to whether or not the object was found.
A CNN is still a search though.
I really enjoy when people ask for more, because that means I get to know more and that's a good thing no?
Listening to Jim Keller talk is THE most mind blowing experience. Lex you do a great job asking questions and keeping up.
May's Law states, in reference to Moore's Law:
Software efficiency halves every 18 months, compensating Moore's Law.
David May (born 24 February 1951) is a British computer scientist. He is a Professor in the Department of Computer Science at the University of Bristol and founder of XMOS Semiconductor, serving until February 2014 as the chief technology officer.
May was lead architect for the transputer. As of 2017, he holds 56 patents, all in microprocessors and multi-processing.
When Inmos was formed in 1978, May joined to work on microcomputer architecture, becoming lead architect of the transputer and designer of the associated programming language Occam. This extended his earlier work and was also influenced by Tony Hoare, who was at the time working on CSP and acting as a consultant to Inmos.
The prototype of the transputer was called the Simple 42 and was completed in 1982. The first production transputers, the T212 and T414, followed in 1985; the T800 floating point transputer in 1987. May initiated the design of one of the first VLSI packet switches, the C104, together with the communications system of the T9000 transputer.
Working closely with Tony Hoare and the Programming Research Group at Oxford University, May introduced formal verification techniques into the design of the T800 floating point unit and the T9000 transputer. These were some of the earliest uses of formal verification in microprocessor design, involving specifications, correctness preserving transformations and model checking, giving rise to the initial version of the FDR checker developed at Oxford.
In 1995, May joined the University of Bristol as a professor of computer science. He was head of the computer science department from 1995 to 2006. He continues to be a professor at Bristol while supporting XMOS, a University spin-out he co-founded in 2005. Before XMOS he was involved in Picochip, where he wrote the original instruction set.
Man the level of intelligence this man is so high it made him so humble, like what I do might not even matter as much in the grand scheme of things. 👏🏻
that happens with any knowledge you gain. the more you know, the more you know how much you don't know.
Last part of that interview was actually chilling. Absolutely incredible interview
I love this! Two extremely intelligent people talking about complex things in a way that even dumbasses like me can follow! Cheers guys!
Keller really puts it into perspective when it comes to layering from a simple switch to an infinity of equations. Impressive.
Jim is such a humble and intelligent guy. He seems like he would make an incredible teacher/mentor if he wanted to go that route someday.
He's a teacher/mentor for hundreds of Intel employees
Listen to what he says he reads and how often. And then listen to what Elon Musk reads and how often. And then compare with the rest of humanity.
great stuff from both sides... I love the open way of Lex and Jim is in a space in its own... great contribution
tremendous Questions Lex. I enjoy your channel and will continue to watch and learn.
Amazing video! Thank you for making this and the others you have done!
Great sign of intelligence on both sides to disagree and then agree again within a few minutes. No need to even finish most of the thoughts, just dance through it in seconds.
"Galavanting through the nether realms of possibility" lol
This is a great topic to cover. Please post more like this in the future. Excellent!!!
06:50 building a brick building with more Moore in mind
16:38 a difference in quantity is a difference in kind
17:54 the prob with early optimization
I listened to the video, I was quiet impressed. Not just a chip architect. Loved the discussion of levels of abstraction among other things. But I disagree with 2 points. (I'm an algorithms guy, my ph.d. is in operations research). a) as Lex said, there's a lot of complexity in not just human behavior, but time-varying road conditions. We're talking the Curse of Dimensionality here and it's not clear to me that a brute-force camera-data-only training approach converges on an answer. Rain or hail or sleet can change the driving conditions (traction) quantitatively within 10 seconds. Shade or a bank or a tunnel inlet/outlet can have very different traction. Static and dynamic friction. If we're talking L5 or even L4 capabilities, to my mind that doesn't mean idealized Arizona-driving conditions. It means L4 capability everywhere on Earth. And every video I have seen if full of people making wild predictions while ignoring just how dynamic and chaotic driving can be. Like 60 mph blizzard conditions with chains, one of which was accidentally cut in the wrong place. Was that in the data set? Maybe it will be eventually. But we're very far away. (My adviser says we'll never get there, which is an interesting statement. I disagree. Then again he's one of the smartest people I've ever met). n! is nasty if n is the number of possible objects in the world. b) I actually found while optimizing Matlab code that there are many situations in which a Search problem can be formulated as a maximization problem. And many times it's actually faster within such a scripting language. But my point is that there's definitely a relationship. E.g. "Search for an inflection point" equates to "argmin_x d/dt F(x)". It's a min versus argmin or max versus argmax type of distinction. Highly related. And they must needs have the same Big-O. c) There's a massive conservation of power constraint which is presently ignored in how people think about "AI". Intelligence means not being able to solve a particular problem, but potentially any novel problem. When people are assuming that if we can play checkers, chess, backgammon, sorry, connect 4 and a million other games better than human, then they're assuming that every game can be played at better than human capability. Maybe so, but there are a finite number of games. There are not a finite number of ways in which intelligence is applied. Let's talk cardinals of infinity shall we? We have bandwidth limitations on training and finite training data. No finite number of dots gets you every dot between 0 and 1 on the real line. We have object recognition but we have no scene understanding. No posited models / relationships / dynamics. Perhaps by the time a neural network masters language it will be able to find relationships between many dynamic entities in a dynamic world in real-time. Btw, I asked my adviser in grad school what he thought would be the most remarkable / impressive thing to come in the next few decades ahead. He said materials / manufacturing technologies, which surprised me. So this basically agrees with what Jim Keller said. We're clearly entering into a new frontier when we start being able to construct things (like SpaceX rocket nozzles) on an atomic scale via 3D printing. And what Jim said about the potential for reducing feature dimensions is amazing. Last of all, I wanted to make the point that most every chip is predominantly a 2D device with a couple dozen layers from what I understand. At some point we're going to break that mold and start working with truly 3D designs. And the same goes for neural networks. E.g. treating LIDAR data as a 2D image is pretty silly. We're making enormous performance sacrifices for treating a sparse thing data set as approximately dense. Many other interesting points on data quantization, significant figures, determinism, and the number 42 ;). I think people are badly failing, however, to anticipate what disruption novel technologies are going to have on our way of life, however. Elon Musk gets it. But if energy becomes free (or an ax+b situation where a is tiny and b is amortized to 0...), that fundamentally changes the entire civilization. The conversation about the evolving relationship between humans and technology has many, many implications beyond self-driving cars. It relates to how our bodies become more robotic. How we start working alongside robots. How we have advisory services / counselors / mentors / teachers and eventually bosses that are 'AI'. (I hate this term, it's an oxymoron). About telepresence via robot. About agency. About cybercrime and warfare. About ethics. About responsibility. About passive versus active roles in life. About turning the world into a zoo on the first day we decide we need to setup a MIMO control system to govern a climate gone insane.
L4 almost undoubtedly will be achieved somewhere in this universe. The same applies here on earth for humans and automobiles, assuming we keep progressing (even at lower than current trajectory) for the next several millenniums. I'm very confident in this assessment.
It may or may not be achievable with solely cameras, hardware and the right algorithms, this, I suppose we'll discover in time. What I am almost certain WOULD work is to successfully synthesize a smarter, more capable "brain" that we could then connect to moving cameras and sensors that it would have full control over, similar to how we have control of where we aim our own eyes when driving. The obvious problem here, is how to achieve such a feat.
The thing about it is, we already know for a fact that the human brain and its equivalent(s) is doable, because, well, here we are with said brains. The evolution of a certain type of matter made this brain and there is zero evidence to suggest that we can't synthesize whatever nature has developed through time/ evolution, and plenty of evidence that we CAN, with sufficient knowledge/ capability.
So, yeah, we'll probably synthesize a human brain and, through developing a clearer understanding of how it functions, maximize its potential to a point where it's simply better at doing any task that we could set our own brains on doing, including driving a car. Better yet, we can just scrap the whole human brain idea and jump straight into synthesizing a far superior type of brain from the onset.
A Cascade of diminishing return curves: moorception...
This was truly remarkable. Thanks for cutting it down to 26mins now I think I might go watch the full one
Best interview ever!
Really enjoyed this. Thanks.
Really great. Gonna check out the audio podcast.
this guy is awesome....... wish he had more content out
This is so good.
Spicy discussion. Liked it very much!
Super informative guest 🤯
Spectacular podcast. Period.
It's encouraging to hear that Moore's Law is still on track from an actual chip architect. Excellent interview.
given search space versus found search space ... such an elegant way to describe the distinction
Great insight
"There are hundreds of billions galaxies...seems repetitive at best" loll
This makes me so excited to be studying electrical engineering
This was a damn good watch.
Keller: "...a change in quantity is a change in kind".
Mathematician: "A profound change in system dynamics from a change in scale is called _emergent behavior_"
Keller: a circle
Mathematician: "The locus of all points equidistant from a central point"
@David M Do you have any idea what emergent behavior even is? There are behaviours that are only aparent on certaing scales. Phase transitions, Consciousness, the list goes on. It is in that sense that people mean when they say "a change in quantity can be a change in kind".
@@iurigrang yes. And that all falls exactly into the definition of emergent behaviour.
I did like learning from Keller’s perspective ... the heuristics of progress
What an interview
Read Bernard Lonergan's Insight: 'Thoroughly understand what it is to understand, and not only will you understand the broad lines of all there is to be understood but also you will possess a fixed base, and invariant pattern, opening upon all further developments of understanding.'
I love how he explains himself . What a mind
These two are not always speaking the same language. Lex is imperturbable.
You put it in words I couldn't. I feel from watching just a few of Lex's interviews he is there to talk and not to listen.
I couldn't agree more, I cringe whenever I see things get confrontational, but Lex handled it like a pro.
@@wentaoqiu4072 Yes, I cringe too. Does that make us cowards, you and me?
@@RalphDratman No, this makes us good people, we believe there are ways to communicate, to deal with things without getting confrontational. Though some might argue it is necessary evil.
@@wentaoqiu4072 Disagreeing on a subject is not being confrontational. There was no confronting in this interview.
"Angry bird apps might be the whole point". Hopefully not.
Love it. The way Jim talks makes Michio sound like he works with crayons.
As important as communication is, jargon literacy is required after a point.
I have sort of an understanding of the convo but I just can’t lol ..I’m a huge fan lex u r role model
I find this fascinating. In 1980, when I was trying to get an overview of how chips were designed and produced, Moore's Law was predicted to end around 1995. A quarter century after that deadline, and with feature sizes now some 100 times smaller and transistors around the size of a smallish virus particle, your guest says Moore's law, perhaps in a somewhat different mode, is continuing apace.
And it strikes me that the public need to understand the role played by the basic science in their way of living. As Keller said with respect to semiconductor manufacturing, "There's equipment, there's optics, there's chemistry, there's physics, there's material science, there's metallurgy... literally thousands of technologies involved."
Meanwhile some segments of the population claim that science and expertise are unnecessary. It might help them to know that without, for example, the highly mathematical study of relativity and quantum mechanics, as pursued over the past 120 years, the airplane they travel on would never leave the ground. The cell phone they organize their life with could not possibly exist. The internet they get their news from could never have been developed. The MRI machine that helped save their life last year would never have been dreamed of.
I see concrete technological achievements as playing a unique role in proving beyond a reasonable doubt that our lives depend on science and mathematics, including (among many others) medical, food and energy science. How skeptical can one remain about science while cruising in a passenger jet 10 km above the surface of the earth?
Because they're idiots. Or intellectually dishonest. A great Carlin quote that still applies today is, paraphrasing, think about how dumb the average person is, and realize that half are dumber than that.
everything you just said is horseshit and absurd and clearly not sane news fake mri's bs and the tech useless
@@movingurbanly4346 Bad bait
@David N interestingly enough correct but most people don't know that...
It's interesting to note that general relativity (not special relativity) is technologically almost useless. The only example I know of were it is used to a significant effect is to correct GPS location for the mass of the Earth, which slightly bends spacetime.
I loved when he asked Lex if he knows how CNN layers work, how they can detect ears, eyes of a cat. Sometimes you have to know your interviewer as well
Daler Rahimov My thought as well.
Lex, I'm sure you get a lot of advice from people so here's mine: you should release your short clips first or along with your long form interview. You can draw people in with the short and get them to listen to the long if interested. I always find that I see your long interviews and not sure I have the time for it but when I do they are always great. Maybe I'm wrong but I know I'd listen more this way.
Agreed
True.
His thoughts on value of abstraction in chip design reminded me of ideas from Yuval Noah Harari's book Sapiens. The ability to abstract may well be one of the most important steps in the evolution from apes brains to the human brain.
@lex Friedman would be great to interview Jim Keller re RISC vs CISC and if RISC will prevail. Also interested which one will be more applicable to the field of AI?
I wish I grew an appreciation for the beauty and art in science earlier in life.
Loved listening to Mr. Keller challenge his host.
25:22 So, a neural network is taking a complex data set and extracting a pattern from it. Now this pattern doesn't explain the whole of the data set or maybe even purpose of the data set. The neural network in our brain doing the same thing to the Universe. So, can our brain explain the purpose of the universe rather than just extracting patterns from it and calling them laws of physics.
parishrut badoni cue Donald Hoffman
We know the answer already. 42.
Damn... never thought of it like that. Wow
the purpose of the universe is the same as any object born into it. to be created, to exist, and then to die. no greater point needed. as for humanities purpose, again to be created/ to exist/ and then to die. no greater point is needed. the point of life is to exist, and anything else is our own subjective point of view. as the absurdist camus pointed out, the stance is to not live a good life because life has any inherent meaning, but that it's possible to live a good life in defiance of its ultimate futility.
16:40 "A difference in quantity is a difference in kind...".
"Quantity has a quality all its own"...
22:05 Yes but they literally weren't in a closet all day. Cellphones are very easy to keep at hand all day.
Moore’s Law is always alive, dead, and going to die all at once.
It’s the Schrödinger’s cat of laws.
Jim did not mention about thermal issues associated with further miniaturization. It may be a critical issue as heat gets more concentrated in smaller footprints. Less electrons to do the switching and rising ambient temperature make it more challenging to make reliable devices. Will it be the end of silicon and time for GaN?
The amount of heat release by the transistors happens because of inefficiency in the switching itself. We use both negative an positive transistors to only use power when we send power to it, instead of a constant draw of power to it. The small space between this is the source of heat as the doping is. TSMC wants to solve this with stacking the types of transistors, as it would decrease the resistance and length of both transistors, therefore making chips efficient. Look it up, TSMC announced this at a Semiconductor expo in Amsterdam
Thanks to TSMC. If we were to rely on Intel we would have 14+++++ on 2025.
@Red Dead well this aged well
TSMC are not at Intels level and people that think they are just have no clue.
TSMC couldn't do it without EUV tho, as 10nm intel would be the same as DUV 7NM. EUV got to the market around 2018-2019, when it got financially viable to use them in mass production.
25:00 hitchhikers guide to the galaxy anyone?
Please: a link to the Rich Sutton paper?
That last point was incredible. We're developing machines right now that are making calculations that we don't even fully understand.
It's how AI works. CGP Grey has a great video on the topic: czcams.com/video/R9OHn5ZF4Uo/video.html
Nice Video!
The actual law is that the time to double is around two years. ie we look at the time to double, not the multiplication factor after two years, even tho there is an equivalence there.
Sure the math may look better to you when t is the variable, but for someone designing the system t is fixed. It's the change in performance when the product ships that's variable.
Moore's Law was that "Transistor density doubles every 18 months".
You can't just redefine it to say Moore's Law isn't dead, under the original definition it has been dead for well over a decade, when Intel stopped going from 386->486->Pentium->P2->P3->P4 and instead switched to core i5 etc was the point when Moore's Law died.
Intel had to change it's marketing and branding to obfuscate the fact they could keep up with Moore's Law and keep a differentiated market place.
I'm going to go with Jim Keller on this one.
Moore's law is not strictly about density but was about the number of transistors that can integrated onto a circuit. It does not stipulate the size of the circuits, topology or geometry of integrated circuits.
Intel chips went from producing 1 million transistors to producing about 48 billion in about 30 years which is a doubling every 2 years.
@@bighands69 Bullshit.
When Moore spoke he was strictly speaking about transistor density.
Yes I'm old, yes I was there at the time.
That bullshit free definition is exactly why it became the Iron law of digitisation for a time.
Moore's law has been dead for a long time, stop making excuses and limiting yourself, it was a useful guide for a while but today it is a crippling constraint on creativity and progress.
Stop worshiping a dead paradigm.
I keep looking at the screw that is holding the whiteboard to the wall as if it was random debris on my monitor. I tried wiping it off nearly a dozen times, yet, it's still there... Guess, I got screwed...
Nice to know we can keep going. As to the universe being "wastefully large", well I think that's a value judgement. Why not think of it as a lovely large blank canvas?
You're misrepresenting his point.
He's the Chuck Norris of CPU design
If I understood half of what I think I did, that was fascinating!
So basically there is the spatial variable along with a time variable for transistor production. It's almost like we live in a time space continuum or something.
Very interesting
So with modifications and abstract conceptual ideas to extend Moore's Law, Moores Law becomes a self-perpetuating sort of Mandelbrot Set only limited by time and technological capability to manifest it ?
That is nonsense.
Lex schooled on this video 🤣
I really enjoyed this talk. Is there a talk on quantum computing.
No matter how much I try to understand quantum and I think "i got it 1 or 0 or anything in between". Then I realize i don't get it.
brush up on quantum mechanics before you try to jump into quantum computing. MIT open courseware has some great lectures on introductory QM ocw.mit.edu/courses/physics/8-04-quantum-physics-i-spring-2016/
@@naught_ Should probably brush up on all other subjects before tackling the quantum realm!
I have no idea but if I had to guess I'd say quantum computing is '1' and '0'
QP is not something humans can comprehend,...to be both alive & dead, up & down
@@interstellarbeatteller9306 Quantum mechanics is isolated enough from other subjects in physics that you should be able to start from scratch. Classical mechanics and electricity & magnetism are usually taught before quantum but this is mostly a formality. Quantum is self-contained, as long as you're just studying the basics there's no need to develop a holistic picture of the different fields of physics beforehand
Search for a youtube / Ted talk on quantum computing in 5 levels of detail. The people here are being absolutist
MIT open course ware is not the place for a newcomer to start. CZcams and blogs and Ted talks are such places. Also, don't get your hopes up, things are extremely primitive. First for comparison though it might be of use to study the quantum aspect of a transistor and how they switch. Cause that's a simple binary interaction. When you get into quantum mechanics you get into probabilities, waves and information theory. I expect it'll get complex very fast.
To tighten the vice of the microphone to your desk isn't a good idea at least not at this where you are sitting. Thank you for sharing
10:11 Don't look at me mate, I ain't got a clue
17:00 Lenin's favorite quote.
I think that was Stalin
@@Michael-it6gb Dialectical materialism. Karl Marx and Friedrich Engels + Lenin. Stalin was much later.
@@Michael-it6gb Perhaps Stalin also liked this quote.
This guy is a rockstar!
fascinating discuss. Moore's Law predicted the growth in the number of devices on a computer chip, a successive doubling per unit time. Gordon Moore said once that predicting exponential growth of anything into the future is risky, because after some number of generations you exceed the global capacity to support any construct at that scale. In today's cloud world, the growth of computing power has accelerated beyond what Moore predicted, mostly because the number of servers being installed in cloud data centers is growing so fast.
Moore never predicted the power of computing. He was talking about the number of transistors on an integrated circuit.
Jim keller has eyes of steel! My man.
But to the more important question what is it all for?, well good faith. On the behalf of evolutionary creation an expansion in human life.
Control, but not tyranny!
Theorist meet practitioner.
Lex does ML that is pretty hands on, wouls say software guy meets hardware guy would be a more fitting description.
@@0dyss3us51I can see that take on things but think it misses the main point.. This for me this boils down to the LIDAR vs. passive photon debate... and the humans can't be modeled vs. x/y movements and accelerations are dampened so it is just a ballistics problem debate. Lots of folks in in ivory towers proclaiming they are correct meanwhile 1.) Tesla's have driven 3,000,000,000 miles under autopilot. 2.) Expect they are statistically provably safer than human drivers (Attention vs. accuracy debate) 3.) FSD is nearly feature complete.... Then it is just moving from 2-3X safer than humans to 10x safer. Etc. Reminds me of the PhD's and Astronauts that were paraded in front of Congress in an attempt to salvage the NASA budget proclaiming 1.) There was no place for private companies in first line positions controlling space flight. 2.) The first stage could never be brought back. 3.) It would have no impact on the $/kg to leo pricing if it did. Mind you many where making these claims AFTER spaceX had already brought back BOTH boosters from the Falcon Heavy. Love Lex and his podcasts. See deep value in his openness of mind and the litany of wonderful guests he interviews. But in this instance he was decidedly shown to be man behind the curtain we are told to ignore. And that is OK. Does not in any way diminish his value in his wheelhouse. Will make his wheelhouse much bigger if he himself will allow it. How as individuals can we ask for more?
@@cclose14111 Hi Chris I am not sure how my comment is missing the point or how all of that you wrote is really relevant, to my comment, but that is fine :)
@@0dyss3us51 Guess my point was it seemed that, ""software guy meets hardware guy" description under sells Jim by an order of magnitude. On about 4-6 different dimensions.
COSMOLOGIST meet ENGINEER
This was a tough interview
But Lex pulled it off and I think he's about the only podcaster that can. I don't imagine Joe Rogan having a conversation on the same level lol.
@@ArmoredNeko Most podcasters are dumb.
@@haliax8149 Most people are dumb.
I know a lot about this, but you know what, I'm a fucking moron when it comes to explaining or asking questions. Lex is brilliant!
Hello sir, i saw you with Andrew NG. When that podcast is coming
Transistors are already not doubling every two years, the "innovations" part Jim is saying is the important thing. 3D stacking, chiplet design, L2 cache distribution, 4-way SMT, these are what pushes the industry forward now. I am excited to see Jim's Royal Core design. That looks to be Intel's last stand before prioritizing foundry
What do you mean they are not doubling every two years. That is a very vague statement.
When Jim speaks I listen intently and nod in agreement but have no idea what any of this means.
Super interesting
Nice talk. I miss smart people since postgraduate.
If you let AI decide how to design a city properly, I think it would look very different from what we know
I love it. My field :-)
Moore's Law is the opposite of Fusion is will be gone in 30 years and always will be.
Ron Ray There have actually been some pretty big advances in fusion in recent years. ITER is expected to produce net positive energy balance when it goes online and is currently under construction.
AI might also help with the fusion problem, as plasma instability is a complex beast that could probably benefit from machine learning.
Instead of focusing on how to make current transistors smaller, we should focus on how to place individual atoms faster to produce the smallest possible transistors at the fastest speed possible.
Brain, meet firehose. Keller said the transistor was a hundred atoms, but Si is 2Å and feature size is currently 50Å? That's only 25 atoms? And what about doping? How much doping can there be with 25-atoms? And how consistent? Keller said he needed ten atoms? At least one atom must be doped? But not two? One transistor can't be 10% doped, while another is 20%? Pretty amazing discussion.
I have never heard of Jim Keller before he clearly knows his shit though. This is really interesting
He's only the best microarchitecture engineer to have ever walked the planet and has merely had his fingers in every major leap in CPU design for the last 40. No big deal.
@@Fyrwulf that's very nice that you know who he is my man. I do too now, I wrote that 7 months ago. Thanks though for all that publicly available information. If you were 8 months quicker you might have taught me something and earned your internet big boy points xx
Optimization is absolutely a search.
Yes coming up with the weights in the CNN is absolutely a search. But after the learning is done the operation of the CNN is not really search.
It's not a law it's just a really good prediction and objectively it's dead by definition. You can change the definition and say it's not dead
Lex getting rekt at 13:30 =)
I've always thought moores law has held true if you just don't simply watch at the capability of a single dye to hold transistors, but a calculation of number of capable transistors in the whole domain. ie; for you can network and do parallel computing, the amount of flops you can produce with the same work doubles each two years. You can make about 1.666 times more capable computer/transistor, and some 1.333 times more of these computers, thus the computational power doubles each two years.
Moore's Law is about the number of transistors present on an integrated circuit. It is not about the size of the transistors nor does it stipulate the type of architecture being used. If you take the number of transistors on the first generation integrated circuits and calculate that against the top chips today in 2023 you will see that it has doubled every 2 years producing about 43.6% growth every year.
The reason why computing power is doubling every years is due to the fact that computing power is more than just Moore's law.
@@bighands69 its a bit under double or how moore said it himself. But it’d be double if you just look at raw available computing power.
And like petra, was carved silicone with multiple meter architechture. If you reverse moores law 2000+ years, it goes to fractions of if 1 would be a 8088