Disagreement With Jim Keller About Moore's Law (David Patterson) | AI Podcast Clips with Lex Fridman
Vložit
- čas přidán 27. 06. 2020
- Full episode with David Patterson (Jun 2020): • David Patterson: Compu...
Clips channel (Lex Clips): / lexclips
Main channel (Lex Fridman): / lexfridman
(more links below)
Podcast full episodes playlist:
• Lex Fridman Podcast
Podcasts clips playlist:
• Lex Fridman Podcast Clips
Podcast website:
lexfridman.com/ai
Podcast on Apple Podcasts (iTunes):
apple.co/2lwqZIr
Podcast on Spotify:
spoti.fi/2nEwCF8
Podcast RSS:
lexfridman.com/category/ai/feed/
David Patterson is a Turing award winner and professor of computer science at Berkeley. He is known for pioneering contributions to RISC processor architecture used by 99% of new chips today and for co-creating RAID storage. The impact that these two lines of research and development have had on our world is immeasurable. He is also one of the great educators of computer science in the world. His book with John Hennessy "Computer Architecture: A Quantitative Approach" is how I first learned about and was humbled by the inner workings of machines at the lowest level.
Subscribe to this CZcams channel or connect on:
- Twitter: / lexfridman
- LinkedIn: / lexfridman
- Facebook: / lexfridman
- Instagram: / lexfridman
- Medium: / lexfridman
- Support on Patreon: / lexfridman - Věda a technologie
hi i love your podcast, i just have a request , because i am almost deaf i ll be very happy if there were some subtitles. thnk you very much
for some reason auto generated subtitles are in vietnamese. Lex should fix it atleast
If you need help with this lex let me know!
Not sure how it was 3 days ago, but if you go to the full interview now there are auto generated subtitles in English. While not perfect, they're pretty decent. Go to 1:19:41 in the full interview for the content of this clip.
Soon Ai will generate better subtitles for all videos in every language
David looks like end stage Walter White in the thumbnail.
Ill assume thats a compliment.
lol omg
i was in lab with him when I was at berkeley. this gun bench HARD at he gym, he's benching 325 at 50 yr old lol
Well, David is also quoting Gordon Moore's quote in this clip that "not every exponential increase lasts forever". But even if the density of transistors doesn't improve by 2x every 18 months or 2 years, but 20% or even 10% every 18 months / 2 years, that's still exponential growth. So his only argument clearly isn't that "Moore's Law isn't holding on anymore", it's also "the exponential increase in the density of integrated circuits is no more", which is clearly wrong. Furthermore, the current slowing down of density or in general performance improvements in integrated circuits will probably turn out to be another S curve as we find a better technology that will speed up the improvement once again.
Also should be remembered that while 10-20% improvements may seem small, we're still improving performance by thousands fold compared to chips from 20 years ago in each of those 10% jumps, and millions fold compared to chips from 30 years ago.
Let’s see. 2x to the 10th: 1024x.
1.5x (50%) to the 10th. 57x.
1.2x (20%) to the 10th: 6.2x.
1.1X (10%) to the 10th: 2.6x.
1.05 (5%) to the 10th: 1.6x.
10%, or 5%, repeatedly, forever, is indeed exponential growth. But there is a world of difference between:
10 cycles of 2x every 18 months, for 1024x over 15 years
And: 10 cycles of 1.05x (5%) every 24 months for 1.6x over 20 years.
If the growth slows to 5% every 24 months, that would mean just 1.6x over 20 years. Hardly what we’d recognize as big "exponential" gains (and that’s assuming the rate doesn’t fade lower to say 2-3% halfway there).
What he said of 2x every 2 years is not holding anymore is correct. Choose the growth rate: 50, 20, 10, 5%. You get 57x, 6x, 2.6x, 1.6x respectively. Big differences.
I do understand what Jim wants, delivered the performance gain obtainable as if the Moor's law would hold. That can be attainable with architecture Innovation , a little gain in the process and focusing on some applications as a benchmark. If you take Moor's law literally it is done and gone. Anyhow , transistor density is just one parameter of the equation and in many cases not even the most important one.
yes, you put it exactly how I would put it
kind of disagree when we hit 3nm we will have 24b transistors on our cpus
Nope. You don’t understand Moore’s law if you think architectural innovation can produce what Moore’s law was producing. Moore’s law is that the number of transistors increases 2X every 2 years, in the same area. You would need REPEATED architectural innovations (not gonna happen) every two years, using the same number of transistors.
Under Moore’s law when transistor density improves by 2X, there was also has the benefit that power consumption per transistor goes down by like 2X. So even with twice as many transistors every two years, or whatever, a chip’s power consumption stays relatively constant, instead of increasing by 2X, compounding, every time the number of transistors increases by 2X.
You can come up with some architectural innovations, with a certain transistor budget, but you cannot come up with repeated, compounding, doubling every 2 years, improvements without more transistors.
Transistor density is such a critical parameter. If transistor density today was that of 10-20 years ago, chips would many times as big and consume many times as much power and cost many times as much IF the chip had today’s transistor count at the density from 10-20 years ago.
Can you name one parameter that is more important, as you said?
@TTB you have to listen to him again, he thinks that it can be prolonged by about a decade. Until now he is right. ASSP are doing more than 2x actually.
@@blanamaxima Listen to Keller? What video?
I’m going off of Moore’s law means 2x transistor density every two years. And Moore’s law being dead means improving slower than this, not ceasing, as many take it.
I’m not understanding "transistor density is just one parameter … not even the most important one". Is transistor density the most important, or not?
There's continued shrink outside of intel. I don't think Jim singled out intel in his statements.
You are missing what he said. He said that Moore’s law, shrinking transistors and improving their density by 2X every 2 years, is no longer holding. He distinguished that that does not mean shrinkage improvements are not still happening …. just not at that pace. Even at those who are doing better than Intel.
He is Great!! Brilliant too. I figured him for a coach. Shocked when he opened his mouth. Never can tell. I love the way he calls out the truth. “ And I’m FACTUALLY CORRECT . ”
Is he right though? If you use the M2 Ultra it does seem that Moore's law is still roughly going. However that computer costs thousand of dollars. I think Moore's law is irrelevant ultimately. I would say total computer performance per dollar is a better metric but even that can be misleading because of accelerators and memory.
For example, if they could make computers run at 100 GHz that would not affect Moore's law. They could radically redesign chips resulting in more processing power. On the other hand you could begin stacking transistors to keep Moore's law going (or even exceed it) but have to lower the clocks, which means every transistor is doing less work.
Intel definitely has been stagnant for at least 6 years
Graphene at a certain angle unti itself is a room temperature superconductor?
So what.. do we daisy chain them with conductors and get a great deal of distance for impedance free?
Imagination of Engineers is the problem with saying "Moore's Law is Dead"
I found that piece extremely odd. Like engineers are stupid and don't understand this conversation/semantics?
The problem isn't that the server chips aren't faster, but that the server farms are favoring stability over speed.
Hardware and software are interchangeable. What if now that Moore’s law is slowing down we start getting rid of software bloat.
Try telling that to the JavaScript/Web developers. You will get lynched. Everything should be web and JavaScript in their view. I mean this whole mindset is at the highest levels of a lot of companies. Web technologies everywhere. An extremely bloated technology stack relying on a crazy number of layers and massive consumption of memory.
Sure, but can you get the compounding improvements of Moore’s law out of improved software, at a pace of 2X every 2 years?
Meaning over 20 years, software gets 2X more efficient 10 different times, for a compounded total of 1024X?
That includes using 1024X less memory and power, which is what Moore’s law implied.
More efficient software undoubtable has huge potential, but writing a program in lower level languages takes enormously longer. People only are only so smart and only have so much time to program. Unless you are going to redesign programming languages I don't think it is reasonable to just expect people who write normal programs to make it 10x more efficient.
Yes corporations operate in good faith...
Moore’s Observation
Well, Intel slowing down and Moore’s law are two different things, so sorry but you are factfully wrong.
Are there any foundries that are defying the decline of Moore's Law?
I didn’t realise that people ever referred to Moore’s law so literally. Most time I’ve seen it referred to is the much broader use.
Now you know.
Software has to get MUCH better. Software is so bloated because designers are just patching over twenty year old work.
I have to agree and m$ is still the gloat master of the world.
Yup get rid of electron apps
At Best Buy, can you find a computer based on RISC-V?
Amd literally going 16 32 64 cores that's pretty moore
But you won't need more than 64 cores for the next ten years lol
@@senormoo4749 Data Centers yes. Consumers no.
We're still going extremely fast. In many ways, instead of growing up in terms of how fast we can process information, we're now growing out in terms of how many cores, how much storage, and how much more efficiency we can squeeze.
I think we could see a restart of Moore's Law in terms of raw processing speed (GHz) if we start integrating different materials, especially if we discover wonder materials such as room-temperature superconductors. In fact, we probably will.
The question is whether that happens before we figure out something like Quantum Computing. I suppose one could help engineer and create the other.
What we're going through now may, in fact, carry a lot more potential than Moore's Law in the long run. What we are doing now is a result of the 50 years of where Moore's Law held true. But it is a fact that today, Moore's Law as a prediction no longer holds true.
Progress is still accelerating. But the way we use silicon is reaching a limit. And that limit is our understanding of the material, and how to use it.
We may be able to build smaller than atoms if we're able to develop a new understanding of the universe and develop new science. It wouldn't be the first time we've done something like that.
In fact, humanity and life, in general, is something like if you gave evolution a brain, what would it do?
Thats done by increasing chip size not making transistors smaller
@@senormoo4749 Lol, why not
David Patterson is generally wrong. He is only technically correct, if you look at every 2 year period, because not every 2 years we saw doubling of the transistor count.
Overall, the industry is way ahead of Moore's Law. Intel 4004 used to have 2250 transistors back in 1971. If we apply the Moore's Law, we should have CPUs with 37 748 736 000
transistors by 2020. (37.7B transistors). Do we have such? Well NVidia A100 GPU has 54 Billion transistors, but this is nothing compared to Cerebras AI Wafer-Scale CPU, which packs 2.6 Trillion transistors. And yet, transistors are not everything, but that's another story...
Epyc Rome cpus have 39.54B transistors.
The intel 4004 cost a few hundred in todays money. A100s and Wafer scale chips cost much more. If you could build a chip the size of 100 wafers for 1B dollars, regardless of whether that is surpassing Moore's law or not, it is not proof computers are fundamentally improving.
Does David knows about AMD?
What are the numbers for AMD?
Who dares question Lord Keller. Infidel.
Intel held back development due to artificial monopolistic marketing reasons, not because they technically couldn't and now AMD leapfrogged them and are leading. My cheap 8 core 16 thread 3800x is proof of that, a few years ago it would cost thousands of dollars due to Intel's greed.
7:39 - Yeah, that's me.
haha right, Its the specifics of his law indeed. way too ambitious to expect anything to keep that consistency but Its a goal to strive for.
Moore’s Law is a misnomer. Correct name is Moore’s Expectation.
Jim Keller is a journeyman computer architect. Patterson has been slapping the monkey in academia his whole life. Patterson has been working on an ISA for forty freakin' years.
Moore's law was not directly about transistor count, the count (and density) is just correlated with economies of scale. I've read his actual papers,(an early one and a revisit after a few years) Moore was making an economic observation that the power per dollar of mainstream IC silicon chips would double about every two years.(not complete packages, it was specifically the bare chips wholesale from the foundary) Go read Moore's papers its really clear, with graphs of cost per transistor from low transistor count chips to high count year by year.
Still I agree that I see the end, just because the technology has matured and you can't squeeze blood from a turnip. The ol' Pareto curve rears its head once again.
it is about the transistor number and the cost of it ! thats is the definition!
You don’t really understand Moore’s law and what it means to make a transistor’s area 2X smaller so twice as many transistors fit on the same sized chip and twice as many transistors fit on a single wafer.
Twice as many transistors on a wafer at a cost per wafer that stays the same or goes up by less than 2X is what leads to the "economies of scale". The wafer size has held at 12" for years and the cost to produce each wafer full of chips has been held steady or increased at much less than the 2X increase in transistors per wafer.
Also, he did not get into this detail of transistor scaling here (but he understands): as transistors density increases by 2X, this also leads to each smaller transistor consuming half as much power. But there are twice as many, so the power for a chip with twice as many transistors stays about constant.
If this power scaling along with density scaling did not hold, over ten cycles of 2X density improvements, both the power consumption and transistor count would go up by 1024X.
Clearly chips do not consume 1000X the power of 20 years ago!
Look up "Dennard scaling" if you want to understand further (Wikipedia, etc.). It will mention Moore’s law.
He made many observations in those papers, sure, INCLUDING the exact definition used in this video. There's literally a graph with his prediction of "components count" vs year. Also, the time horizon of his prediction was up to 1975, so whatever is happening in 2023 has nothing to do with that. No disrespect for Moore.
@@olemew Count and count as related to the economic value are not the same thing. Details matter.
You can produce any count in any year if you spend enough and make a huge IC.
Doesn't seem to like Intel that much :)
but more efficient software could be like virtual moores law in terms of performance for a while yet.
No. Its about doubling of transistors.
People are (well, certainly I am) buying new phones because the old ones are forced to become obsolete by the lack of security updates to the old phones.
It would be nice if phones got security updates for 10 years like Chromebooks. Many only do about 3 years which is still well within the useful life of the phone hardware. I understand why they do it, not enough people are voting with their wallet for better longevity.
Moore's law did not hold from the beginning. Transistors do not double every two years if you keep the surface area the same. I plotted this once. It actually works out to about 28 months which is not even on the same chart vs 18 months or two years doubling. Obviously if you increase the surface area you can make it double every two years. en.wikipedia.org/wiki/Transistor_count
So many things can affect computer speed beyond transistors. Cost, design, clocks, size, and power draw all need to be taken into account. If Moore's law only looks at transistor count then it is not telling the whole story of computer hardware progress.
I guess most people buy new phone for more memory, better gpu performance and better battery and charging. If you cant afford new phone every year you better wait for the biggest revolution in battery tech ever that will come once tesla does their battery day. Nobody in any segment will want to use junk batteries that wear out, overheat and charge slow.
With phone technology where it is at, it should be because your phone is broken or out of date. Of course people just love having the latest even if it makes very little difference in their lives.
Hes a tad sugar coating how capitalism and physics work.
Don’t piss off Heisenberg.
We've hit the wall with conventional wafer technology.
Breaking bad...
I see why this guy is an "educator" and not a professional. Clock rate hasnt been a measure of performance in 5+ years. IPC gains every generation have been significant while clock rate has stayed almost the same in CPU's.
0:36 seconds, this fellows say Jim Keller words are 'just marketing'? Marketing? This guy here is has already admitted his RISC baby was whipped in the 80s by Intel founder Mr. Moore and company, then proceeds to pitch this mark V version of a CPU he only helped to design, then accuses Jim Keller of marketing?
hold my EUV
depressing 😭
When “Doc”/David was younger I recall that he used to really “like me”...a lot.
Kind Regards,
Bugs Bunny
what?
@@racecondition3176 I’m not sure.
@@racecondition3176 "whats up Doc?"
Ghislaine Maxwell didn't kill herself
Even Jim Keller was unable to fix intel's broken 10nm
You can't be really sure about this. Jim do's a lot of specification and conceptional design. Ryzen needed 3gen to fully shine, Apple needed 4, Tesla did't it in 1-2 trys but i think Musk supported Jim a lot. With Intel i think we have to wait for 12gen to see what Jim was up about.
Jim Keller is a designer not a fabrication expert. It is like asking a chemist to fix your printer drivers.
@@MrHaggyy Damn, he had his hands on everything
Heisenberg?
hah
As a end user, moore's law is kind of pointless to me. Packing more transistors on a wafer is less important to me than good design. Looking at apple M1 and compare it to the current Intel desktop CPU, good design is more important. Then there's writing good software that is efficient. Moore's law isn't to blame for software bloat, but a lot of programmers thought "if transistors are doubling every 1-2 years, I shouldn't worry about writing efficient code." I agree with jim keller's philosophy there's still a lot to do. Getting hung up on moore's law is counter productive.
Yes, but part of Moore’s law, and Dennard transistor scaling, is that as transistors get smaller, each transistor consumes less power. If that were not the case, a chip with X times as many transistors would consume X times as much power.
So, packing more transistors on a wafer HAS been more important than good design, you just haven’t noticed it, or have taken it for granted.
What you said about good M1 design versus Intel, or efficient software, still holds.
Look up Dennard scaling for more background.
Programmers only have so much time. If their program is not running slowly they might figure it is better to spend their time updating and adding features rather than improving efficiency.
Honestly the improvements are pretty awesome. 14nm gates to 7nm is a 100% difference. Its just they didn't think to milk it like they probably should have because the gravy train doesnt last forever. This is what takes some of the fun out of listening to a Divine Being like Jim. Hes beholden to politics. I personally think the property of greatness is at an inverse value. Gods shouldn't be Benjamin's b.
Note that if a process improves dimensions in the X and Y dimensions from 14nm to 7nm, there is a 4X improvement in density, not just 2X, or 100%.
I don’t know why you say he is beholden to politics. He is just taking what Gordon Moore said and distinguishing between whether that holds OR there is still improvement, just not at 2X every 2 years. I respect his statement, even as I have often done what he is criticizing as misinterpretations.
> 14nm gates to 7nm is a 100% difference
But these numbers are made up and don't actually reflect something real anymore. It is just marketing now.
The future is parallel processing on ARM
LOL: meh.....throw 20 cores at the problem!!
Parallel is slower than running on a single core and not everything can be parallelized.
@@autohmae True, but also many tasks are of the parallelizable variety... And overhead inefficiencies aren't terrible typically. Still, if we look at transistor density, Moore's law is coming to an end - given the constraints of physics. I think the only road left once we hit the silicon ceiling is power efficiency and quantum computing, if large scale quantum computing ever becomes a reality. Both situations still have limits unfortunately, but perhaps it's time for paradigm shift.
Breaking News: David Patterson says "Short INtel stock (INTC)"....Breaking news.
Such a cool and clever thing to say! He also said he sold his INTC stock at a huge, 'Moors Law type' profit!
@@250txc Financial Markets reflect the current state of store of information pertinent to future earning capability. It's will be interesting to see how knowledge of slowing/end of moore's law bleeds into the meat market which is stocks! How long till they catch on? Probably NOT till a competitor DIS-RUPTS price performance (and that's NOT amd)! LOL. (Patterson himself was a MASSIVE disrupter (risc), he knows of which he speaks...) and who is most likely disrupter? CHINA! (Soon to be "banned"(voldemorted) as the world INVERTS) so....
I believe that people like-minded to Jim Keller would oppose David Patterson’s statement “because you can measure that the number of transistor did not double in recent years you can reason that Moore’s law ended”.
But the disagreement is not in what the definition of Moore’s law is - most people agree that you should not broaden it artificially. However, the argument is that the number of transistors did not double because the industry did not want it to double - even tho they very likely could have.
Simply doubling the transistor count is not the key priority for semiconductor companies. Many factors are required to give their customers the best possible performance-to-price product, including architecture and compilers.
Want a much faster server? Buy AMD.
tastes great----less filling.
This is incorrect, density *is* going up at historic rates. See this chart:
docs.google.com/spreadsheets/d/1NNOqbJfcISFyMd0EsSrhppW7PT6GCfnrVGhxhLA5PVw
It's commonly thought that it failed because Intel had a hiccup with 14nm, but Intel isn't the only semiconductor company and their hiccup wasn't permanent.
On the list are SoC's and server procesors which cant be compared to normal desktop CPUs. If you watch carefully you will see that 2010 intel procesors had around billion transistors and new 10core i7 has only 3.3 billion. For 6 years (not even for 9) thats not 8x and that was his argument. For 12 years it was like 4.5X. Not to mention that performance is way less than transistor number improvement. Its less than 2x in 8 years. czcams.com/video/CAfVeggsmpo/video.html
@@vlada881 As I said, Intel had a hiccup, their 10nm was meant to come out in 2016, yet only really came out in limited production in 2019. A modern desktop i7 is still 14nm.
The primary reason SoCs have a different trend is that they primarily use TSMC, which didn't have Intel's issues. Transistor density does vary a little between processor kinds, but a SoC is still largely comparable to a desktop chip; a transistor is a transistor.
OTOH, performance and transistor density are not comparable metrics; Itanium used a lot of transistors for the time, but performed terribly. Intel's performance stagnated because their new architectures were designed for new nodes that got delayed, so they re-released Skylake. It's not surprising that performance didn't improve much for them. But AMD's performance over the same time period skyrocketed.
@@veedrac Intel wont go to another node until Q42021 (Q12022) with "Alder Lake" and thats minimum 6 years of "hiccup". No transistor is not a transistor. You are comparing apples with oranges. SoCs have logic cores, GPUs, memory, coprocessor etc. Only part of the transistors go to CPU. Server CPUs have bigger die and higher TDP so they are not comparable. They are only comparable with other server CPUs. AMD is still behind Intel for few % even with twice more transistors and 7nm litography (10nm Intel). For example ryzen 1600AF has 4.8 billion transistors but it has much less performance to 10 core i7 with 3.2 billion transistors. If you think that we magically can go back to early 2000s rise in performance, thats not gonna happen. It`s gonna slowly dissolve more and more to the point that it doesnt matter anymore.
This dudes body looks like it’s melting and he expects us to take his criticism of Keller seriously. I’ll take the word of the dude who respects his own physicality and calls for exceptionalism and optimism in his field of work. These intellectual pessimists need to look in the mirror.
Lmfao. It is physics. What he said is true. Also, wtf does this have to do with his look? Moore’s law is dead and everyone know that for a long time already. That is why the industry is moving to heterogenous design and different way of packaging. If Moore’s law is still true, we wouldn’t need complex engineering paradigm today.
Moore's Law is Dead
Sad. Judging by not listening to and understanding what was said. He is correct. Keller (like many) has twisted or adjusted the meaning of Moore’s law (and somewhat rightly so, because people like OP here cannot listen and reason about nuance: Moore’s law being ‘dead’ does NOT mean improvements are dead, it means they are no longer 2x every 2 years).
Moores law doesn’t have a fixed doubling time...... anyone who thinks its every two years is a fool