[ML News] Microsoft to spend 100 BILLION DOLLARS on supercomputer (& more industry news)
Vložit
- čas přidán 8. 06. 2024
- Some updates from industry in the Machine Learning world
Links:
Homepage: ykilcher.com
Merch: ykilcher.com/merch
CZcams: / yannickilcher
Twitter: / ykilcher
Discord: ykilcher.com/discord
LinkedIn: / ykilcher
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: www.subscribestar.com/yannick...
Patreon: / yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n - Věda a technologie
![[ML News] Llama 3 changes the game](http://i.ytimg.com/vi/kzB23CoZG30/mqdefault.jpg)
![[ML News] Llama 3 changes the game](/img/tr.png)
![[ML News] Chips, Robots, and Models](http://i.ytimg.com/vi/tRavLU8Ih4A/mqdefault.jpg)
I feel Agi will be invented in the year of the Linux desktop
In order to power AGI, we need fusion reactors.
And in order to design fusion reactors, we need AGI.
@@andybrice2711 and for control fusion reactor we will use Linux desktop
@@attashemk8985 I hope that runs Debian or something ROCK SOLID.
Nobody wants another `xz-utils` incident.
@@andybrice2711 Nah, just use all resources for power plants and let the population plummet.
More paper reviews please
Video is good. Just one question, where is Iliya?
They put a cap on him for sure. Since october last year something happned with their approach to ai that caused panic.
Free Ilya!
on the back of a milk carton
He is being kept hostage by Sam until he create AGI
Ilya is so dead
Fantastic. So ungewohnt. So many ML news 😊
I love ML News
hell yeah, monday comes early this week 😎.
😂 That last sentence is a killer !
Love your no nonsense updates.
Artificial yottabyte learning intelligence (AYLI)
Very good. Very good. 😀
Nice to see how it's an anagram for Ilya Sutskever's name.
OpenAI's definition of AGI is different from that of academia. In other words, OpenAI "AGI" is a marketing term and nothing else.
"open"AI also only marketing term
I don't think academia has consensus on what AGI is, or how to measure it
I dont think academia even has consensus on what "intelligence" is in the first place
Is it Monday AGAIN????
The "AI market crash" is going to look pretty funny.
Low end of the IQ curve: "Predicting the next word is all you need"
Middle end: "noooo, it's just a fancy auto complete, that's not how the brain works, AGI is impossible, a lot more research is needed"
High end: "Predicting the next word is all you need"
The catch is that the next word prediction needs to be correct based on information which is (1) not in the training data and (2) not in the prompt. GOOD LUCK, little AI!
@@clray123 pretty sure that an AI that searches for answers in the internet before making a complete answer is already there, is just the think that we silly humans do to make most of our tasks, that an some
@@travian821 Not really because it does not scale, and the main problem of the AI is that it has to generate the next token in (more or less) constant time. There is no AI which has a "pondering loop" inside. What is there is AI that is generating "function calls" or software which calls AI in a loop multiple times (possible feeding it data from outside) to generate an improved result. But if we need such external software, and the logic hard-coded in it, what does it tell you about the true capability of the "AI"? Language models are not Turing-complete, meaning that they cannot even execute common "easy" algorithms with a sensible amount of resources. So a better way to think about it is that we currently have "efficient cloning of text/image/audio, driven by a prompt", something like a fuzzy database, into which you send queries using natural language and receive fast responses. But the fast responses are only as good as what's already inside the db; and absolutely no "reasoning" (as in iterative planning) is involved.
@@clray123I think “correct based on information which [...]” and “information not in [...]” could both use a bit of clarification, though in somewhat different ways.
To elaborate:
For the second thing, it seems a little unclear what it means to say some information is in or isn’t in some dataset.
If there’s a random variable which is sampled from some distribution, independent from the sampling of the dataset from the process that produced it, then, it seems like the information of “what is the value of that random variable” is clearly not “in the dataset”. To clarify, I mean this in the sense that, if the distribution the variable is sampled from is deterministic, then the amount of information that the value of the variable constitutes, is zero.
This is a rather restrictive condition though, I think, and I don’t think it is exactly what you mean?
For the first thing: do you mean like, the criteria for the answer being correct depends on this other information?
Yannic spitting truth around 6:00
Did you checked Quibic already?
helllo from ethiopia
how many bazillion quadrillion flops do you have down there
I believe the bigger problem with Microsoft is not their dirty paws on AI (i.e. today's universal photocopier / data faker), but their rising dominance in the (enterprise) identity management. Imagine being a company which de facto has access to any data of any* other company on the globe because you can impersonate any employee in there (*except for companies that don't/are forbidden to use Microsoft's IdM, e.g. like in China). This is what Microsoft is increasingly capable of today and other companies, including IT companies that service non-IT companies providing critical infrastructure, are falling for it left and right and outsourcing their identity management. So instead of employee X proving their identity to your company server Y, it is Microsoft's server Z claiming that they have verified employee X's identity. This should be a huge issue in security, but nobody seems to care.
I guess a text-to-audiovideo model will win the (latent) space race
... wait, this is old news from last month!
It's only been 3 weeks but I swear it feels like last year...
Long Cray stocks!
Sora good for music clips
It has been named Deep Thought.
I N T E L L I G E N C E
I pray that the AI that manages to make society collapse will be named "Deep Thought"
Stargate will be the largest (in number of parameters) based on combinations of ANI, that will be marked as AGI. MS betting the bank on that they will win the AI race. And as always, when it comes to MS as a company, non of the IPs will be created with the walls of MS. So they are trying to win the race with others (90% sweet and 10% perspiration) just by throwing money on it.
As with Windows, they are going to fail and fall flat on their ass, but it's kinda unsettling given that they and their pals are now making a sizeable chunk of S&P 500's. Meaning that unsuspecting grandmas and the like with their retirement savings accounts will soon have to bleed for the unlimited corporate greed and power hunger of those few people.
30 BILLION QUADRILLION
They must have a pool of sharks with "LASERS"
Hm, that's 3*10^25 .
Why do we assume that Microsoft is naive enough to invest so much money in openai only for openai to turn around and declare 'AGI!'?
Yes.
well all my models all take like forty zillion bajillion, so there
Yannic, *this* is Monday. ;-p
Can we expect one quadrillion likes?
I don't get this obsession with AGI being the most important goal. Models which excel in specific tasks could be more revolutionary than models which replicate human-like intelligence.
if only we could automate the humans which make narrow ai's
Humans are *_desperate_* to relieve themselves of responsibility. They want to repeat what they're told but don't want to be blamed for being wrong. They know they can't claim machines are responsible for their own decisions, because machines aren't alive. AGI is these people's mental ticket back into Fantasy Land. They think they'll finally be able to have their cake and eat it too.
AGI creates the models you've described. All of them.
humans are obsessed with relieving themselves of responsibility. they want to repeat what they're told, but not be blamed for when its wrong. they know they can't blame machines for decisions in their current state. AGI is their ticket back into fantasy land.
@@stuartspence9921 Look up the "wenger 16999". Sure it has all the tools, but it's just not a practical tool to actually use.
a $100B data center IS AGI
Odd to be verified like that
Maybe Elon had nothing to do and thought to himself "yep, he looks like that guy from CZcams" and pressed a button.
It will be worth it if they use it to build an actual Stargate. Otherwise, I'm not convinced.
Tokens dont burr, engines do.
I think Emad is right, there's no way stability can compete with the other for profit companies, unless they go full on free and open source. That would give companies that rely on art and music made by starving artists (most digital artists) a reason to continue propping them up with funding. Game companies, hollywood.. they all benefit from and very much need cheap labor making their digital assets. it's why so many of those big companies bankroll blender. Free tools means cheaper labor, and that more people can learn to master those tools. But if a company has to pay to use a program, they'll pay for the biggest available companies product, that has the best tech support and legal team. That's gonna be microsoft. I think this could well be the end of stability ai.
All the big tech companies are stuck in this crazy expensive arms race, having to train ever more expensive models, now '100 billion' worth... in case it turns out transformers can actually justify the cost... but these companies know this is all very sketchy speculation. The 'best' AI researchers in the world have no idea if transformers can become more useful in a commercially viable way, they'll just take their 1million salary and make gpu go brrrr. It must be kinda terrifying for these large tech company execs like Nadal (not that we should care about their emotional state) as we've already seen the latest round transformer hype sort-of peter out... the $ use cases being relatively minor for even the most state of the art models (notices even Altman only mentions 'coding') but nobody knows transformers might are capable of if we keep throwing billions of compute at them to stir the pot of linear algebra or slap in more data (that you just 'found' on the internet) Nobody even understood transformers would be good for Q/A, back when they were designed for translation. Good time to be NVIDIA or AI 'researcher', bad time to be anyone else.
You're 100% right, nobody knows, but what I was always thinking, it's gotta be a better way to run this things than GPUs, like a special chip designed for ai...
Remember MS makes software for the military. You can guess what the next words will be.
I think they're now firmly in the "steal as much as we can before the house of card collapses" territory. Making investments of ridiculous size clearly reminds of the "too big to fail" risk management techniques so successfully applied in 2008.
They said billion quadrillion because they didn't want people to giggle at 'sextillion'
What did Ilya see?
What the f*k did Ilya see???
Who is Ilya Galt? Eh... John Sutskever? Eh... Ilya Sutskever? Yeah, where is Ilya Sutskever???
So that computer would be what, 500k times better than the one that trained GPT-3. lol
I prefer when the videos stick to factuality instead of mediocre commentary on "safety", "opensource" binaries, and "AGI".
Septillions of (16 bit?) flops? :/ 🤔 probably not then.
30 billion quadrillion? Wtf is even that?
DOES THIS GUY WANTS ME TO WEAR SUNGLASSES TO WATCH HIS VIDEO OR WHAT!!!
Only 2 views in 12 seconds? Yannic's reign is over 😞
Was expecting to model views(second)=second³ {0
How much you can extrapolate to 12 hours with that prior?
First!
Old news
😐☝ 1 Million FLOPS (Dr Evil)