[ML News] Groq, Gemma, Sora, Gemini, and Air Canada's chatbot troubles
Vložit
- čas přidán 15. 06. 2024
- Your dose of ML News!
OUTLINE:
0:00 - Intro
0:20 - Gemma & Gemini
3:40 - Groq
6:30 - Nvidia EOS Supercomputer
7:15 - Gpulist.ai
8:20 - Demis Hassabis on scale
10:10 - Hardware wars
12:05 - Sora
15:10 - Gemini 1.5 Pro & Long Context
18:45 - Air Canada must pay for chatbot mistake
23:30 - Giant Rat Balls
26:25 - Various News
References:
blog.google/technology/develo...
altryne/status/17...
paulg/status/1760...
groq.com/
mattshumer_/statu...
/ 1759483896322781584
wow.groq.com/news_press/groq-...
tianle_cai/status...
/ 1759728119005712837
/ 1759720197055791188
/ 1759704303810519271
/ 1759709223276228825
www.techpowerup.com/319172/nv...
andromeda.ai/
gpulist.ai/
archive.ph/G6POi
www.tomshardware.com/tech-ind...
futurism.com/the-byte/ai-dest...
_akhaliq/status/1...
_Borriss_/status/...
/ 1758650919430848991
tsarnick/status/1...
MartinNebelong/st...
OriolVinyalsML/st...
/ 1759804492919275555
mattshumer_/statu...
haoliuhl/status/1...
github.com/lucidrains/ring-at...
bc.ctvnews.ca/air-canada-s-ch...
arstechnica.com/tech-policy/2...
www.cbc.ca/news/canada/britis...
kareem_carr/statu...
arstechnica.com/science/2024/...
www.vice.com/en/article/4a389...
karpathy/status/1...
karpathy/status/1...
www.nature.com/articles/d4158...
/ 1757359611399532921
cohere.com/research/aya
OfirPress/status/...
github.com/mut-ex/gligen-gui
www.projectaria.com/datasets/...
StabilityAI/statu...
gordic_aleksa/sta...
huggingface.co/gordicaleksa/Y...
huggingface.co/datasets/nvidi...
interestingengineering.com/sc...
archive.ph/caW1Y#selection-49...
newatlas.com/robotics/seeing-...
os-copilot.github.io/
www.businessinsider.com/apple...
techcrunch.com/2024/02/16/ant...
archive.ph/Gbcgb
Links:
Homepage: ykilcher.com
Merch: ykilcher.com/merch
CZcams: / yannickilcher
Twitter: / ykilcher
Discord: ykilcher.com/discord
LinkedIn: / ykilcher
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: www.subscribestar.com/yannick...
Patreon: / yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n - Věda a technologie
![[ML News] Llama 3 changes the game](http://i.ytimg.com/vi/kzB23CoZG30/mqdefault.jpg)
![[ML News] Llama 3 changes the game](/img/tr.png)
I really like this video format about news. It is great to have someone with your great background and reputation to share his insights. Thank you.
_"I think I'm seeing colour…I'm actually not so sure now."_ - Guy Using Computer Wearing Sunglasses
"You can redefine AGI and 'something else' as you wish, and then you can always find something else that is still wrong so that you're still correct. If you do that for long enough, your name will become Gary Marcus."
9:07 😂
I really like this [ML News] bit. You should try and do one of these weekly. I'm not in computing or IT industries, or really even hobby but I like to try to stay informed on the technologies and I found you explain things very well so I get notifications on your vids. I don't watch all of them but some catch my attention. This "news roundup" format is really good. Thanks!
Surprised you didn't mention anything about the new 1.58 bit quantizing paper, lot of great potential in this
not really . . . ternary numbers have been around since ever, and they did not took off. Silicon works better with binary.
Because it's dumb for anyone who has ever done quantization on a transformer.
"You do it long enough, you become Gary Marcus "...made my day!. 😊😅
RingAttention is not an approximation. It's true quadratic attention with massive parallelization. IIRC, from the paper, training the WMT model took roughly 1k TPU-minutes per 1M token training sample. I think it was 4M tokens per batch, ~8 minutes per batch, 512 TPUs. Insane.
Very happy that Andrej Karpathy no longer working for OpenAI. A clever, nice person and a great teacher; very much unlike in character like the company which now pulls all the strings within that "non-profit" "startup".
10^25 flops limit, no worries, just use integer operations only.
😂😂
Amazing as always. Thanks for your videos Yannic, both ML News and Paper video formats are incredibly awesome.
These news videos are amazing. Please do them more! Ide love to see this become a weekly series.
Agree.
I also want more, but weekly is expecting a lot from someone who actually works on machine learning (as opposed to those churning out videos on AI who clearly don't, which becomes obvious to anyone who does when they fail at discussing something technical - it's much better to stick to those who do). The series used to be roughly monthly and that, along with his deep dives into particularly important papers, made this my favourite channel on the tube by some margin.
@@EdFormer good point. The main reason i love this series is because yannick actually knows what hes talking about. I prefer high quality news updates when there are relevant updates to give, rather than more frequent low quality updates.
another episode of my favourite series!
thank you
Elderly politicians who don't understand how to program the alarm clock on their coffee maker, creating laws about bleeding edge technologies that no one really completely understands, is brilliant.
Air Canada really told the court to take it up with the bot 😂
Even if the chatbot was a legal entity, separate of Air Canada, since it is operating as an agent of Air Canada, the end user would still sue Air Canada and Air Canada would be responsible. It would then be up to Air Canada to sue to the chatbot for damages. It works the same way with subcontractors and similar organizational structures.
I am looking forward to the Air Canada v. Chatbot law suit.
Air Canada may win with Chatbot. The court system is discriminating against LLM and does not accept argumentation based on list of hallucinated precedences...
@@useodyseeorbitchute9450 I'd imagine the chatbot would be it's own lawyer too which is never a good idea.
@@i9169345 Hilariously, it might even win. I heard about some models getting 90th percentile on the American Bar exam.
Will be waiting for the ring attention video!
Thanks for the Groq memory analysis.
Really love this format ❤
Evaluating black-and-whiteness of an image using black glasses was the best idea 😂
Thanks for ML News, Yannic!
Commenting on the FLOPS limit, I spent a large portion of my programming life doing systems that have been mimicking floating point calculations with integer arithmetic - for speed, in the lack of the FPUs then. A reminiscence of that is multiplying by 355 and then dividing by 113, an equivalent of 6-decimal digit precision multiplying by Pi. Curious of creating an integer arithmetic transformer and LLM.
Qualcomm has a card called Cloud AI100 (yeah, stupid name). But their "ultra" variant is 128GB in a single 150W card using LPDDR5. And it might be half as fast as a single H100 for int8... Which sounds exactly like the one I need for my workstation.
The nvidia eos system trained the new Starcoder2 model. using like 430 H100 (also new the stack v2 release)
I smell a future business model - convince a bank teller chatbot to tell you that the bank owes you money, then sue.
As far as I can tell from the 'technical report' the compute for Sora refers to the compute that went into training the model, not the compute used for the generation.
Thanks, funny and inspiring.. as usual 🙂
You can already your bind user-defined scripts to user-defined voice commands using Talon...
OPEN AI = CLOSED AI
Does Sora compute refer to inference? My first impression was that it was about compute that went into training.
Yesssss ML News is back 🙌
The arbitrary threshold in the EU AI law is the result of France lobbying on behalf of Mistral.
That being said, laws are always full of arbitrary numbers and one could argue have to be. Take age limits for various things.
ChatGPT:
"""
Absolutely! There are always some risks, here are a few:
1. Proximity to sensitive eye tissue [...]
2. Potential Allergic Reactions [...]
[...]
"""
How do you distribute these models? I want to be able to use my 512mb (X10) usefully.
Whats a asymmetrical distributed architecture?
One correction, sora isnt a single shot clip, it can do multiple shots and angles, see the wool helmet scifi clip
_Progress in hardware has followed an amazingly steady curve in the last few decades. Based on this trend, I believe that the creation of greater-than-humans intelligence will occur during the next thirty years. I will be surprised if this event occurs before 2005 or after 2030_ - Vernor Vinge 1993 - "Technological Singularity"
I think the obvious risk of the eyelash robot is that it is going to spaz out and drive those tweezer spikes thru your eyeball and into your brain.
The cool thing about Groq is that they are still using 9nm chips, which is ancient technology at this point. They signed a deal with Samsung (official) already, so let see how much the performance can be when they started using 5nm by the end of this year.
I can see a bright future for renting out GPU clock cycles and using the hardware to heat water and buildings.
Groq is trying to do what Graphcore already failed to do.
you should do a video testing gemini when you get access
11:48
I don't think I've ever seen Yudkowsky's statement about bombing datacenters *not* be taken out of context. The actual context is this: *if* humanity were to put in immense efforts to prevent the creation of AGI or ASI, then bombing datacenters would be among the many things that would be done - "be willing to destroy a rogue datacenter by airstrike" is always described as "Yudkowksy wants to bomb existing datacenters, as in right now."
It is still bizzare bullshit
Bruh
Yeah, another monday!
If anyone would decide to adopt Groq/Graphcore, the spatial computing of SRAM-Based Computing, to run application such as LLaMA-2 70B, it would be mandatory to acquire more than hundreds of cards to have sufficient allocation for weight stationary and KV Cache. and it would be implausible to achieve latency at optimal ms/token to do the tensor parallerism.
what kinds of applications do small models have?
* Toys
* you can fine-tune them and see that your fine tuning works
* general experimentations
* promoting of your ecosystem and general warm fuzzy feeling of "we give it out for FREEEE!"
@@JurekOK yeah those are good applications. i was thinking about it later on, you'd probably have pretty good luck using them to categorise.. which might sound trivial, but, like the toys application you mentioned, you could break down unstructured text into data points, !
20:00 To be fair, the chatbot that the lawyer used said this would be a good argument. To say that the chatbot they deployed was a different legal entity. Some sort of chatbot-inception? 😆
Who is eagerly waiting for Sora?
I’m waiting for the open source version that isn’t woke lobotomised.
Thank you for your "AI Act" comments, I have tried fighting it with no success
Sounds like that article was written by LLM, they always find something bad to say to balance the response out.
Good stuff, appreciate not having "ai" in the title and mocking of Yudkowsky
Nobody cares
Yep, ring attention video please
Why is 7B chosen? Many models are released with the 7B variant. Is there a reasoning behind it?
I was told the sizes are chosen to optimally fit the GPU's memory.
Because at 16 bit precision, you can fit these 7B parameter models into about 14GB of memory so GPU cards with 16GB can host the model to perform inference.
My worry about that eyelash robot is that its manipulators look very … stabby
ring attention is exact
And the blind don't need a robot dog. That's dumb. They can have a pair of glasses with cameras in and AI that tell them what the pertinent situation is. the dog form is quite needless.
a blind person could become quite fully functioning with modest AI. So much so that if you weren't paying close attention you wouldn't know they are blind.
Can you do a deep dive on AI accelerating AI training? AI helping on improving datasets? I got the feeling that's driving massive additional growth
"If you do this long enough, your name will be gary marcus" 🤣
20:28 🤣🤣🤣 -> 😭
Hold on a sec… Why do I have to watch MediaMarkt and Kärcher commercials in the background?
WTF?
Ah, Eliezer Yudkowsky, the other side to Yann Lecun on the pessimist coin. If we listened to either we would not have current AI technology.
I think the problem with Sora may be that it can only generate animals, landscapes, human faces, scifi neon pictures, waifus in scifi neon landscapes playing with animals and not much else. Edit: I forgot, it can also do teddy bears and streetviews.
Gary is still right.
I remember GraphCore achieved 1000T/s years ago and no one is talking about it today😂😂😂
Ring attention isn't *that* new... I read the paper back in November
24:00 To be fair, this only happens IN RATS. 😆
I wonder if the Air Canada thing sets a precedent where companies have to be responsible for things that chat bots say.
dimiz hassabiz
Demi as Demi Levato
Hassabee as is Bee 🐝
"Don't Panic Yannic" at it again!
Except about regulations on training ML models to dox people. Then panic.
*Potential and limitations*
While this long video helps the user gain information, an overload may cause a headache, or aversion to science in case of user religiousness, we hope this research can further improve the search of a cure to religiousness to prevent misuse.
I thought that people following woke religion are freaking the most seeing results of machine learning, as such simple algorithms are hard to shame into not noticing some patterns. There is effective whole field of fighting with "bias" which is supposed to protect adherents of that particular religion from detecting patterns that would make them deeply unhappy.
21:45 how come? who is would retain a lawyer for 10-20k Just to get a 600$ refund?!?
I remember when people were renting GPUs for bitcoin mining. Wonder if that's still a thing.
E: Yes, it still is.
Gemini : Gemma :: Yannic : ???
GeminAIc
"The height of persistence is to enter the wrong password until the computer agrees"
Now you can attack LLMs until they agree that company owes you money :)
someone's gotta tell Eliezer about the risk of eye infections from fake eyelashes. i think humanity has about 1% chance of surviving that 🤔.
We need LPU chips in all Android devices
Oh dear, that fake eyelashes gives me nightmares. What if something NaNs out and the robot just drives its forceps straight into your eyes?
That was my first thought. I would need to see strong safeguards before letting a CNC machine near my eyes. Maybe it has a hard end-stop though.
@@andybrice2711There is nothing that can convince me to put that thing anywhere near my eye.
The "person" who commented about risk potential for the article while ignoring the possiblity of poked out eyeballs was probably ChatGPT.
I'll do it for 750B. In advance of course, to my swiss bank account.
23:26 there’ll be a new wave of tiktok ai chatbot hacking influencers
Says a lot about Air Canada and their integrity! Not surprising
Pff no shit Groq is so fast if it's running on pure _sram??_ That's insane. That's the on-die cache memory your computer only has a tiny bit of because it's ridiculously expensive and near impossible to scale up. Yeah, my computer would run just about anything stupidly fast if I just dumped all the ram and replaced it all with raw CPU cache...
Massive layoffs, massive reduction in available jobs, skills becoming obsolete
Love how Yannic has no idea how black&white videos look like
LLM already attempting to rewrite recent history.
8:55 i agree and also disagree with Hassabis here. I agree that there's still so much space to be explored in the optimization problem in ML and that searching for better optimization is extremely useful.
But i'm not sure i agree that there is absolutely has to be something else than scale to get to AGI. Maybe that the current methods are sufficient. In the last 5 years we have seen a complete paradigm shift with transformers only through their scale, so it could be that other types of emergent behavior also can come through scale.
But in any case that general problem solving was achieved some other component in the total architecture provided some iteration over steps.Which makes sense, you'll need time to think and explore different paths in the problem space. You cannot just waterfall your way to the best answer.
Only for simple problems where you'll need at most, say, 3 iterations to get to the right conclusion, the raw LLM approach works.
Who cares about unrealistic rat balls, just look at all the cute whiskers! P.S. The editors have probably used AI to green light the paper.
Hey ChatGPT tell me a joke!
ChatGPT: "Gemini"
OMG, you kill a basket full of young cute kittens if I don't get a free ride...
I really think that user generated content should not be considered the intellectual property of the social media platform it's posted to. If I post a video to CZcams that plays Micheal Jackson's smooth criminal in the background, that does not magically transfer the IP rights of Micheal Jackson's song to Google. Even if Micheal Jackson posted his song to youtube, that still does not make Smooth Criminal Google's IP. Reddit does not (and SHOULD not) own the copyright to content that I post to their website, I own the copyright to works created by me, user generated IP is user owned IP. How much money you spend on being a web hosting service does not factor into the IP equation.
lol..."You are not going to get more from scale"
Yeah...ok...who is deciding how to measure "more"
This whole industry is so full of shit for all the wrong reasons
Eliezer has managed to convince people he knows something, but based on the interviews i've seen, he doesn't know shit. he keeps talking about "the ghost' in the model because he loves the anime ghost in the shell. People should stop listening to him.
I do think there are legitimate concerns about the risk of Artificial Super-Intelligence. But yeah, his beliefs don't seem grounded in reality. Like his speculation that ChatGPT might be sentient, when it's basically just a stateless word-calculator.
Mostly he has managed to convince himself that he is very smart.
His only achievement is writing 1000 page Harry Potter fan fiction. Dude’s a fraud.
Ehm yannick are you aware that the top 3 most cited AI scientists (Geoffrey Hinton, Yoshua Bengio, Ilya Sutskever) and virtually all AI company CEOs actually agree that this stuff poses an existential threat? Seems a bit naive to giggle and just skip past this, time to take the countless warnings seriously IMO.
Otherwise nice vid, like the format!
boo hoo muh existential threats (to zionism monetary hegemony)
Yes but they are going to solve the problem if you give them money
@@samanthaqiu3416 Well, to those players as well...
We have quite a few genuine risk... and bunch of people who try to become celebrities by giving cryptic and totally worthless warnings.
The existence of risk is not in doubt, but those laws are not going to do anything about it.
In fact, to destroy humanity the AI needs to control considerabele resources. And the motives of a rouge AI and a big company or an oppressive government are well aligned. So giving privileged access to the technology to big actors while depriving the individual of the same power is only increasing the risks.