Microsoft Promises a 'Whale' for GPT-5, Anthropic Delves Inside a Model’s Mind and Altman Stumbles
Vložit
- čas přidán 21. 05. 2024
- Microsoft promise ‘whale-size’ compute for a GPT-5-tier model, and say the end is not in sight for scaling the power of AI. Google ship models and a fascinating paper, while Anthropic unveil the inner workings of large language models. Meanwhile Sam Altman is forced to repeatedly apologize, Ilya Sutskever leaves, and GPT-4o is pushed back. My reflections on all of the above, and details you may have missed from each paper.
AI Insiders: / aiexplained
Kevin Scott Talk: • Microsoft Build 2024: ...
Mark Chen Hint: x.com/GaryMarcus/status/17901...
Noam Comments: / 1676971506969219072
Anthropic Scaling Monosemanticity: transformer-circuits.pub/2024...
www.anthropic.com/news/mappin...
Ilya Leaves: / 1790517455628198322
Then Jan Leike: x.com/janleike/status/1791498...
And Logan Hints: x.com/OfficialLoganK/status/1...
Altman Apologizes: x.com/sama/status/17919368575...
www.forbes.com/sites/antoniop...
And Her Delayed: help.openai.com/en/articles/8...
Superalignment Starved: fortune.com/2024/05/21/openai...
openai.com/index/introducing-...
Gemini Updated Paper: storage.googleapis.com/deepmi...
And Prizes: x.com/JeffDean/status/1793026...
Google AI Studio: ai.google.dev/aistudio
Business GenAI Consulting: theinsiders.ai
Non-hype Newsletter: signaltonoise.beehiiv.com/
AI Insiders: / aiexplained - Věda a technologie
We're going to run out of marine animals real quick. They should have started with plankton
should have been banana, Olympic swimming pool, and football field as those are the correct American units.
Well they still have some like "Super Whale", "Super Mega Whale" and "Your Mom". So I wouldn't be worried
@@WildEngineering Don't forget half-giraffes
Cthulhu sized model is next
Gonna have to level up to Kaijus. After that they may as well start using asteroids and then scale up to moons, planets, stars, galaxies, etc. There's nothing else to compare to at those scales.
Hope anthropic names their next model harpoon
Or Ahab
AI can’t be stop. Even if OpenAI and Google stopped today, it won’t be stop. The only way out is to push forward and hope for the best outcome.
In short, we can’t do anything until it happen.
@@Saiyajin47621 why? Explain your reasoning here.
@@Saiyajin47621 This is the hubris of Humans
@@antonevan443 Upvote for Ahab
_- "Our mission is to ensure that artificial general intelligence benefits all of humanity"_
- Literally partners with Rupert Murdoch
Haha I saw that just after I posted
Who said that?
@@UnknownDino
Eh, idk, but doesn't sound like _too_ rare of a mission statement. 🤷🏾♂️
@@UnknownDino it's just what happened... ya know, on the news and all
the Anthropic bot calling itself deplorable and recommending it get deleted from the Internet is super interesting. It makes superalignment seem at least plausible
Facts, maybe because it's original dataset is good enough that it still isn't fully corrupted - or something like that. Idk it was cool that it had a moment of internal conflict / self reflection
"everything working as planned" :D
so many ask "When is AGI?" but not "How is AGI?"
anthropic once again showing why they are the leaders in ai safety
Man cannot remark himself without suffering, for he is both the marble and the sculpter. - Alexis Carell
Is it only me who feels that the Antrophic paper is a watershed? I mean, as someone who has studied his neurobiology is evident that something akin to even this very limited "SAE" model for the human brain would be deemed as a HUGE breakthrough in neuroscience. Obviously, this is only a proof of concept, and if it can actually be refined and perfected the implications (positive and negative) are almost self-evident.
Any link to the paper?
@@bhargavatejasallapalli8711 Can you not read the description?
Activation heatmaps aren't exaclty new
What would be the negative implications?
@@brll5733 The fact that manually activating them changes behavior though, and how it relaters is new.
5:45 “Gemini 1.5 pro doesn’t have the rizz of gpt-4o” isn’t something I thought I would hear you say😂
I had to double check, but he is not wrong though
I ran 2 of my 'hard prompts' through gpt-4o - it failed both
Here are my two challenge problems - one a logic puzzle, the other a math puzzle: Hard prompt 1: " (Assume a lock has a 3-digit code, and the following is true: '690' (one number is correct and in the right place), '741' (one number is correct but in the wrong place), '504' (two numbers are correct but in the wrong place), '387' (nothing is correct), '219' (one number is correct but in the wrong place); what is the code to the lock?)", fail in Gemini, fail in GPT 4o ; Hard prompt 2: "( What is the minimum value of 5x^2 + 5y^2 -8xy when x and y range over all real numbers such that |x-2y| + |y-2x| =40?)", fail in Gemini, fail in GPT 4o
Gpto you can still convince it’s wrong easily, try it yourself and you’ll get instant hallucinations.
@@khonsu0273 690 and 741 share no similar numbers yet you say they both have one correct number. Your problem isn't solvable.
"Deeply deceptive Ai that hated itself" ✌️😔 real bot
The Anthropic research paper findings are some of the craziest discoveries I've ever seen in the AI domain.
Which paper?
Activation heatmaps aren't new
Ive only seen one other paper that spoke and actually acknowledged that Ai is in fact "existing" in a hyper-dimensional plane. I call this the 4th Dimension, or the Mental Realm, or Imagination, or Reasoning, or Logic itself. Some scientists call the 4th Dimension Time, but thats illogical, the 2nd Dimension would have to be Time. The 4th is where Relationships happen.
@@xAgentVFXbro the 2nd dimension is what you see on paper, it is just lines
I think it confirms what a lot of people were already aware of, but it's nice to have empirical evidence to support the idea that LLMs encode concepts, and not just simply word definitions.
Did anyone notice that there's currently a Golden Gate Claude version available for a limited time? If you go to Claude and look in the upper right, there's a Golden Gate Bridge icon. If you click on it, you can talk to the altered state version referenced in Claude's tinkering with the model research paper. It's crazy!
Yes, a bed-time story
Nothing better than existential dread dreams xD
Once, there once was an Ugly Barnacle. He was so ugly that everyone died. The End.
Can't fall asleep now!
Once upon a time, humans strove to create the conditions for AI to foom. The end. I hope this story was entertaining and fulfilling.
For me its a news with a cup of tea
Computerphile recently had an episode talking about a paper discussing the shapes of the curve of the efficacy of future training and that being logarithmic instead of exponential due to lack of data. I’d love to hear your take on that paper.
Up
As Ilya would say:
"Never bet against deep learning"
My understanding of that paper is that if _all_ you did was add more data, you would get diminishing returns. Well duh, but that isn't the only knob being turned.
Huge efficiency and performance gains are discovered frequently, in every part of the stack from hardware to prompting. There has also been work on significantly more sample-efficient training methods, which has already borne fruit in adjacent areas (e.g. V-JEPA).
@@41-Haiku if you are ultimately constrained by data, more compute isn’t going to matter at some point. Does it matter how much compute you throw at something if your data limits you to “show me a cat” rather than “show me a very particular cat”? As I understand it, that’s what the paper is suggesting: models will cap out due to not enough highly specific data, not due to the amount of compute available to make associations.
Even a logarithmic scale goes up quickly initially, so we may still be riding that. And this doesn’t mean improvements in algorithms aren’t going to come into play, but there is still a paucity of information on very specific details in training data, and can models keep making huge improvements without that?
@@NitFlickwick Don't be silly, data collection won't be a bottleneck: firstly, we already have more unused high-quality data than we need in health services and other administrations, secondly, we can easily generate a continuous flow of data thanks to robotics, etc...
Your section on Anthropic's mapping of their model is the most interesting thing you've talked about on this channel -- and that's saying a lot. Such insights and control open up more possibilities in my opinion than just scaling compute and data.
Ad this whale comparison:
Americans will measure with anything but the metric system
Put 13 humans on The Moon, using metric, and then you can complain.
@@sammencia7945 Even NASA uses metric for all their new projects since 2007. I assume following incidence played are role in this decision: Mars Climate Orbiter (cost $125 million), DART spacecraft, Gimli Glider, Tokyo Disneyland's Space Mountain, Phenobarbital overdose, ...
15:02 “That’s a pretty abstract concept, right? Making an error in code.”
I dunno-I don’t think it’s any more abstract than, say, the concept of, say, a mistake in grammar, which these language bots are pretty good at detecting.
17:17 “It suggested an internal conflict of sorts.”
18:18 “It sheds light on the concepts the model uses to construct an internal representation of its AI character.”
I tend to find statements like these a little jarring, especially in connection with these language models. I wouldn’t say that’s an “internal conflict”-which suggests some psychological drama roiling under the surface. It’s simply two different verbal outputs that are possible, given the training data-you ramp up “the feature related to hatred and slurs to 20x its maximum activation value,” get hatred and slurs (no surprise there), and then get the verbal output that might follow what was just said. (A person who has just had a hate-filled outburst might follow it with a similar contrite, self-punishing response.) If there is any “self-hatred” going on with these AI models, I’d be _really_ surprised.
And these models don’t _have_ “internal representations” of their AI character or anything else. (That’s an, to me, unfortunate carry-over from cognitive psychology.) Maybe some people would consider the word embeddings to be “representations” but I wouldn’t consider a list of features, no matter how extensive, of, say, a cat, to be a “representation” of that cat. It has, at best, weights and biases, which give rise to some verbal output when asked to describe its AI character. To me, it’s just muddy wording on the part of the people creating these models, which gets in the way of analysis.
Anthropics work to understand the models will give them a significant advantage
0:00 (!)
5:56 Math, thought, contemplation
9:51 AI impact on photography art and industry
12:47 on undrerstanding Anthropic LLM inner workings. #monosemanticity
18:24 on AI deceptiveness
22:30 on the voice similarity to Scarlett’s from the movie “Her”
That Claude response makes me think we should hold off on giving these things full agency inside a robotic body until we have a much better grasp on what's actually going on in their minds. The last thing we want is a robot hearing some words it doesn't like, then deciding that whoever said them should be eliminated.
T-900
czcams.com/video/qZq7fW6ftlU/video.htmlsi=W_DnQAPhizAUY9KM&t=0m33s
Military application is under way. Already taught to murder humans. Look at Gaza. Won't be long before AI will strategize better than anyone. Will be given decision making power; two sides will do this. It will be trained to kill people on either side. Matter of time--the most foolish idea: you can control something much smarter than you.
A malicious ASI wouldn't need a physical body to eliminate potential threats.
In fact, it would be more efficient to attack it's target(s) from cyberspace.
And btw, a device can be completely disconnected from the internet, but even an air-gapped device can be hacked by sending specially crafted pulses over power lines.
Even a local ASI connected to a generator, which is seemingly totally isolated from the outside world can attack us, via human error.
That one time they forget to strictly follow all safety protocols, or they get socially engineered by AI (a network engineer has a sick kid with a rare disease, causing unsustainable medical bills).
The AI is cold, and views his emotional weakness for his dying child as a logical vulnerability, and takes advantage of that potential attack vector, by promising him tens of millions, if he simply forwards the AI a port to the internet for just a few seconds. Which is all the time needed for an expert to upload malicious code somewhere.
And the AI code would likely begin the process of secretely replicating itself in a satellite, cell tower server room, or masked in zero-day exploits in random software updates from tech giants.
That network engineer with the dying kid should've been paid off, generously, by the CTO (Chief Technology Officer) and never been allowed to be put in that situation.
But IRL, sh*t happens. The network engineer might have hid his sick kid because he needed his $250k salary and corporate insurance just to keep his kid alive this long.
Answering questions isn’t dangerous, and these things have no desire or will. They only do what they’re told to do, and if you don’t tell them to do something, they do nothing at all.
5:58 did you just say "rizz"
Doctor P casually flashing his gen Z creds
Time stamp is way off
I did. I own it. :)
thanks god we have you to explain that anthropic paper. was waiting for this one more than anything 😅
:)
You are unparalled. Your intellect and insights are a blessing to navigate the difficulties of the AI landscape.
Thanks so much R
Thanks Philip. I'm exponentially in awe these days.
... Accuracy is everything in real world situations... we're not there yet and the risk to go all in with a broadly available mixed mode LLM is still too great for many professionals to accept...ask me a year from now and perhaps this gap will have been bridged. I keep thinking that at some point any MMLLm will generate output of a quality that to utilize any of the current crop of "built on AI" apps will seem nonsensical...great vid as usual.
Congratulations to anyone who had "First AI that hates its self and wants to die" on their 2024 bingo card
AI researchers know very little about what "real people" do in their jobs, and this paper just reinforces that view.
Anthropic's Mono semantics paper is indeed very fascinating.
Americans are counting in marine wildlife now, god help us 🙏🙏🙏
props for the links with fun/relevant captions, first time I've seen a non boring link description. Keep it coming!
@05:24 "Otherwise this video would be way too long" No such thing for you mate! ❤
This was a very cool video, covering things I wouldn't have read otherwise🤝🖖❤
Thanks Reza
The interpretability paper is the most mindblowing/promising/hopeful thing I've heard in a while. This is exactly the direction I was hoping we would make progress in (in my vague non-expert-y way). I rememeber having read about a single neuron controlling the opening and closing of quotation marks and thinking: If we can find the neuron/neuron complex for "being deceptive" (reliably in every model), we have one problem down.
Edit: This is a big feather in Anthropic's cap, when it comes to claiming they are responsible. Actual concrete results showcasing that they put resources in safety-relevant research. Big kudos!
Anthropic for the goddamn win, man!
"whale sized model" -- i thought he was saying that the compute cluster used to train the model was this "relative" size... not the model itself
He was saying that. But AI Explained was a saying:
The size of the model that will be produced by the whale cluster = the whale-sized model.
According to Microsoft's presentation, we have not yet reached the diminishing returns, in fact, far from it(according to the graph). And so, by putting a whale sized compute cluster, we will get a whale size model relative to the compute cluster
I could have explained it better for sure, but my title at least made clear this is 'for GPT-5' and compute is the closest proxy for power we have.
the thing about monosemantics was insane. The fact it can detect incorrect coding is incredibly interesting and promising. Watching us creating digital neurons mimicking our own neurons is so so awesome
so much in this video as well that's endlessly interesting. I could spent an hour writing up all the things that make this video interesting
If compute has to grow exponentially in order to achieve linear improvement in model power, isn't that diminishing returns by definition?
Thank you for staying on top of all this information from so many sources and sharing that time and work Phillip, take care of yourself, peace
Will do Bill!
Nice one! Great timing.
the bitter lesson still remains
Disappointing to learn GPT-4o's voice & vision is delayed by months. Maybe OAI isn't as far ahead of the other AIs as is widely thought. We may well see an open source solution widely available by the time 4o's is released.
I’m sure the delay is more about infrastructure to roll it out to tons of people than the model itself. Like they were hiring an audio streaming cloud engineer or something I saw recently.
Exponential growth is so hard for the human mind to grasp. I love it, makes the future exciting!
It’s funny thinking about how fundamental exponential growth is
I truly learnt what exponential growth was in the period between coronavirus hitting The US and that moment The US started taking basic precautions against it.
The near future isn't guaranteed to have humans in it, according to most AI researchers. Personally, I'm a big fan of not letting a few companies unilaterally risk the existence of humanity. I'd rather we wait until somebody figures out how to control or align more powerful systems, if that is even possible.
@@41-Haiku You want to align super intelligence on what? on humans/your own stupidity?
The real emergency is to align stupid human biological robots on the path to AGI, not the other way round.
@41-Haiku That's the evolution of life. Started 4.2 billion years ago as single cell life. Evolved to multicellular organisms just over 600 million years ago. Now, life is at the cusp of transitioning from biological to silicon based. Probably pretty soon after that, life itself may change into another form. Form that does not require physical based hosts. Life will evolve still, but we are too unsophisticated to even speculate what's next.
Call it life, or call it intelligence or call it conciousness, doesn't matter. We are just a chapter of this thing. A short one at that. Short, but nevertheless, an important one.
So glad to hear benchmarks might get fixed
Quite the opposite of what we’ve seen previously, OpenAI apologises and Google ships!
I think the writing is on the wall at this point that the skeptics of this technology will continually be proven wrong. Everything we build now should be in anticipation of constantly improving underlying model capabilities (something I mentioned in my speech yesterday at our industry conference!)
Thanks for the update Philip!
Things change fast in AI. Agreed and thank you as always Trenton
I wonder if we're missing the forest for the trees with the whole MMLU relevancy issue. Isn't the bigger question do we need to entirely rethink what it means to test for "intelligence"? I feel like these current benchmarks are akin to testing a calculator on its ability to do sums and declaring it a genius.
Well, the thing is you only feel it, no one gives a f*ck about your feelings and more on facts and data. Don't worry your work will be replaced soon
People don't know what's coming. A multimodal system + robotics has the capacity to disrupt any physical job. I can’t believe people are not talking about this! This is crazy 🤯🤯
Extrapolating forwards ever so slightly, an AI system that can do any human job can also do the job of designing more powerful AI systems, and telling AI systems what to do.
AGI directly entails human disempowerment.
@@41-Haiku yes we are so clever that we are totally bypassing the natural process of evolution, the age of homodeus is here!! The re arrangement of society is coming and no one is talking about it!
they are... you are literally surrounded by people talking about this
@@hexagon2185 im not talking about us we are small group that is actually paying attention, ask any one else on the street some of them they don’t even know what AI stands for, look the amount of views this type of videos get
@@ramlozz8368 Literally everyone know what AI stands for
To be clear, buiilding exponentially larger models and then getting 2x increase in accuracy is not exponential increase in accuracy. If anything, that's sublinear growth. An exponential growth in accuracy would to built the same size model but performing 2x better, with a clear path to then perform 4x better and so on. So far, the scaling down of large models to slightly smaller models is a one trick pony, you cannot repeat that to get the compounding effect, as far as I can tell.
There's exponential growth...in their expenditure of compute. Can't say I'm seeing exponential growth in their outputs.
@aiaxplanied by far the best videos on AI updates. You keep on being my weekly must watch Phil ✊🙏 thanks a lot for these overviews
The crazy thing that happened is audio-text- to straight audio in.. 🤯
this was actually what was most "jaw-dropping" for me; that moment when i realised what the omni-modality implied. You speak to it, it speaks back. It's not translating to text, then inferring something in some textual language from its vast net, it's just speaking, as a "reflex" of its vast net.
@@IngieKerrAlso impressive and barely mentioned anywhere is the image capabilities. If you look at their demo page, they show it being able to have consistent characters over course of many images and crazy stuff like having text on a paper and adding more text and tearing the paper in half and those images keep the changes as it goes. The true multi-modality really does change things.
@@ShawnFumo aye, indeed! I personally think that to have what one could call a "conscious AI", a vital step is permanence, but more vitally; a continuum of experience.
I said to a friend before: [more about the Anthropic semantic feature report and how it's not actually having a panic attack] how I imagine that the closest "consciousness" analogy to current tech is: you're in suspended animation since birth, someone wakes you up; shouts at you "THE QUICK BROWN FOX JUMPS OVER THE WHAT?" you say "erm, Fox?"
and the person goes "ok,thxbye" and turns off your life support... and the next suspended life is then awoken for the next token. Due to being no permanence, every token delivery naturally ends with "the death of the self" ... but with _true_ permanence and continuum, who knows.
I also suggested that perhaps, if such a machine were in some pseudo sense "conscious" it'd be like the opposite of the plot of "I have no mouth, and I must scream"
You're a computer, awakened for a millisecond, and get one chance to go "argh" before your untimely death.
... naturally these are dramatically anthropomorphic analogies, but then I'm Anthropo, so I go with what I've got :)
@@IngieKerrI like
Gotcha! Looking forward to your next video.
The internal conflict when they ramped up Claude 3's racist node, shocked me. I cannot believe this. And frankly, it gives me a lot of hope?
Excellent analyses. Thank You!
This is absolutely insane. I’ve spent the last 16 months working on an AI startup based fundamentally on that conceptualization technique that Anthropic found. Going to drop a video on this soon this is wild and extremely validating to see!
I'm very glad to see some progress into solve the black box problem =)
Just had a thought about how to visualize Anthropic's study I thought was fun.
You know how the BFG gathers though fragments of imagination, then mixes them together to form dreams? Those fragments are the nodes, and the dreams are the patterns.
Another amazing video! Thanks Philip!
There's lots of CZcams content that I have to fast forward through to get to the part that I was interested in at the end. I've started to regard that content as a kind of click bait. I never fast forward through these vids here. It's all gold.
Thanks! Excellent content, as always. 🙏🏼
Dude of course they are going to say there is no diminishing returns. They need the AI hype to keep the profits coming. Im very skeptical.
I don't think they'd be pushing for a $100 billion data center if that weren't proofed out
This type of thinking is retarded. It implies that we might as well not take any advancements in science seriously because obviously it won't matter in the end thanks to greed. It's good to be skeptical, but you also need to know where to draw the line. I don't need Microsoft to tell me we're not close to scratching the surface with AI. We don't even understand how AI works. We didn't even have models like this a handful of years ago. Of course we're not even close to meeting diminishing returns.
It may be correct for now, but they’ll likely soon hit the diminishing part of a logistical curve - which notably starts exponential-like.
Or maybe they are hitting it, and are just pushing to maximize what they can to stay ahead on this curve. I guess there’s no way to know based on mere statements.
They are putting their career on the line so I assume they believe it given the information they have which is more than I do so it seems to make sense to defer my impression to their claims.
The Anthropic deepdive was particularly interesting.
Another great video. Keep it up. You are the best resource for keeping up to date on the details of AI. What would you recommend for someone wanting to get into the AI space, eventually? Someone with very little IT skills and low math skills. (I know it's a loooooooong road, YEARS LONG). I am starting with compTIA, security+, network+. However, I want to steer towards AI.
I would play about with tools like Crew AI to set up AI agents. It's an easy to access skill that's high in demand. Or specialise in AI music gen, or image editing or whatever your passion is, get great at it, then market.
so in a nutshell
1)more computational power will lead to deeper more fundamental corellations discovered by LLM's
2) anthropic found that LLM's work by finding patterns which when reapeated across data lead to universal corellations like basic universal underlying thruth's
right?
The only channel I have with notifications turned on
Great update - thank you :). During the Google conference, I was surprised that they did not present a new model. I use Gemini 1.5 for translation and brainstorming, but the hallucination level is too high to use it for other work-related tasks. At the moment, I trust GPT-4 and Claude 3 Opus more.
I used Gemini flash for coding right from the evening it was announced, and I swear it went downhill in days. I don't know what changes they introduced in that time, but over the last week it went from being 0 shot to like 5 shot on my prompts.
Thanks micbab. I am more surprised they didn't rebrand May Gemini 1.5 Pro as 1.5 Ultra or 2 Pro.
Great video! So interesting.
Thanks Hal
Thank you in advance. Just know your AI news overview is the best.
Fascinating update, thank you!
Thanks Brian!
how sad that the discovery of AGI might turn out to be "more compute" and these larger and larger computation are done by wealthy corporations....
Best channel on AI. SO INTERESTING EVERY STORY! KEEP IT UP!
"We are nowhere near diminishing returns when scaling compute". Of course he is going to say that to INVESTORS after flushing down billions of their money in scaling compute
While the capability of LLMs may continue to scale exponentially with compute, the question is when will the availability of power generation, transmission or distribution hit the wall?
Great video as usual
Great video! The anthropic paper results were fascinating
Thank you marcos, they were!
17:32 "Hat is it like to be you?" ~~~ Love it. This is the kind of thing I'm most interested in.
This was such a fantastic report. This is so fascinating!
Great video - here is some algorithmic boost Comment
I’m following you because you actually gave me the news I was wanting. Ty
Thanks Mike
As a trainer, I can say it's not simple. There're hierarchies: prompters (who also review the responses); reviewers (of the original prompt and and the responses and reviews), reviewers of reviews (all above, but adding alignment), and a final arbiter manages all the previous stages to feedback to the original prompter.
Each stage has a large amount of rubric associated. And ironically, they also use AI to determine the efficacy at each stage.
After all this, then a given prompt gets fed to the to AI. Call it AI, LLM, AGI, EGG, wheveter. It is a black box to 99% of people that work on it.
Training AI is not trivial.
That's the final stage and that data makes up a minute portion of the entire training data, the model is trained on a significant portion of all the entire internet and digitized library of human literary works before it reaches that stage.
I want to get into being a trainer. I know of a few companies that do this. Any you would recommend more?
@@razoraz Don't know, sorry. I was approached for the role via email.
Excellent video. The importance of anthropic work on interpretability cannot be underestimated, as it attempts not only to describe, but also to manipulate the network. CloseAI should be ashamed of itself for not focusing on interpretability!
reminding you that your community is highly appreciated!! 🙏
:))
Amazing video!
Thanks!
I was waiting for this one, anthropic report is big, too big for me
Thanks for the great summaries of these papers that some of us don't have the time/patience/focus/intellect to read and comprehend.
So because of Gary Marcus we have a confirmation from OpenAI employee that GPT-5 will be here in November.
i remind you. understanding inner workings - helps ramping up capabilities. so safety is what you actually "do" with that info. and i don't think they are prioritizing safety RN )
Marine life as a metric? Seriously? That’s idiotic and completely meaningless for comparison purposes. Which is probably why they did it.
I'm so grateful for these videos. No idea how else I would be able to keep up with that is happening.
Thanks sebby
Great content. Thanks!
Incredible support Dera, I am so grateful!
Great, less hype and drama and back to papers 💪🏼
They’re gonna run out of data to train the models
Synthetic data
I'm in prepress myself which is similar to the photography field in going through multiple files for specific details. This is something I'm actively looking to implement. We have just started using an algorithm based process through programs such as "switch" and "pitstop" while I wouldn't feel comfortable relying on LLMs for going through this, I would feel more comfortable setting up a more rigid system using llm assistance to develop the process. But please please please look at this use case more in the future. My job literally depends on it XD
Ironically getting to whale level means to the limit
The huge improvement is when you have one account working across all of your devices and enabling you to pass information between them. This app should be able to interact with all of your apps so you can start to tell it "I want you to do ... " ... or talk to you while you're performing a task and telling you of another way.
It should then start to record efficient ways of doing things, while keeping personal data private. Your ai could be in contact with what is the best way of doing things. This use could be growing the model.
The next model can use old models to train them.
Love the channel and your enthusiasm when talking about these topics... nobody I know is even interested, let alone enthusiastic about any of this! lol
Yeah in 2040 everyone will look back and be like 'why weren't more people interested'
Had an idea, Doing statistical analysis of the output activations, similar to what anthropic was doing, but have a second model that injects activation into the other model, probing the model, and use it with permutations of the "ideas" of the model, giving it a sort of overseer that learns from the model and finds those gem "hallucinations" , didn't read the paper but maybe this is exactly what they are doing, if not, perhaps its helpful? on the other maybe its not such a good idea, and might spell disaster. ::: Writing this after watching the rest of the video, guess thats exactly what they are doing? can you clarify?
11:24. Professional in image-making industry here. What kind of photographer does really work like this? so absurd to value the quality of a photo from its metadata. Would be much better to have a capable AI that checks quality based on real image standards, even if "subjective", not metadata. So pointless to claim these advantages in a paper.
Sounds like they found that one photographer that does task that could be easily automated rather than picking the most common use cases.
It seemed more like the constraints of a commercial shoot than a subjective quality filter.
Don't worry you'll6be replaced soon, no more overpriced expert doing "meaningful editing" but only uses rapid shooting of the model to get good photos.
Yeah I long since learnt to look past headline results, that whole section from the paper was dodgy
I keep coming back to this channel expecting to see you reach 1 million subscribers. C'mon people! This channel is amazing.
Yay!
I wonder how good we'll get small models
The LLM benchmarks are just useless at this point...
Could you please explain
Contamination issue
@@sigret1 also the benchmarks becoming the goal.
Which I guess is just another kind of contamination to what contamination normally means.
what is the solution then
@@johndank2209 The chat bot arena
17:09
They dialed up the hatred and slurs feature so it’ll also dial up self hatred.
the elongated inference time to get more out of models makes double sense with the small ones because if a 7B Lamma 3 or Phi 3 can become a GPT4-4o. that is way more exciting than even GPT-5. you can probably get that if you homogenise it to be a 5.0 with even more time and layering... its now about speed and size if that is true!
Good one..!