ChatGPT Is a Dead End
Vložit
- čas přidán 19. 06. 2024
- ChatGPT can only go so far, let's talk about the limitations of LLMs and topics like grounding and exploration that are important research directions for future work.
GTC Signup: nvda.ws/408jS7w
Raffle Signup: forms.gle/NyxKRw2tF6Un9CaT6
Sessions I'm attending:
- bit.ly/nvidia-science
- bit.ly/nvidia-real-world-rl
Outline
0:00 - Intro
0:48 - GPU Giveaway
1:06 - Human-Level AI
2:38 - My Thoughts on ChatGPT
3:52 - ChatGPT a Dead End
5:23 - Path Forward - AI Feedback
8:17 - Path Forward - Grounding
9:11 - Beyond LLMs
10:23 - GTC & Giveaway
11:37 - Outro
Social Media
CZcams - / edanmeyer
Twitter - / ejmejm1
Sources:
ChatGPT: openai.com/blog/chatgpt
Anthropic AI Paper: www.anthropic.com/constitutio... - Věda a technologie
The thing about LLMs and projects like Gato is that what they've learned is a semantic prior over sequences. It's a way to learn a sort of common sense. What we do with that going forward will define the next 10 years of AI research. These self-supervised foundation models are going to bootstrap some amazing things, by virtue of exposing a semantic space to build on top of. This sort of thing can enable very efficient learning, of the form that we've always had in mind when thinking about AGI. But of course the prior is not the end, it's just the beginning.
*An addendum:*
Some things I didn't quite get to in the video:
You could theoretically ground a text environment's feedback if it had some source of truth. As someone in the comments pointed out, a verifiable task like programming could work, or perhaps some sort of multi-agent setup could work. That being said, I think this path is a lot less obvious and potentially would lack many important things you can learn from the real world.
Also there has been some good work recently on grounding language models that is promising. I didn't mention it in the video because I wanted to focus on the issue with pure language models, but if you're curious you can Google "grounded language models".
Also let me know what you think of this format. It's different from what I normally do, but I quite liked it!
I basically came here to say that if reading your CZcams comments is in any way grounding for you then apparently there must be grounding going on in linguistic activities. Scaling the feedback of chatgpt will probably be easier than it looks, because instead of using thumbs up down you can have the model observe wether people try to get better answers by rephrasing their questions or wether they move on to new subjects.but you already realized yourself that treating language as ungrounded simply because it is language is too simplistic. And you probably came to that conclusion after being engaged in linguistic activities and not by observing new facts that you uncovered directly by yourself. Which means we humans can also acquire new knowledge just by observing linguistic discourses.
I really liked your video. Particularly you good demeanor. Embodiment seems positive, but I'm not sure it really addresses the issues on learning. I would even go beyond leaning, and call it insight. You clearly demonstrate insight when you took time to bring up this subject. That's a really intriguing quality of humans. Because you see humans exhibiting good insight in some areas but complete absence of in others. And you could have a paraplegic from birth be a scientist. Insight is also interesting in that it may have different levels. You may instantly respond to a question based on previous insight. And you may just be problem solving, while still applying insight. Or like you actively constructing a representation. Just the emergent features of LLMs show how much is there to learn. My guess is that there are many more paths to AGI than we would expect. It is an interesting time.
We’re on the same wavelength! My research is all about equipping RL agents with LLMs to try to get them to coordinate their efforts in English. When they are invested in the world around them, their language should be grounded as well.
Very cool research topic.
It's weird to think of language as existing in a vacuum, but this is effectively what LLM's do. They have no sensory data from the world, so the language they read / predict is disassociated from the things that language represents. I think this is ultimately what people are talking about when they say that LLM's are "bullshit generators".
I agree. Learning is about formulating conjectural sentences that 1) say something about the world (example: “if I release an apple, it’ll move in an upward direction,” 2) comparing such claims to what actually happens in the world (example: we see what happens if John releases an apple), and 3) revising the sentences in light of our knowledge of the world (example: we give up the sentence about apples moving in an upward direction). Now, If if AI has no “ontology,” if it has no way of thinking and talking about the world outside of words, if all it sees and pushes around are words, then it cannot deal with meaning and it cannot learn.
That said, I don’t think it’s necessary to give an AI a body and sensors and have it move around in the world, even though that would be a useful experiment. When I watch a video or read a journalist’s narrative, I know what the actions are and what they mean even though I am not experiencing them. The reason is that I have notions about the world, what things are out there, and how they behave that are not linguistic, even though I may use language to express them. Perhaps one way to approach this would be to create artificial languages (like in physics) with their own ontologies for precise conversations about the world with rules about how truth claims can be revised in light of other information. Then one could create an interpretive bridge between natural language and the formalism. The calculation of truth claims and assessments of fact take place within the formalism, and the conclusions are then translated into natural language.
Some thoughts:
I think most people who understand the technology, understand how limited it is.
Yet, I do think scaling might really make it a lot (and I mean, a lot) better than what it currently is. Not Human level intelligence, of course, but much much better than current outputs.
The main problem seems to be how much contextual information can actually be held in storage. For example: If someone wanted to write a large fiction book using just AI (with minimal editing), they couldn't - not because the current AI gives bad data, but because it would be impossible to have an AI remember enough information about the background of the fictional world, characters, prior chapters, etc. to actually give you coherent output for an entire book. So, the limit here is not in the model itself, but in how much data can be stored.
Ironically, the heavy censorship on ChatGPT gives away how powerful contextual info actually is (after all, basically all they do to censor the AI, is giving the AI a large block of pre-written info before the prompt of the user - the "jailbreaks" for chatGPT to avoid censorship also work in the same way, putting a large "conditional" before any further prompts, making the AI generate text that disregards the prior censorship). If someone were allowed to have the AI consider a massive chunk of info as context before their prompts, even current AI models would be a lot more useful.
The "allow the AI to search the internet" is basically just doing the same: attaching additional contextual information to a prompt.
Part of that issue, of course, is how much computing / RAM it takes to actually run these models. You basically need a server right now, so people are dependent on the honestly downright pathetic service OpenAI etc. give you as a single user.
For image generation, the revolution didnt start with DALLE - it started with StableDiffusion, because that allowed users (and tons of various competing companies) to actually run image generation (and even model creation) on their own devices, scale it up however they want, without censorship or restrictions. I suspect the same will be true to text generation or other tasks: when computing power increases, and some highly-efficient public model is released, AI will massively improve the lives of many people. Before then, it's a neat tool, but highly limited in scale and scope.
"but because it would be impossible to have an AI remember enough information about the background of the fictional world, characters, prior chapters, etc. to actually give you coherent output for an entire book."
I have to disagree. If you were to break the book down into chapters while briefly summarising previous chapters. You could get closer and closer to a model that could write a whole book. Even when we write large books or essays, we don't remember every single piece of information we wrote down, but we remember the key takeaways and summarise it in our memory.
To your point about the AI not being able to remember an entire book it's writing, I disagree. That is a current limitation with interacting with ChatGP. Keep in mind ChatGPT has memorized an immense amount of human knowledge. I think for writing a full novel, you would have to train the model on what its written so far so it would know the book already within its weights instead of just its field of attention within the prompt. A different workflow than current ChatGPT but still quite doable and in the near future.
@@Jack-gl2xw My text actually agrees with you. As I said, just improvements on current language models could make them so much better and more capable.
Its not currently possible, though, which is why I cited it as an example for current limits - for example, due to what you and me both said: chatGPT and other providers are severely limiting what a user can do, and the hardware required to run, let alone train, a text AI model requires large server infrastructure right now. When that changes, and a single user is able train a custom model extension, then the capabilities of current language models will vastly increase.
@@TexanMiror2 it's already vastly increased with GPT4. It can now consider 32k tokens or 50 pages of text
@@absta1995 Thats good to know, thanks!
Great video! I appreciate the thought you put into it. I agree with the general thesis, but I'm curious about these points:
What did you mean when you said that LLMs that retrieve information from the web can still only provide information from the internet? How unique does the information have to be? Most people only say things that they've heard before.
"By design, it's limited to the text on the internet." Sure, but that's a lot of text! It's certainly way more than any human will ever experience in their life.
If what you mean is that the models need to be able to learn from more than just text, then I agree. The human brain has specific regions for different modelling purposes. Language is understood in Wernicke's area, produced in Broca's area, and prompted by the prefrontal cortex. We can think of the former two areas as roughly equivalent to the LLMs we have today. The other equivalent regions that we need for AGI will be a visual cortex, a somatosensory cortex, and the prefrontal cortex for decision-making.
With that in mind, I agree that LLMs are not sufficient for AGI, but they are an essential building block if we want to mimic human intelligence.
The point of the web retrieval part was that beyond a certain extent, LLMs are not going to conjure up new information that you can't find on the internet. Even with that limitation they are still incredibly powerful, but that is a limitation that doesn't apply to humans. Even if most things humans say are not "new" knowledge, they have the capability for it, and that's what drives science/technology further.
@@EdanMeyer Yeah maybe but I still think that's up for grabs. For example, an advanced LLM model in the next few years could look up current research on a topic and write comprehensive review papers. I think that's within the realm of text + prediction.
I would argue that really good review papers provide 'new' value (assuming they include analysis). I guess you would argue that a review paper is just an amalgamation of known information. But I think if the review paper includes opinions about what the current research means or could point to, that would be new information that you can't just search up on the internet.
But I think we need to define what we mean by new information. Would you consider a unique drawing (created with generative AI) new? Or is that just an amalgamation of what you can find on the internet?
Although, I agree with most of your statements I also believe there is room for improvement in understanding what is Artificial General Intelligence and what is human level intelligence. When you break a part the human brain you realize its not just one huge junk of neurons. Different parts have different task and substituting one part as a filler for another just wont work. 1 possible way of achieving AGI is having a Frankenstein Model and instead of treating every single model as an individual we bring them all together and make them a part of something bigger. The second one is the one that requires innovation that hasn’t been discovered yet. And this is the reason why people keep saying AGI won’t be possible anytime soon.
But until we achieve proper AGI I’ll be using “chadGPT” to write me boilerplate code.
I thought you all should know: the closed caption AI generated this;
"... Human level AI will not come from Chachi BT...'
We should also let Chachi BT know, in case it thinks it's sentient.
Sorry Chachi BT
In Hindi, Chachi means wife of brother of one’s father
This is an incredibly important perspective, thank you for the time in processing and communicating it! I am very positive you're onto something and I very much look forward to keeping pace with your own contributions in this area of innovation. Keep at it!
I think even human level AGI will need a certain amount of human intervention during training. If not for the learning itself, then at least for the purpose of alignment.
Something like cooperative inverse reinforcement learning might allow human feedback to be incorporated in a more scalable way.
Absolutely, if we want it to work with/for humans, humans are going to need to be part of that process
Alignment is absolutely impossible.
@@tyrjilvincef9507 Why is it imposible? If we can align humans, it should be possible to align human-trained AGI
There are still a plethora of tasks LLMs can perform and their is still room for them to improve. While I do agree on the fact that it won't bring us AGI. I think it will be highlighted in the history books as a stepping stone towards it, purely for the insane increase in the amount of spending that is going into AI R&D across the globe.
Definitely, it's a crazy time we live in
I agree with you. Here's my issue with LLMs: They are models of language. Yet, language itself is only a model of reality, created by humans for the purpose of communication. LLMs will only ever get better at approximating what is already a model. And language, as a model of reality, is not powerful enough to express any concepts that humans are capable of conceiving. There is a vast amount of concepts that humans do not need to communicate, as they are universally perceived and understood by any humans, due to our similar biological functions, sensory organs and environment. So language is only a model for information communicated between humans, and not for all information processed by humans. Thinking that all this information could be expressed using language would even be considered absurd, without touching the topic of LLMs at all.
The AI could be given the ability to search the internet, be instructed to find quality training data, and then be trained on the data it finds. That would be the AI training itself and learning new things under its own directive. And not only that, the AI could be instructed to create experiments and then humans would execute those experiments since the AI doesn't have a body. The AI would then be trained on the results of this experiment, meaning it actually learns based on its own directive not just on things humans have found, but things it itself has found.
I keep saying this for ages. LLMs are just that Language Models...LARGE ones. Intelligence predates language. Cats are intelligent, dogs also, some animals show abstract reasoning. Abstraction is a condition for language not a consequence. What LLMs lack is a model of the world that is continuously updated by experience. If you make sitational questions for LLMs they fail miserably (A is on the left of B, all in the corners of a room, etc).
IA today is passive: it can't explore the world an test and try it and it will never be able to do that without an internal model of the world that is constantly update by experience. This is intelligence: that capacity to learn and generalize experiences with a *model of the world* , a multi level model of the world with capacity of abstraction: capable to separate properties of objects from objects (consider the texture of a leaf independent of the leaf the color independent of the leaf and generalize this concepts to other objects. It is particularly compelling to study intelligence from the evolutive point of view: why it happened and how. Rodolfo llnás (a researcher that studies the Cerebellum and created a model of it) believes that it evolved because living things that move and need this model to navigate the world and its dangers. This is backed by some beings that have a movable period o their lives and then become sessile organisms and lose their brains after that. this may seem a bias as the cerebellum that llnás brilliantly studied is related to movement, but makes a lot o sense.
The claims of sparks of GAI (or whatever the term they used) from the current owners of AI are simply preposterous.
But the hype is too loud for anyone to hear. Even Geoffrey Hinton said similar things (although he has some self conflicting declarations). There are some dissonant voices in the choir, a lot in fact but media don't like headlines like "hype unfounded" or "this won't be done with that".
Now I will watch your video I needed to take this out of my chest....again.
If GPT doesn´t know something you ask it will invent an answer. That alone is terribly dangerous
That description is, itself, misleading. GPT does not know anything at all except the probability of certain tokens following certain other tokens and therefore it either NEVER invents answers or else ALWAYS invents answers depending on what you mean by "invent". There's a recent video from computerphile showing what happens when there is little or no training data for an input token but, from the point of view of how GPT works, it is no different to any other token, the probabilities for succesors are just lower. And note that these probabilities are not probabilities that any given succesor token is the "correct" one, only that it was the most common in the training set so we can't use this as a measure of "certainty" or groundedness in the answer.
@@SmileyEmoji42 I´m sorry Nick but we are talking about here how effective and reliable the model is for the real world and the real/final users. And if the model is giving completely wrong answers to some specific questions (as had happened to me many times) it doesn´t matter what its inner working details are. It´s still wrong
@@mauricioalfaro9406 My point was just that, if you give a misleading explanation of why LLMs sometimes produce incorrect outputs then people tend to propose unworkable fixes based on that misunderstanding. E.g. If it were true that it sometimes just "invented stuff" then you might reasonably propose that developers add code to say "I don't know" instead but that is not how it works.
Seems to me that a chat AI could formulate hypothesis about human responses and test them in the population, then use that in the training model.
This will be hard to implement until someone comes up with version of this models that include structural inhibitory responses.
To me that is the main missing piece, straight up hallucinations is an unstable ground for learning.
Excellent video. I'm also skeptical whether the current model of ANNs even **can** result in human-level AI. It certainly seems to me that the differences between artificial neurons (high precision, synchronous) and biological neurons (low precision, asynchronous, multiple forms of encoding information) might be a critical factor in ability to achieve genuine intelligence (and not the mere mimicry of intelligence. If nothing else, the poor energy- and data-efficiency of ANNs might place a practical limit on their ability to produce human-level AI, where alternative types such as SNNs promise potentially much greater energy- and data-efficiency (thanks to being closer to biological NNs, which are indeed incredibly energy- and data-efficient).
I would argue a lot of the advancement on a fundamental level will come from specialising the artificial neurons to be more "compute optimal". For starters, how much precision on the floating point operations is needed? After all, NN aren't a precise science, rougher calculations could give the same results/better results with fewer/the same compute volume.
That's not all: which activation function has the best neural complexity per compute time performance? Will back propagation be optimal for large scale continuous learning?
And that just the software, hardware will be way more specialised for AI in the future, which will further increase computational efficiency.
The main advantage of ANNs over brains is just the speed at which it functions. The brains neurons fire on average about once per second, meaning one hertz. Meanwhile modern computer chips have cycles in the gigahertz range. Even if that isn't a one to one conversion into the speed of an ANN, it's many, many times faster than a human brain.
I think the philosophy behind ANN achieving AI is based on the universal approximation theorem. That is, artificial neural networks can approximate any function given the appropriate weights and architecture. Thus the logic is that neural networks can be constructed that approximate whatever the human brain is doing. Indeed biological neural networks seem much more efficient energy wise, however a lot of research labs believe that these artificial neural networks are enough (such as OpenAI from my understanding)
@@kandoit140 any turing complete language, such as assemby, can implement any program. But global, complex computer systems have not been developed in assembly, and compiler and processor optimization has continued to be a major driver of software advances.
Accordingly, only when currently slow and costly training and runtime for reference tasks become fast and trivial (energy-wise, compute-wise) will there really be a path to complex intelligent systems in non-datacenter environments in a way that transform society the way personal computing revolution did.
Current LLMs are pretrained on sequences of language embeddings, but we could just add embeddings of images and learn the sequences of images of all youtube videos. Thus we combine the real world intuition provided by videos with the theoretical knowledge of language.
I'd like to see more about how knowledge graphs (semantic networks) can be used in combination with ai, cause this could lead to quite interesting concepts
on the same page! while watching this video i got an idea for yet another potential configuration...
starting with the concept of a loop that generates a LLM prompt containing/based on "state, action, reward"...
log each input/output/score, if the agent previously encountered the current combination of (relevant) states & generated a successful action, repeat the action without prompting the LLM unless it failed. nearest neighbor or RL?
so this config would explore actions via LLM & information lookup, then a model would build up past scenarios to know what actions to take, even if the action space is choosing between several models/prompts to pass values to.
7:50
Gpt 4 multimodal and has access to plugins.
It can write a program that visualizes chess board positions, that it then uses as a plugin to visualize board states.
Similarly it can use a plugin to get stockfish position evaluation and next best chess move predictions and use those to learn.
Makes sense, RLHF makes existing models more useful to human use but the underlying systems that the model 'learned' during training have not changed much. It does not know when to distinguish when it actually has all the information to answer a question or to try to reason out the the answer when it doesn't. It just puts out the most probable answer even if it is completely wrong (and openAI training it to just say 'it cant do X' is an overly large band-aid since now it will start saying it can't do something when in other contexts or even on a different chat it clearly can).
do you still think AGI is 10 + years away? Given the recent development in AI?
Great video. Recently have been getting interested in AI. Curious about your thoughts on neurosymbolic systems? Would be cool if you made a video on that. Heard of it from a video of Noam Chomsky and Gary Marcus discussing about how shallow Chat GPT really is
Do you think the issue is mainly the lack of grounding towards objectives we care about or the text only nature of ChatGPT?
In case we are really talking about the goal of unsupervised learning on sequences of any kind of data not being enough with respect to grounding, I think that the capability for simulators to learn how agents behave internally will be enough to turn any large model into an agent that acts into the world like an AGI.
As models get larger they learn more and more in a single backward pass: one could take a prediction model like GPT that can handle any kind of input, ask it to simulate an agent that does something, and feed back into training all the new experiences it observes. With a very long context length the model could also meta-learn a lot of new tasks it observes, in the worst case one could ground the model towards an objective by describing manually what happened outside the environment, until it figures out how to do that on its own.
I disagree that grounding requires non-language because in the end language is representation. Minecraft runs on the minecraft protocol which is a language, whatever you attach to the sensor of a robot the sensor will report data in a language. We already see ChatGPT succeed in converting structured data and natural language back and forth.
While I agree grounding is a problem, I feel like it is similarly asking for something superhuman of the AI. After all, humans can become conspiracy theorists too when they self-reinforce with no grounding.
To improve grounding, humans develop new languages which focus on consistency rather than expressiveness (aka maths). ChatGPT able to do short maths but not long ones reminds me of how humans can do short ones by memory or heart but long ones only with paper support. System 1 vs System 2 as formulated in Thinking Fast and Slow. So somehow learning consistency or verifying it with an integrated logic system is key imo.
Maybe one problem is that LLM are not capable of executing computation. I guess if you ask ChatGPT, it could give you a more or less working source code for a very simple chess player, but it would not be capable to execute that. Maybe we should give our LLMs the ability to execute some code in a very restricted VM?
You might be on to something
i wonder if that would make backpropagation more difficult?
totally agree, the path to getting beyond mimicking the average internet user is very unclear. I wonder, assuming people will use LLMs more and more for "search", how will we organize new information in a way that it gets through to the LLMs without getting drowned out by the (false but easy to generate) outputs from the existing models? One risk is that models get "stuck" with whatever the state of knowledge was when the first LLMs were trained with no real way to make significant changes 😅
(nice horse btw!)
Hello man, I love your videos. I just wanted to say you should update the anthropic link in the description, its broken. Other than that this is an awesome video
What would happen if we created a model with a latent space where the tokens are actually higher level "ideas". I.e. if currently tokens are (more or less) ~1 word, what if we had tokens that could be a sentence that represent an idea?
In theory that is what the mid-level layers of neural networks are doing
Two days agao the palm-e paper came out, multimodel training for a pretrained LLM plus robotic embodiment.
Personally, I think the main issue is still the lack of an abstract memory that a model can reason over until it decides to produce an outcome. Something liek the Differntiable Neural Computer. The Compressive Transformer architecture does something in the right direction, but it does not allow a repeated reasoning loop, iirc.
Great video, thanks. Well thought out, and well communicated.
Microsoft are already starting to ground their A.I (ChatGPT) in the real world, this paper --> titled "ChatGPT for Robotics: Design Principles and Model Abilities," authored by Sai Vemprala, Rogerio Bonatti, addresses that by using language models directing a robot to complete tasks in the real world like getting a can of Pepsi from a drawer or using tools like cooking tools.
The trouble with grounding such systems in the real world is that it (a) it is unsafe - You can't just have it try shooting someone and then mark that as wrong and (b) It doesn't scale - The real world is too slow for the 1000s of learning events that you need even with human reinforcement.
It will probably work for a few specific actions in a controlled environment but we're not going to get to AGI that way.
Does it have a model of the world ?
What do you think about palm-e where it showed positive transfer?
Has anyone experimented with letting an RL agent request feedback? It would be different than humans providing feedback as that occurs only in context of refining a previous request. However, it would be interesting (to me at least) for the agent/LLM to be able to self-identify where its knowledge is uncertain or where it needs grounding.
The solution sounds a lot like Rich Sutton's Alberta Plan, or Yann LeCun's JEPA. Both include a world model and an actor-critic, if I remember correctly.
So they simulate how their actions affect the world-state, and choose the actions which lead closest to the goal state. Then if the state changes differently to their simulation, they learn from that mistake.
It seems that LLMs have more and more emergent properties as we scale them up (data and parameters) such as ability to count, or theory of mind. The question is where does it stop? It could well be that it never stops, or it stops well past the AGI level (which would be my guess)
With that said, I do intuitively like the idea of embodiment. I would define it as follows: you need sensory input, model outputs and an environment, and embodiment is simply the loop between output, environment and input (very much like in RL). I'm not sure if the environment has to necessarily be distinct to the model (such as in the Ha world-models or the Hafner dreamer)
I do agree that having at some point a symbolic representation of rules, concepts or mechanics seems important (but the LLM already has symbolic tokens), so not quite sure it needs to be explicitly baked-in
Overall I think scaling up LLMs will continue to blow our minds
I agree that scaling LLM results in better models, but what does that mean exactly? it simply means more parameters to store more information. This does not give the model more capacity (such as calculating things etc...). This is due to the causal nature of such information which cannot be encoded in LLM, as they only pick up associations. For example, there was a paper where scientists tried to make LLM reason about math, which they did by simply fine-tuning on a math corpus (where is the causal information of math encoded in the model?). They did not get 100% accuracy on simple math calculations which is pathetic if we want to call it "intelligence" in my mind.
I think that nothing meaningful can be obtained by not including a causal part in a model.
Nice video, looking forward to the GTC.
I had similar thoughts about this and was actually listening to the Making Sense podcast by Sam Harris where he talks with Stuart Russel (the guy who literally wrote the book on Machine Learning). You should give that a listen, I'd love to hear your thoughts on their conversation.
Great video. Although I find it quite sad to see how many people overlook the fact that LLM (NN in general) are simply mirroring the training data and do not actually have a causal representation. Any application including LLM is a guaranteed failure and people do not realize that.
You should do a search for othello-gpt or check out the paper "Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task". They teach a transformer by passing it legal moves in the game othello and using probes determines that t he model actually creates the board state within it's internal neuron outputs. So LLM's can model things simply by predicting next word. What's lacking IMO is true long term memory. They're basically reincarnations of the main character in the movie "Momento" who is someone who lost his long term memory and unable to remember anything beyond 5 minutes ago.
I know very little about ML, but what would happen if a future hypothetical LLM were able to add a new attention head to itself before learning about topic [x], and then fine tune itself with the newly added head by e.g. reading all the research papers about topic [x]?
There isn’t an obvious reason to do this. If you were to add a component layer into an NN, it would affect everything that comes after it and those component layers would need to retrain as well. Adding new layers makes sense for things like transfer learning where you want to take a model that’s done well for one task and so you add a layer or two so it can fine-tune itself to your task. But that’s not the same as a persistent model that constantly changes itself. It would make more sense to just give the model a lot of architectural capacity (i.e., as many attention heads) as you can afford computationally-and financially if we’re thinking energy costs from training. If you do this, the model will learn which attention heads are relevant for forward passes on its own. Adding more to the architecture isn’t really going to solve this issue. It can help with fine-tuning something to a domain (for instance, taking an LLM and adding layers to it so it’s fine-tuned it to a specific domain of knowledge) is a reasonable motivation for this. But for a general purpose model it doesn’t make much sense.
@@Bloooo95 thanks for this - fascinating read!
@@xntumrfo9ivrnwf No problem! It’s also important to note that LLMs in general are trained to produce/generate “original” text that reads as legible language. That’s it. That’s its goal. LLMs like GPT-3 and ChatGPT work so well because they are *very* large models (so they have a lot of learning capacity to learn complex relationships) and they were given a lot of data to learn on. They were effectively given a massive corpus scraped from the entire Internet. Of course, OpenAI can’t go through that corpus and label the veracity of all that data. It’s simply too much data to churn through. So these LLMs can’t really be trained on the idea of what is truthful information nor are they expected to. They just have to produce text that sounds like legible language and that’s it.
For me, the most obvious drawback of current LLMs is a limited context window of transformers, RNNs with attention have a much more similar "information flow" than the current transformer models.
7:14 Couldn't the model use hermeneutic exegesis as a means of grounding itself in a perspective derived from logic with the understanding that words are derived from symbols which represent conceptual ideas.
Create an agent that knows nothing about the world, but can receive audio and video inputs. It is capable of moving through its environment and learning what objects are by pointing a laser light at them and asking for an audio input with a [*ding*] or some other cue.
This is intended to simulate the way a child would point at an object and a parent would explain what it is. It creates both "visual" and "sound" memories for the model to use to understand the world. Heat and pressure sensors could probably be used to mimic a rudimentary sense of touch, increasing its dimension of understanding the world.
Intuitively, I mostly agree that more intelligent AI systems will likely need some grounding. However there are some fields where text might be sufficient for a complete description. For example mathematics or philosophy.
A human in a cave could try to advance theoretical mathematics. The only thing in mathematics (I know of) that is grounded in the real world are a few very basic axioms. But those could easily be included in the initial dataset.
But I'm also not sure which abilities would help a human avoid mistakes in mathematics, that a LLM couldn't have. From what I can tell, new discoveries in mathematics are just creative combinations of previously known truths following a few very simple rules.
So maybe it's possible for a LLM to learn mathematics on it's own?
i think chess is an interesting example, because it can be played completely in your mind and it *does* know the rules and theoretically has all the info it needs to play correctly, the problem is for it to access and apply that knowledge.
once it follows the rules reliably it should be easy to have it learn from playing against itself. however how would you make it check itself for following the rules? it might be possible to set up an adversarial training where one side has to provide convincing arguments that a move is valid or not, but that would probably just train one half to be more convincing and the other side to ignore all arguments.
Have you seen the recent PaLM-E paper? Do you have any thoughts on it when pertaining to this topic?
Yeah, came out when I was recording this lol, will check it out sometime this week
@Edan Meyer PaLM-E could be a great candidate for your next video. I wonder if multimodal models such as Microsoft's Kosmos would count as some grounding too (between different modalities)
@Antonio Flores Montoya Its on the way
This is also an argument for "embodiment", by a lot of AI researchers. RL is kind of this way. The structure is, "agent" in an "environment". Though RL is incredibly sample inefficient, compared to a human infant.
My personal definition of AGI is an AI that can perform better than humans on any given specialised field. To add more context, an AI that can perform better than the average human that works within that specific field. A field could be like maths, all sciences, all economics etc. My definition isn't specifically interacting with the physical world but its intelligence in any given field is above average (To give a specific answer, it can answer any question or analyse any test results better than the average human doctor), which I think is entirely possible with these types of LLM's.
9:07 a verifiable environment like code works
Good point, I was trying to think if there were any good text-only envs, and now that you mention it this might fit the bill
I'm sorry but after working retail, I do not think most people have the ability to learn without supervision.
lmao
Great video, I love it. I disagree on the need to ground learning to some source of truth outside of language. Why couldn't you just use wikipedia? For instance, imagine the agent was exposed to running around and witnessing things outside. Say it generated text summaries of what it saw and grounded its learning in those memories. How is that different than grounding its learning in wikipedia articles? I would however say that knowledge of physics gained from our senses form the basic units of meaning for us that allow us to learn more advanced concepts out of those. Maybe that is what you mean.
100% agree, helpful but it cant invent something new. I absolutely like to compare current large language models to just be very good lossy compression algorithms. For example if you have a raw image (the input data the ai learns) , it can only output a jpeg because of the decay in the network. Therefore i say we only get General AI if we find a lossless algorithm that provides unlimited compression on random data (everyone says its impossible, i say we just have the compute capability today to do it).
I don't know where you're getting this link to compression, but unlimited compression is easily disprovable - To take an extreme example, you can't compress 257 different files into a single byte each because one byte can only have one of 256 values.
@@SmileyEmoji42 Did i ever claim to compress it to a single byte? There would be a minimum size lets say 248 bytes. 248^256 is more than enough to store anything there ever is and ever will be. My claim was NEVER compressing 1 byte to 1 bit or 256 byte to 1 byte....
@@snapo1750 A complete explanation of information theory is too long for a comment but, in essence, compression involves a trade between size of lookup table versus effciency of compression e.g. You COULD compress all known documents to the size you propose BUT only by using a lookup table that was the size of all those documents and that still doesn't help with future documents. Inormation theory has proved that (statistically) random data is inherently uncompressible.
@@SmileyEmoji42 I also do not disagree with a lookup table has to be the same size or even bigger. what i am talking about is dynamically computing a lookup table that results in the through compute in the original file. lets make a simple example assume pi is infinite, this would mean every document there is and ever will be can be computed through pi. (Maybe a stupid example but its a point i make). doing it with pi is not feasible as the compute exceeds the compression. My whole point is creating the lookup table through an algorythm dynamically (how ever that would look like). There has to be something that humanity misses. Therefore i agree with your statement, but your assumption that i store the lookup table is wrong.
I think not enough people are asking is AGI actually worth while?
No one said that LLMs are AGI or will become so. Its looks like though that LLM can be a key component of that. Thats all.
Maybe just the interface to put things in words
I disagree because I think you've missed a couple of possibility. 1. For 'world' exploration, they could play in a text-version of minecraft etc. Look at the old text-based versions of MUDs. For example, you could create a text-based world where you have multiple agents that need to word together to farm and trade. They need to communicate with other, built trust, betray, harvest together, and so on. Have evolutionary pressures to learn such things.
2. Math. Math can be a grounding for a lot of things. Reward agents for learning new math, exploring new math, etc. A lot of math is hard to create but easy to validate.
Future AGIs will be "grounded" in human languages by LLMs. "Dead End" isn't the term I'd use.
Hold up, what if you made a model that had a Minecraft account and you had it interact with real players on a live server and they can talk through the chat bar and collaborate to complete goals… this would be hard to implement but interesting idea
Those behaviors can also be learned behaviors and we can even improve our ways of teaching ourselves things, I feel like an AI should have many different components each very capable like our brain has many different components that serve many different purposes like for imagination memory and image processing and whatever even though most of it can be somewhat interchangeable but I think in AI should be the same way that should have many different neural networks or programs that can do many different things that together would form an AI so you get a network that could also do speech recognition image recognition they could also be a language model and all are interconnected and work together! We also need to change the way that an AI learns since back propagation will not work forever and our current AI is only work off of a very very old model of the brain of how he thought it used to work but that is now very outdated and a traditional AI can't even think they cannot even have a logical thought they are just simply complex memory banks I take an input and give a related output all they're doing is just adapting and AI doesn't learn It just adapts right now they're technically isn't such thing as a machine learning unless that machine has a logical thought or process to back it up or build off of it.
came for the AI, stayed for the horsey
David Shapiro has a good take on this concept with his RAVEN project.
If you create an AI that doesn't take human feedback and it only takes sticking to principles, to "improve itself", you'll create, invariably, an ethical monster.
Well, we can put that AI into a world with other AIs..
Didn't Tay become a monster due to human contact?
@@v-ba SAO season 3, 4 moment
In the same way, if you create AI that _only_ takes human feedback, you create something equally disturbing.
@@_RMSG_ Everyone does this with their kids. It depends on the feedback and 'upbringing'. Human feedback does not automatically lead to something equally disturbing.
Interesting point of view but I guess LLM already exploring quite a lot of those idea and start not being focus only on language, but sound and vision. Research and test have already proven that Multimodal model have better accuracy! AWS, OpenAI, Microsoft, Meta, … are all going this path and seem even the next GPT model might be a multimodal one.
Researchers from Meta recently released a paper about ToolFormer or how AI can acquire new skills.
Are things going to stop there! Of course no. Does that mean GPT like model or other LLM are dead end? Definitely not, they will evolve in different directions. We are just at the beginning of the journey and this technology is at his infancy stage. Just wait the end of the year and maybe come again on this though, ok 👍
Share quite the same intuition, interraction with reality is key !
tesla may currently have the best shot if its the case that you need a physical robot with senses. They have the capital and engineers to make thousands of robots with the training compute and battery built into the torso.
Humans often improve by contacting other humans and asking them questions. That may be part of what we need.
THANK YOU
Why do you cut the video so much?
a video of GPT-4 please
What about Google PaLM-E?
Came out the day I recorded this lol
There are some works like PaLM-E that work on grounding LLMs and I think it's a promising direction
If LLMs are already capable of uplifting everyone from poverty, would it be smarter to just focus on LLM while we solve for the ai alignment problem before we really pursue AGI?
In nature, the development of emotion and "unique" perspective motivation appears to take a mother and then the entailments of social indoctrination. I'm not sure we should be so hellbent on creating computer network based conscious entities with independent will.
I think that trying to find a universal metric of fact, which is needed for a generic ai, is bound to fail, facts are social consensus based on reasonable interpretation of evidence, and they are constantly revised when new discovery get added, then filtered through ideological, political and moral lenses. And even then, evidences are never clear cut. That mean it's not on fact we need to ground the ai, but on social inference, else the ai might conclude very logical and factual solution, like extermination of human like in the paperclip problem. Which mean principle, like the 3 laws of robotic, aka the alignment problem, might be more important than factually grounding the ai, such as it can reason based on evidences and principles. Problem is human never agreed on fact or principles to follow. By nature we need ai to be universally moral, which is an unsolvable problem in the absolute, which lead to ai being partisan by default. As a minority, most llm assume I'm not using it, if I ask for help on cultural knowledge beyond the dominant culture, the ai I should not do that to not offend myself, like asking for proposition of name that aren't occidentale in nature. Which mean it's only good at operating on occidentale version of facts bu nature and everything else, any other perspective is erased, that's pretty political as a design.
P =/= NP … what else is there to talk about?
After hearing about that free GPU, it got really hard to focus. Hey Eden, I have a laptop without any GPU and I have been wanting to buy one for years now but as fate would have it, I am not able to save enough to buy and money always end being used for one need or other of my family members. Can I please please have that GPU? Thanks, Rajat.
Its kind of anoying how many people lose all conection to rationality when it comws to ai
Wow, thanks for saying this. I hate that ChatGPT is confused with an actual reasoning engine
Eh. Read the article by Stephen Wolfram.
Emergent features are real.
Read the PaLM paper.
@StefanReich how do you define "actual reasoning"?
The example with chess is not 100% correct since if some human is not familiar with chess and does not know how to play, it will never learn it in a self-supervised manner. In such a case, one needs a teacher.
Yes, it assumes a basic understanding of the rules
@@EdanMeyer exactly. And this is why it is a bit unfair to say that if llm cannot teach itself to play chess then it is an evidence that it is not able to learn like humans do. Indeed, I am agree with you that it is unable, just think that there exist more proper example than chess.
@@EdanMeyer from the other hand, it is possible that I didn't get your point correctly. Sorry if so.
ada by deepmind would meet your definition
It's certainly in the right direction and I think, but it's capacity and learning speed are still far from human level
I guess I didn't specify that in the video, but even once we have the right general frameworks, we will need to make them efficient enough
langchain enters the room
OpenAI talking about AGI... when NPC dialogues in games are dtill based on trees...
Oh dear
Ability to identify truth from falsehood.
I believe there is a way to get way above agi by continuing current path. If you think about it, predicting the next word is pretty much "going with the flow" or improvising, or thinking but in real time. The hard part is the fact that it goes step by step the hard part is asking the right questions. But could be possible by forcing the llm to answer a bunch of questions. Reflecting on that. Asking the llm what the next questions are and so on. The compute is already pretty inexpensive so you could make it reflect a whole damn lot for pretty cheap actually. You can even ask it to fact check himself on the web
I believe RL with AI feedback that "thinks really hard" could actually scale pretty far.
The second point there is to it is that the conversations we have with bing and chatGPT actually helps him get feedback and sharpen it's reasoning. If you get:
- Strong reasoning and analytical capabilities via "chain of thought"
And
- A ton of feedback
If you point enough compute at it you might be able to make it learn in real time?
I think this paradigm will take us to ASI
I think what tesla does with FSD is ultimately going to lead to the closest approximation of the brain - you have different areas for different tasks communicating with each other. although it is impressive what PaLM-E can do with just one net. on the other hand I think that the fastest way to superintelligence would be a transformer successor
What nonsense
well duh, Human lvl AI requires multi-modal ai
Some would disagree, thought tbf I think most of those disagreements are around the definition of the problem
Does GPT intend to be self-sufficient as you describe, though? I don’t think it does. There’s a need for an easy method of access for current knowledge - GPT functions more like a better Google than it does as a human companion.
In that sense, it’s certainly not a dead end. I’m sure you know that, though, sensationalist titles just tend to get clicks.
Does the Vtuber Neuro-sama and the various mods she uses to interact with games count as "novel research" in this field, in the specific direction you are describing here? ;)
AGI is not coming any time soon. This is because algorithms like ChatGPT do not think in any way. ChatGPT processes words, humans process meanings represented by the words and this is gigantic difference. For ChatGPT it makes ~ 0 difference whether it "reads" essay written by autistic 10 year old or philosophical masterpiece written by some genius. Processing these very different inputs take approx the same time and they both change ChatGPT only depending on the words that have been used. Human on the other hand can read one paragraph and be completely transformed by it, example: "win stupid games win stupid prizes". This is because human is GUESSING meanings behind symbols. ChatGPT is basically one giant database, useful but completely unable to create any understanding of anything. If you describe an abstract concept with multiple different examples, human's job of UNDERSTANDING gets much easier, mere AI only gets more confused (I'm talking generally about these bayesian algorithms here). Why some people stare on paintings in museum for hours? Because they do not process mere pixels but IDEAS. Forget about beauty! They try to understand something hidden BEHIND the dots, colors and brushes and it make take them hours, days or years before they get the IDEA
I don't agree with the comparison of language models to giant databases. It has been demonstrated that large language models like Google Research's palm have emergent properties, like common sense reasoning skills that allow it to answer complex questions that require strong world understanding beyond just simple facts, by chaining multiple logical inferences about the world. Your explanation here about comparing human processing time to computers is also quite human-centric. There can be numerous different ways to engineer intelligence, just like we did not have to create a mechanical replica of a bird to be able to fly. Just because computers don't process information in the exact way that a human processes information does not mean that the computer has no understanding. I would argue that language models do indeed have some level of understanding, which is what enables them to answer complex questions that require logical inference that have not been seen within training data, but they do not yet have complete true understanding of the world. I think in the coming years when these models become more and more multi-modal and grounded in the real world they will begin demonstrating true general intelligence.
It is better for AI to stay at the current stage. We can't cope with the changes and it is pretty risky to create AGI
It’s not a dead end when it comments out my code for me, writes my unit-test, writes scrape scripts,
Writes React components, and Django views….
ChatGPT is not suppose to achieve any of the things you are getting at, it’s purpose is to make humans more productive at SPECIFIC tasks.
The rest of y’all are just making videos on hype
you would look great and younger if you shaved your head. thanks for the video
I think you are overestimating human intelligence.
So build that then, if you can't you just prove that you do not know what you are talking about
Bro you bald