Karl Friston's Free Energy Principle
Vložit
- čas přidán 9. 06. 2024
- I don't promise that I'm explaining this WELL, but trying to understand things often involves a lot of explaining anyway!
link.springer.com/article/10.1007/s00422-018-0753-2
patreon.com/thinkstr
I've been chewing on FEP for about a month. I am slowly beginning to understand the concept enough to explain it to someone else and you're attempt has helped me. thanks.
Thanks! I have a lot more to learn.
such complicated, difficult topic, described in digestible way! Thanks. Subscribed :-)
Thanks to you, too! These videos are very fun to make.
Thanks for this. Friston is a tough nut to crack in application.
I think I've learned a lot SINCE making this video. It'll take me a few more tries to actually understand the story.
Hermano muchísimas gracias, no sabes la cantidad de tiempo que llevo tratando de entenderlo, en Latinoamérica resulta difícil encontrar material en español que sea fiel a lo original. Soy estudiante de abogacía y estoy haciendo una investigación. De verdad me has ayudado mucho. Espero que todos tus proyectos se cumplan y de verdad te deseo lo mejor. You are the best !!!
¡Gracias por verlo!
I think that a better analog for the FEP is in RL, than a generic Neural Network, just because with RL, you have a limited number of resources to both learn and optimise your bees' behaviour. I think in his seminars, Friston points specifically to RL as something the FEP generalises (as a instance of Expected Utility Theory IIRC).
I've gotten FEP to work pretty well in RL! I can't give much information yet until I publish a paper, but it can encourage exploration to minimize uncertainty.
Very unique and great deductive skills. Subscribed!
Thanks for watching! I wanna do another on the FEB, but I need to publish a paper first...
Excellent explanation for those of us that have forgotten all of our college math. Thanks Kindly
Thank you for watching!
Even if we face a surprising situation the memory and memory related neural network to the area involved in action selection and planning can help conserve energy.
Oh I like that! It sounds like reinforcement learning reduces free energy retroactively to avoid surprise in the future.
So that in the future if anything significant/surprising can happen we can act properly, we don't make mistakes that can be costly.
@@Thinkstr if I'm correct free energy is available energy, and we minimise free energy due to how much we lose if we don't.
@@ritvicpaarekh6963 It also ties heavily into the statistical mechanics and the second law of thermal dynamics (Entropy). Energy is information Information is energy (Feynman) Life finds a "sweetspot" value of entropy so as to maintain it's existence.
Nicely explained.
Thanks!
Thanks for helping deconstruct this concept!
Hi! Thanks for watching!
Got a sub from me too :) keep up the good work!
@@joshismyhandle Awesome! I want to make another vid about the FEP, but I need to publish a paper about it first, and that might take a while...
I look forward to seeing how your views have progressed. Learning is a marathon, not a sprint. Share your progress in your own time :)
I think you did it justice.
Thanks! I've actually been working with FEP for a few months now and I understand it a lot more, so I'll probably make another video about it eventually.
Why do we (some) love surprise parties? Gifts? Adventures?
Great question! An interesting aspect of the FEP is that it's possible to be TOO afraid of surprise. What we really need is to seek out safe surprises so they can't surprise us in dangerous ways in the future. Like, we wanna see legos on the floor so we don't step on them.
@Thinkstr agreed, but the answer to that question must be even more complicated and interesting. Firstly, it is clear that there is a tension between decreasing the unpredictability of the environment and exploring the environment to create better models of it. This itself seems to be rooted in a deeper paradox, which is that the more an animal explores, the more the range of surprises increases. For instance, QM was a very surprising theory. Why didn't we just stick with classical models?! Second, I think seeking out novel interactions and environments has the potential benefit of increasing the size of our Markov blanket. For instance, meeting someone interesting with surprising points of view increases the range of stimuli we have access to a well as gives us more possibilities for affecting our surroundings. I think symbiosis operates this way. In any case, the point is that something very intricate happens with the free energy principle as one goes up the scale of biological complexity. I think the FEP holds, but more must be said for how the temporal cone of animals increases, when clearly a larger temporal horizon increases the potential sources of surprise.
@@kvaka009 Trying to apply the FEP to reinforcement learning, I've found agents seek out surprise until nothing can surprise them anymore; then the agents understand their environment so thoroughly they know how to get extrinsic rewards. I guess knowing is half the battle.
Very interesting discussion that helps me to think about these concepts...
There is indeed an apparent contradiction between (1) minimizing surprise, that is, acting in a way to best predict the next observation or sensory input to come ; and (2) decreasing the unpredicability of the environnement, as you phrase it @kvaka009.
How does one learn and adapt from a changing environment without expose oneself to novelty ? Is being averse to uncertainty/ambiguity an efficient way to learn in an environment that is - at first sight - umpredicable ?
To me, this aversion to uncertainty/ambiguity sounds like the perfect opposite of curiosity, but in the active inference litterature it is not. As I currently (and erroneously) see it, it's as if the brain would just stick to the same and predictable routines, and will be averse to aknowledge any novelty or richness left available to gather in the environment, thereby hindering any potentiality to learn and adapt.
Certainly i'm confused with some terminology and concepts beneath... I'm sure there is something I dont understand.
I'm doing a PhD in this field (trying to link oculomotor control to Friston's active inference theory), and me and my supervisor are actually kind of stuck on this weird contradiction.
Hope to see your responses in my notifications guys ! 😂
@@Thinkstr what you described corresponds to a pattern classically obseved in FEP litterature, the fact that agents engage primarily in a "epistemic", information-seeking behavior, so as to gather knowledge about the environment it is in, and then engage in a reward-seeking behavior. This is the reason why FEP is tought to resolve the explore-exploit dilemma, widely used in cognitive psychology and ethology (should I stay exploiting this actual food source, or should I seek another potentially non-existant, more rewarding food source ?)
See for example : pubmed.ncbi.nlm.nih.gov/25689102/
Keep doing what you’re doing dude
I'm tryin, ha ha! I've learned a lot about fristons since this video, but I have to finish a project before I talk about it. Hopefully soon!
There is some critique of this principle, because 'minimizing surprise' does not say what is important about the world or your choices. Let's say this principle is a general model of how the cortex works, but does not describe how to make efficient use of this machine. There is no 'attention' to guide it's choices.
You're right! Where extrinsic or intrinsic rewards come from is another issue.
@@Thinkstr Oh. Thanks for the quick answer.
@@zerotwo7319 No prob, thanks for watching. This is definitely one of my more popular vids, but it's also one I should revisit; I've learned a lot lately!
Liked and subscribed cuz its a habit!❤
My professor says some of my results are publishable : D Gosh, it seems to be happening!
@@Thinkstr congratulations and well deserved!
Can you put the links of paper you have used to prepare this video?
I should! I'll see if I can find it again.
Edit: Ah, I have found it! link.springer.com/article/10.1007/s00422-018-0753-2
I ❤️ your bee example. At 3:25 you show a maze. There is a very cool (free) app in development based on active inference research. You can try your skills at playing connecting the dots, it's called ThinkAhead by Giovanni Pezzulo. It only works on Android OS at the moment. 😊
Thanks for watching! I haven't worked on anything bee-related for a while, but my biggest reinforcement learning project sort of started out that way.
@@Thinkstr what's your reinforcement learning project?
@@silviopina_111 One agent is a predator, which chases another agent, prey. Here: czcams.com/video/PyStTy-LrfM/video.html
You explained it very well. What did you not completely understand?
Oh, thank you! There's a lot more math involved that I don't reference yet, and I'm still trying to make sure I've got it right before I get deeper into it.
@@Thinkstr Cool 😎 Appreciate your effort
@@raj3shv These are soooo fun to make, haha!
Can you make a video, please, about apdated model by Friston - Solms with prediction error
Ooh, thank you! I'll look into that.
Oh YES, please!!! 🙏 I am no scientist and always sucked at math, but got interested in the FEP by reading the great Mark Solms "The Hidden Spring". I would SO LOVE to see a "part 2" or update to this, based on your new knowledge!!! It might might Be interesting to compare the old and new video in terms of: "I thought I understood it as this, but now I understand it as that that". It always really helps to see the process.
Also, small "typo" at the start... it sounds like you say "Kurt", not "Karl" Friston 😉
The mind is a probability, or predictive matrix. Our mind parses the likelihood of outcomes. The FEP is about the 'self' as a tool to operate and improve this system. The self helps the matrix refine it's predictions, and the self helps balance our expectations(the probabilities are often influenced by the self) and our actual sensory input. Here lies the real crux. Do we use the self to change our outcomes (with our actions) to match our preferred results? Or do we change the sensory input(change the focus of the matrix) to help influence the outcomes we desire. There lies the entirety of the human condition.
“If you make people think they're thinking, they'll love you; but if you really make them think, they'll hate you.”
― Harlan Ellison
@@alohm I'm so glad to be studying this area of math. It's gotten me thinking about numbers AND the human condition!
So if something with a Markov blanket system exists over time, it must resist entropy by gathering evidence of its own existence, minimising free energy or surprise, via a change in its modelling or actions. The first line of thinking makes sense, if something exists over time and is consistently able to resist entropy, it must have a strategy to do so whether consciously or not.
Whether that strategy is gathering evidence of its own existence, I'm not sure.
For example, I'm not trying to confirm my own existence by any means, I'm looking to understand it, even if understanding it moves me towards an increasing accumulation of evidence that disproves my existence. I'm not actively seeking the blue pill if you will, I seek the red as it moves me closer to the truth. This may be my strategy for resisting entropy, by better modelling the world, others and my self, but the aim isn't reduction of uncertainty, it's growth. Growth in consciousness. Growth in level of awareness of being, in relation to understanding of self, other (people/species) and whole (nature/universe/God).
Perhaps this reduces down to a systems ability to process information at the various Markov blankets in which it exists. Or awareness of that within its blanket, outside its blanket & the interface between the two. The quantity and quality of its data set, the ability to process, model & synthesise that information, then translate those learnings in to revisions of its modelling or actions, in a real time continuous learning loop.
Minimising uncertainty might be a key survival strategy, but is mere survival the highest motivating force in the universe?
I like the fuzzy line between math and philosophy. Depending on who you ask, maybe there are more motivating forces, or many those other motivating forces are just survival wearing a mask.
I dont promise to understand but while you keep it entertaining ill keep watching
Ha ha, and I'll keep making them!
Nice try as a first attempt. It is a somewhat slippery concept.
Thanks. I'm actually making another video on it soon, and I think I've understood it a little deeper.
@@Thinkstr yay!
@@Thinkstr would you say "free" energy could also be called "lost" or "wasted" energy? The word "free" conveys something positive, but it seems the whole point is to minimize it, having less of it, to avoid entropy, so "free" sounds weird. Or maybe "unused"?
@@silviopina_111 I think "unused" might be a good way to explain the idea, but I'll call it "free" until Friston changes it himself, haha
@@Thinkstr well I hope he has a chance to see what your video!