This is one of my slightly more important posts in comparison to most of the others, yet not anything that valuable or insightful that I don't discuss in greater detail such as 'Objective is Subjective' and 'Confessions' which discuss similar topics but in a different context. It's an interesting read, in my opinion, but feel free to skip it if you wish to save time, or aren't too interested in certain topics from physics, neuroscience, theoretical computer science or cognitive psychology that I mention below.
A few days ago I came across this article, and it's one of the best articles I've read in weeks for several reasons, including but not limited to me getting to know that there exists a person who is applying the principles of physics to understand the brain, which aligns with one of my projects to quantify and objectify psychology and psychiatry, and some of his habits and career choices that were insightful and amusing to be but are irrelevant to our present discussion so I shall discuss them some other day.
In this post, I'll attempt to describe the free energy principle as I understood it or rather as I understood from the writer's understanding and attempt to explain it, which as you may already have guessed might not be accurate and do Dr. Friston's ideas the justice they deserve, but then I'm also going to connecting it with some concepts from cognitive psychology, electrical engineering, theoretical computer science, physics, and my personal experiences.
I'm going to talk about the connections between Dr. Friston's Free Energy Principle, the Explore-Exploit tradeoff from Theoretical Computer Science, the Second Law of Thermodynamics from Physics, Cognitive and Social Psychology, and my own personal experiences. I will first explain, then tie the concepts together towards the end.
The free energy principle, proposed by neuroscientist Karl Friston, is a framework for understanding how living systems, such as animals and humans, interact with their environment to maintain their stability and well-being. It provides a way to think about how organisms seek to minimize prediction errors and maximize the accuracy of their internal models of the world. the free energy principle offers a way to understand how living systems seek to reduce uncertainty and maintain stability by continuously updating their internal models of the world and adjusting their actions accordingly.
One way to understand the free energy principle is to think of it as a way to balance the desire for exploration and the desire for stability. Just as a ship's captain needs to navigate the waters while avoiding storms and maintaining a stable course, organisms need to explore their environment to gather information and find resources while avoiding danger and maintaining homeostasis.
An analogy to the free energy principle can be found in engineering and control systems. Imagine a self-driving car trying to navigate a road. The car has an internal model of the world, and it uses sensors to gather information about its surroundings. As it moves down the road, it constantly updates its model and uses control algorithms to adjust its actions to minimize prediction errors and maintain a stable path. In this analogy, the car's goal is to minimize the "free energy" or prediction error, which is equivalent to the organism's goal of maintaining stability and well-being.
The free energy principle can be seen as a biological counterpart to the second law of thermodynamics. Just as the second law states that entropy (or disorder) tends to increase over time, the free energy principle states that living systems tend to reduce prediction errors (or uncertainty) over time.
Now I'll come to the part that's more relevant to my personal experiences, and explain how it fits in with an important concept in cognitive psychology and theoretical computer science, and how I used this concept while trying to come up with a framework to make some important decisions related to my career and academics.
First I'll try to explain the concept of explore-exploit tradeoff from theoretical computer science, something I came across while listening to a TED talk by an interesting personality whose works are a topic for another blog post.
The explore-exploit tradeoff refers to the decision between exploring new options and exploiting existing knowledge. Here are some examples:
Restaurant selection: Imagine you're trying to choose a restaurant for dinner. If you always go to the same restaurant, you're exploiting your existing knowledge of that restaurant. But if you try a new restaurant, you're exploring new options. The explore-exploit tradeoff is the balance between sticking with what you know and taking a chance on something new.
Investment portfolio:
In investing, the explore-exploit tradeoff is the decision between investing in familiar, established companies (exploitation) and investing in riskier, less established companies (exploration). In many video games, players have the option to explore new areas or stick with familiar ones. For example, in an RPG game, a player may choose to grind and farm for experience points in a familiar area (exploitation), or venture into new territory to find better loot (exploration).
In marketing, a company may choose to exploit its existing brand recognition and focus on familiar products (exploitation), or explore new markets and product lines (exploration).
These are just a few examples, but the explore-exploit tradeoff is a common theme in many fields and is a key aspect of decision-making under uncertainty.
In social situations like dating, for example, a person who is dating may find themselves stuck in a pattern of only dating people who are similar to their exes, exploiting what is familiar and comfortable. However, this can limit their options and prevent them from exploring new relationships that could potentially be more fulfilling.
Similarly, while making friends, a person may be hesitant to branch out and try to make connections with new people, instead relying on their existing friend group (exploit). However, by not exploring new social opportunities, they may miss out on forming meaningful relationships with people who could bring new perspectives and experiences into their life.
So now you must have got the general idea and a basic understanding of the concept. In cognitive psychology, there is this concept of rationalization, which basically says that we act based on our intuitions, desires and preferences, and then later come up with clever rationalizations to justify our behaviour or beliefs to avoid cognitive dissonance, a phenomenon that occurs when when two beliefs you hold are contradictory.
In other words, the mental discomfort experienced by a person who holds two or more conflicting beliefs, ideas, or values. Examples: a person who considers themselves an animal lover, but regularly eats meat, may experience cognitive dissonance, or a person who smokes cigarettes but believes that smoking is harmful to their health. You get the idea.
My brain is pretty good at coming up with rationalizations, so I have to be extra-aware of it, and so I have developed some meta-cognitive strategies to keep it under check and try to detect when my brain is just trying to justify my own preferences that might be irrational and lead me to made irrational decisions.
It is able to come up with rationalizations for any side, but with some practice now I'm sometimes able to catch it and question: is my brain rationalizing based on what I intuitively have a preference or subconscious desire for, or is there logic and evidence and good reasons to back up my brains arguments and reasons? And this way I'm able to somewhat catch my brain red-handed whenever it's doing that..
Deviating from the main topic, I have come across some people who say that it's easier for people who are less intelligent to be rational because they don't fall for such traps. Here's my reply if you happen to believe so: Now first of all I'd like to make it clear that I don't consider myself smart, I don't even know what it is supposed to mean, as everyone has different definitions from it, but I do know that my brain is good at rationalizing, and I don't think I'd prefer to be someone whose brains are not capable of giving them rationalizations and new ideas, because from psychological experiments I've learned that everyone's brain is like that- so essentially I'd be losing my ability to think and generate new ideas and still not get rid of rationalizations and cognitive biases, and I don't want that; I'd rather have the biases and the ability than the biases and no ability.
Coming back to the main topic: Rationalizations are connected with the explore-exploit tradeoff (and hence the Free energy principle) because there comes a point after which we have to stop overthinking (which is also something I do a lot) and close off all the alternatives to focus on what we have decided, the one alternative we have chosen after some deliberation, because if we don't do that, we would always wonder about the road not taken and never get anywhere, metaphorically speaking, thinking about what could have been instead of what is, and fail to see the benefits of what we do have, so our brain uses rationalizations to help us justify our choices.
But of course we need to make sure that we are not rationalizing without thinking at all, because then we would never know if there is a better alternative, but like me, you also must not do too much of overthinking and overanalyzing such that you never get anything done or spend cognitive resources, effort and time not commensurate with the expected benefits or probability of getting a desirable outcome from those efforts. That's why even if you put in a lot of efforts and are able to make an infinitesimal improvement to the existing plan, you probably shouldn't, but again that depends on the stakes and your values, so we must not generalize so liberally.
I make use of such concepts all the time while making decisions where they are applicable- organizing all my disorganized ideas spread all over my notepad at different places that I could organize and combine to write a blog post by weaving them into a coherent narrative (as I am doing now), or explore new sources of knowledge- such as reading new books- to acquire new pieces of information that I could later connect with other pieces of information I have acquired in the past, or my past personal experiences.
I also came across this dilemma where I was confused whether I should explore other disciplines such as the mathematical sciences or just stick to what I knew I like and I'm good at. I have written about this in a blog post about busting the myth that psychology is not a science. And going even further back, and at a much larger scale, there is another instance that my brain helpfully enabled me to recall while I'm writing this; oh, mysterious subconscious processes working in the background, I'm going to get you one day.
This is related to the concept of learned helplessness and positive psychology. Without going into too many details, I'll describe the concepts and my experiences posed as hypothetical questions. In brief, negative emotions seem to serve a purpose, but sometimes they are excessive and prove to be a hurdle in allowing you to achieve your dreams.
Learned helplessness is a psychological phenomenon where a person believes they have no control over a situation and thus stop trying to change it, even when change is possible. Examples: A student who consistently receives poor grades and eventually believes they are not capable of getting good grades, even if they start putting in effort. An employee who experiences repeated failures in the workplace, such as being passed over for promotions, and eventually becomes passive and stops trying to improve their performance.
So I thought- how do we know when our emotions are genuinely telling us to work on something we have a higher probability of success but that we desire less but would probably succeed at if we tried, and we are these emotions just an unnecessary hindrance to achieving our dreams and goals, and if we pushed ourselves beyond our limits by trying harder, we could actually achieve those goals?
This also relates to one of my first blog posts about the mismatch between ability and interests, and this is an important question because if we keep trying at something that we have no realistic probability of attaining success at, and if it's the negative emotions themselves that's making us les likely to attain those goals- because if they didn't exist then we would never feel like giving up and we could just try till we succeed and also enjoy the process without any guilt or anxiety- then we need to know if they actually serve any purpose, or if we'd be better off without them.
This is also a great decision making tool that I have added to my toolkit with techniques such as taking into account all the opportunity costs and probabilities associated with every alternative, as well as running a cost-benefit analysis and thinking about every component in the chain of cause and effect, so it's just going to help me overthink even more. I'm sure that once your grasp and internalize this concept you'd also start seeing the explore-exploit tradeoff and analogical principles everywhere in your life and while making decisions.
And that's good because then you'd be able to make more calculated and informed decisions by coming up with a structured framework to tackle such questions, which even though probably won't give you a simple and clear answer, because there are seldom simple and clear answers in life with it's sheer complexity.
We can, however, get closer to the truth and get a clearer picture of the problem which is an essential step to arrive at the right solution, if there is any, or more realistically the best possible solution given our constraints even if it's just good enough, because as we just saw, at once point of time we have to quick exploring and searching and settle for the current best.
Let me know if you have any comments and feel free to connect!
Before ending, here's something related to the explore-exploit trade-off combined with history and chemistry that I found in my notepad but don't think is mine because I'm not that talented at creative writing and poetry. Enjoy.
Throughout history, some have dared to journey beyond the known, venturing into the uncharted waters of possibility. Others have clung to the safety of the familiar, content to reap the rewards of their well-worn paths. Yet the alchemist, ever seeking to transmute the mundane into the extraordinary, stands at a crossroads. Shall they continue to test the boundaries of what is possible, chasing after new concoctions and novel elixirs? Or shall they rest on the laurels of their past achievements, relying on tried and true recipes to bring them success once more?
The choice is theirs, but in that decision lies the potential for greatness, or the risk of stagnation. The quest for knowledge has often been at odds with the desire for comfort and security. Those who explore the unknown are often seen as foolish or reckless, while those who exploit what they already know are praised for their prudence. But the truth is that both approaches have their value, and it is only by balancing them that we can make progress.
The chemist, like a seeker on a path, must weigh the balance of two worlds. On one side lies the thrill of discovery, the rush of finding new properties and hidden depths within the elements and compounds of the earth. On the other side stands the practicality of the world, where useful substances must be created to aid and better the lives of those around them. The alchemist must stand at this crossroads, choosing to explore the unknown and plunge into the depths of the mysterious or exploit the known and master the familiar. The choice is never easy, but in that decision lies the power to unlock the secrets of the universe, to illuminate the mysteries of creation, and to create substances that change the course of history.
Update: I was made aware of the fact by some experts that they were expecting to learn why the free energy principle is called "free energy" and not "Minimization principle" and how exactly it relates to the actual free energy concept from thermodynamics like Helmholtz free energy, which is the "free" work a system is able to do while maintaining the temperature.
Another person told me- "If he had not attempted to link his argument to the second law and had not referred to it as "free energy", people might not have been distracted and be able to focus more clearly on the substance of his argument. In my opinion, this was a very, very unfortunate mistake, and had he not made it, his model would have been given the same gravitas of his earlier mathematical advancements."
My response: As I mentioned in the post, I don't think I understand the Free Energy Principle that well to comment on why Dr. Friston chose to call it by that name, and, as you point out, the minimization principle would have been an equally good, if not better, name. Still I will try to pretend for a moment that I know it well and some possible reasons that it is the case.
As far as I understand, it does not relate to the original concept of free energy you refer to, but rather it attempts to explain how how living organisms and intelligent agents maintain their internal states in order to remain alive and functional. In this context, "free energy" could be referring to the difference between the energy that an organism or agent has acquired from its environment, and the energy that it expends in order to maintain its internal organization.This concept is inspired by the principles of thermodynamics, where free energy refers to the amount of energy available to do useful work. In thermodynamics, free energy is often represented by the Helmholtz free energy, which is the energy that a system has available to do work while maintaining its temperature.
The Free Energy Principle in neuroscience and AI is related to the thermodynamic concept of free energy in the sense that it also deals with the energy available to an organism or agent to perform work. However, the concept is adapted to a different context and refers to a different kind of energy. So, It is called the "Free Energy Principle" because it refers to the energy that is available for an organism or agent to maintain its internal organization, and is not directly related to any of the thermodynamic concepts of free energy.
Now that could be the reason, as I said I'm not sure, because only Dr. Friston would know why he did that, and the original article says that his theory has baffled many of the greatest minds today, so my understanding is probably not sufficient to answer such a question, and I could be mistaken, but then again it could, because things with a very low probability also happen once in a while, so my reasons being based on a half-baked understanding of the principle are not necessarily wrong, but yes they probably are.
This is his reply to my comment: "Thinking about what you wrote, maybe there is a more in-depth link to free energy in thermodynamics after all: the living organism has to maintain homeostasis. Therefore, the work done it can do on the environment is the (Helmholtz) free energy (at least for warm-blooded animals).
However, you also write that he is concerned about the amount of work/energy the organism needs to maintain homeostasis, which is not a free energy principle.
As regards your strict separation of thermodynamics vs biology/neuroscience, I would not necessarily do that. Thermodynamics is an effective theory of macroscopic phenomena based on a microscopic theory. It is transferrable to whatever microscopic and macroscopic mean (this is why you have the notion of Shannon's information entropy, for example) So whatever, the role of "temperature" is in your system, you could formulate a free energy principle for. However, since we are talking about biological systems I would expect the physical notion of temperature to be certainly part of that."
So, if I understood him correctly, he tries to explain how The living organism needs to maintain balance, which requires work and energy, similar to the concept of free energy in thermodynamics. However, the concern is not just about the amount of work needed to maintain balance, which goes beyond the free energy principle. It's not necessary to strictly separate thermodynamics and biology/neuroscience, as thermodynamics can be applied to microscopic and macroscopic phenomena. The physical notion of temperature is likely relevant to biological systems when considering a free energy principle.
Comments