Regression To The Mean
Why Perfection Rarely Lasts
Whenever correlation is imperfect, and thus causation is unproven, over time the best score will get worse and the worst will get better
Regression to the mean shows that following an extreme random event, the next random event is likely to be less extreme.
The concept was first presented by Sir Francis Galton and the more comprehensive statement of regression to the mean is that:
In any series of events with complex phenomena that are dependent on many variables, and where chance is involved, extreme outcomes tend to be followed by more moderate ones.
It is not a mathematical law but a statistical tendency.
Regression To The Mean - The Imperfect Correlation Of Factors/Causes and Chance
To be able to understand regression to the mean, we first need to understand correlation.
Correlation is a relationship, thus action A relates to outcome B.
The strength or weakness of the correlation of shared factors [or causes] will show the level of dependency of the outcome [or score] on those factors.
If the correlation is strong there will be little to no regression to the mean, whereas as a weak or non-existent correlation between shared factors will likely show that there is a regression.
Examples:
[1] Strong correlation of factors - TEMPERATURE
The only factor determining temperature – velocity of molecules — is shared by all scales, hence each degree in Celsius will have exactly one corresponding value in Fahrenheit.
So the temperature in Celsius and Fahrenheit will have a correlation coefficient of 1 and a plotting of temperature score will be a straight line.
[2] Weak to non-existent correlation of factors - BOTTLED WATER CONSUMPTION and DEATHS FROM AIR CRASHES
If we looked at all
countries in the world and plotted death rates from air crashes per country, in any specific year ,
against per capita consumption of bottled water, the plotting of scores for each event would show no
pattern at all. Thus the correlation coefficient is 0, i.e. there is no correlation.
We need to be especially careful of regression to the mean when we are trying to establish causality between two factors. As we have already established: Correlation is a relationship, thus action A relates to outcome B. Whereas: # Causation can be defined as action A causes outcome B. # Causation results from a perfect correlation between action A and outcome B.
This is something that many people, social media, the general media, and
sometimes even trained scientists, fail to understand or recognise.
Regression To The Mean - The Dangers Of False Attribution Of Cause & Effect
Regression To The Mean - Examples
Can you recall a wonderful evening out? The weather was fabulous, your table was great and restaurant was excellent, the food and wine were superb and you were with great people.
Then you tried to repeat the experience and somehow it wasn't as good and you were disappointed. Why was this? It was because your perfect evening was due to a random series of chance events that all fell in place on the wonderful evening.
Whenever you try to repeat the perfect experience, chances are that at least one thing won't be perfect the next time.
Regression to the mean is prevalent in sport and can explain the “manager of the month curse” in football.
This award is usually won by managers who have had four or more wins in a row, often because of a combination of skill and luck.
When the luck runs out, the “curse” strikes.
Regression to the mean can be harmless, but it becomes a problem when it is misinterpreted:
"For example, imagine you ran a hospital and were told that hospital-acquired infections were five times higher than average last month. A colleague tells you they know the cause and it can solved by using more prophylactic antibiotics.
You agree and in the following month you’re told that prophylactic antibiotic use is through the roof and infections have come down. Your mind makes a causal connection and you’re now convinced of the need for widespread prophylactic antibiotics, a potentially dangerous connection given that the unusual infection rate could have been due to chance events.
Now your hospital budget will be tighter because of the costs of using more antibiotics, and you’re contributing to serious problem of antibiotic resistance."
Countering The Dangers Of False Attribution Of Cause & Effect
Here are 4 suggestions for identifying and countering false attribution of cause and effect and understanding the regression to the mean
- Gather more data
- Identify the variables
- Wait and watch
- Rinse and repeat
Return to: Mental Models
LATEST ARTICLES
Like A Prayer - Life Is A Mystery
It Isn't The Process Of Prayer That’s The Problem, It’s The Way It’s Framed. Regardless of what we feel about Madonna or her song the topic of prayer often arouses strong reactions. Usually, it is som…How Things Really Are - The Inbuilt Design Flaws
Chaos, Disorder And Decay Is The Natural Order Of Things. Nobody has the perfect life. We all struggle and strive to attain health, wealth and personal happiness. Yet these three big areas: our health…Intuition Or Anxiety - Are There Angels Or Devils Crawling Here?
How To Tell The Difference Between Intuition and Anxiety. How do you know whether the voice of your intuition is real or just the product of your inner anxiety? Several months ago I was having a drink…What Is Truth - How To Tell A Partial Truth From The Whole Truth?
How the truth and nothing but the truth is often not the whole truth. My great aunty Flo broke her arm and died. It is true that she broke her arm in 1923. It is also true that she died in 1949. But t…Duality And Life Beyond Your Thinking Mind
Duality and life beyond your thinking mind focuses on the limitations of time, foreground and background, duality and "stuckness". The first aspect of duality and life beyond your thinking mind focuse…The Conscious Mind Is Limited - Be Aware And Be Prepared
Being aware is the first stage of being prepared. The conscious mind is limited in so many ways. There are some who would argue that there is no such thing as conscious thought and that it is represen…Your Inner Map Of Reality - Here's Why You Think The Way You Do
The big picture of how your inner map of reality creates your feelings, thoughts, and behaviours. Your inner map of reality is based on the filters of your own ethnic, national, social, family and rel…The Failure Of Cancel Culture - Suppression Not Engagement
Why we need to wear our beliefs lightly and develop negative capability. Throughout history people have campaigned to fight beliefs, ideologies, and injustices that they perceived to be oppressive, di…4 Big Reasons Why We Get Stuck In Our Attempts At Personal Change
Most People Spend Their Entire Life Imprisoned Within The Confines Of Their Own Thoughts. This first of the 4 big reasons why we get stuck is, in my view, the most important. The "self-help industry…How Do I Change And Why Is It So Hard?
We Would Rather Die Than Change, And We Usually Do In my experience, the vast majority of people who say they want to change don’t change. Most people reading this won’t change because they don’t real…The Illusion Of A Separate Self - Windows 11 With Self Awareness!
Beyond the content of your mind you are so much more than you think you are. When we talk of "myself" this is the conventional way of referring to our self image which is in fact the ego's constructio…Finite And Infinite Games - Dazed, Confused & Ultimately Transcendent
This is not an instruction manual, it is a wake up call!
Over this past week I have read Professor James Carse's highly regarded "Finite And Infinite Games - A Vision Of Life As Play And Possibility"…The Gap And The Gain - How Your Brain Sabotages Your Happiness
How Your Lizard Brain Sabotages Your Happiness We are hardwired to measure our progress in any and all areas of life where we have goals and aspirations. We can't not do it. But what we measure and ho…How To Wake Up - 4 Simple Practices To Help You Wake Up Now
So What Exactly Does It Mean to Wake Up - What Is "Enlightenment"? There is nothing magical, mystical or mysterious about waking up we’re actually having glimpses of enlightenment all the time. Enligh…Situational Communication - Different Strokes For Different Folks
Situational communication is about taking account of 3 often ignored factors about the other person. You are a situational communicator when you recognise that effective communication is not an event…How To Influence Without Authority - 6 Key Tips
The secret to how to influence without authority is that you get what you really want by giving other people what they really want. We live in an interconnected world and knowing how to influence with…Change Questions To Change Your Outcomes
Asking The Right Questions Is Critical For A Successful Change. Every time we initiate a significant change - whether in our personal life or in an organisation - we will most likely over-estimate our…Group Culture - The Invisible Software That Rules Your Life
Group culture is: "How we do things round here". We like to see ourselves as free agents making our own choices and living authentically but the reality is that The Matrix has many layers and we are u…Why Getting From A to B Is Not Aways A Straight Line
In circumstances of significant change, the progress from A to B will not be in a straight line. We run our lives largely on auto-pilot. In most circumstances your experience of getting from A to B is…The Art Of Persuasion Planning For Success - Here's How To Do It!
To be successful in the art of persuasion you must ensure that certain things happen. To be successful in the art of persuasion you must establish a framework of what has to happen to get you to that…The Art Of Persuasion Advanced Communication Skills - Gaining Buyin
Create The Environment Where They Want To Buyin to Your Proposal In order to build the win-win you have to uncover what it is that the other person really wants or needs, and to do that you have to as…The Art Of Persuasion The One Fundamental Principle - Create A Win-Win
The art of persuasion is based on the simple idea that you get what you want by enabling the other party to get what they want. Being a nice friendly person with good inter-personal skills may be a go…Communication Persuasion And Change - Key Skills To Survive & Succeed
It's not the strongest that survive, nor the most intelligent, but those who are most responsive to change, the most persuasive, and the best communicators. We are living in an age of unprecedented ch…The Eisenhower Box - What Is Important Is Seldom Urgent
What Is Important Is Seldom Urgent And What Is Urgent Is Seldom Important. The Eisenhower Box is a time management and decision-making model devised by President Dwight Eisenhower to help him prioriti…Zen Enlightenment [Satori] - The Stink Of Zen
Lost In Our Delusions About Enlightenment. There is something in human nature - a desire to glamorise, sanctify, objectify and idolise – that elevates people who have offered deep insights to the huma…5 Zen Mindsets For Mastery - In Any Area Of Your Life
The Wisdom Of A Person Who Masters In Any Art Is Reflected In Their Every Attitude. The state and quality of your mind has a very large bearing on the quality of your performance in any area of life t…Dealing With The Toxicity Of Online Dating - 6 Key Tips From A Clinical Psychologist
Toxicity Is The Price Tag Of Accessibility. In the early days of online dating, users were vetted and had to go through a registration process and agree to comply with a code of conduct designed to en…Why Understanding Ergodicity Is Critical To Your Long Term Survival
How Not To Be Fooled By Randomness. Ergodicity is an ugly word from the world of mathematics. It is an umbrella term for two sets of conditions of probability and outcome. These two conditions form th…Dealing With Imposter Syndrome - Ego Is The Enemy
How You Frame A Situation Has A Profound Impact On How You Respond To It Emotionally. Imposter syndrome is a psycho-emotional experience of a fear of being found out as incompetent despite ongoing evi…The Challenges Of The Road Less Traveled
Issues You'll Face When Playing The Long Game. The challenges of the road less traveled is loosely based around the phrase popularised by M.Scott Peck with his book "The Road Less Traveled". This arti…