Regression To The Mean
Why Perfection Rarely Lasts
Whenever correlation is imperfect, and thus causation is unproven, over time the best score will get worse and the worst will get better
Regression to the mean shows that following an extreme random event, the next random event is likely to be less extreme.
The concept was first presented by Sir Francis Galton and the more comprehensive statement of regression to the mean is that:
In any series of events with complex phenomena that are dependent on many variables, and where chance is involved, extreme outcomes tend to be followed by more moderate ones.
It is not a mathematical law but a statistical tendency.
Regression To The Mean - The Imperfect Correlation Of Factors/Causes and Chance
To be able to understand regression to the mean, we first need to understand correlation.
Correlation is a relationship, thus action A relates to outcome B.
The strength or weakness of the correlation of shared factors [or causes] will show the level of dependency of the outcome [or score] on those factors.
If the correlation is strong there will be little to no regression to the mean, whereas as a weak or non-existent correlation between shared factors will likely show that there is a regression.
Examples:
[1] Strong correlation of factors - TEMPERATURE
The only factor determining temperature – velocity of molecules — is shared by all scales, hence each degree in Celsius will have exactly one corresponding value in Fahrenheit.
So the temperature in Celsius and Fahrenheit will have a correlation coefficient of 1 and a plotting of temperature score will be a straight line.
[2] Weak to non-existent correlation of factors - BOTTLED WATER CONSUMPTION and DEATHS FROM AIR CRASHES
If we looked at all
countries in the world and plotted death rates from air crashes per country, in any specific year ,
against per capita consumption of bottled water, the plotting of scores for each event would show no
pattern at all. Thus the correlation coefficient is 0, i.e. there is no correlation.
We need to be especially careful of regression to the mean when we are trying to establish causality between two factors. As we have already established: Correlation is a relationship, thus action A relates to outcome B. Whereas: # Causation can be defined as action A causes outcome B. # Causation results from a perfect correlation between action A and outcome B.
This is something that many people, social media, the general media, and
sometimes even trained scientists, fail to understand or recognise.
Regression To The Mean - The Dangers Of False Attribution Of Cause & Effect
Regression To The Mean - Examples
Can you recall a wonderful evening out? The weather was fabulous, your table was great and restaurant was excellent, the food and wine were superb and you were with great people.
Then you tried to repeat the experience and somehow it wasn't as good and you were disappointed. Why was this? It was because your perfect evening was due to a random series of chance events that all fell in place on the wonderful evening.
Whenever you try to repeat the perfect experience, chances are that at least one thing won't be perfect the next time.
Regression to the mean is prevalent in sport and can explain the “manager of the month curse” in football.
This award is usually won by managers who have had four or more wins in a row, often because of a combination of skill and luck.
When the luck runs out, the “curse” strikes.
Regression to the mean can be harmless, but it becomes a problem when it is misinterpreted:
"For example, imagine you ran a hospital and were told that hospital-acquired infections were five times higher than average last month. A colleague tells you they know the cause and it can solved by using more prophylactic antibiotics.
You agree and in the following month you’re told that prophylactic antibiotic use is through the roof and infections have come down. Your mind makes a causal connection and you’re now convinced of the need for widespread prophylactic antibiotics, a potentially dangerous connection given that the unusual infection rate could have been due to chance events.
Now your hospital budget will be tighter because of the costs of using more antibiotics, and you’re contributing to serious problem of antibiotic resistance."
Countering The Dangers Of False Attribution Of Cause & Effect
Here are 4 suggestions for identifying and countering false attribution of cause and effect and understanding the regression to the mean
- Gather more data
- Identify the variables
- Wait and watch
- Rinse and repeat
Return to: Mental Models
LATEST ARTICLES
-
Your Relationship With Failure Is Critical To Your Success
Just Because You Have A Spectacular Failure Doesn’t Mean The Game Is Over. The thought that you could fail can really slow you down. But the problem is not the failure, the problem is your relationshi… -
Lost, Stuck & Going Round In Circles?
Are You Lost In Transition? To be in an alien North African city, late in the evening, in the dark, completely lost in transition from the main market square back to the obscure location of our Riad… -
Seeing Is Believing Or Is It?
Do You Believe What You See Or Do You See What You Believe? Like the characters in "The Matrix" we are fooled by the illusion that what we see is how things really are. Why? Because this is the mind's… -
What Difference Did You Make?
What Impact Will You Leave In The Lives Of Others? If you knew you were going to die tonight and you were given a few minutes to reflect before that happened, what would you say was the meaning of yo… -
Compassion Is Selfish And You Need It
If You Want Others To Be Happy, Practice Compassion. If You Want To Be Happy, Practice Compassion. Compassion is derived from the Latin words "com pati" meaning to suffer with. It has been defined as… -
The Placebo Effect - 2 Key Takeaways
The Power Of Expectation Of An Outcome Regardless Of The Lack Of Any Substance To That Expectation. Traditionally, the placebo effect occurs when false, or dummy, treatments of inert substances [that… -
The Red Pill
This Is What Will Happen After You Take It.
"This is your last chance," rebel leader Morpheus tells Neo, holding a coloured pill in each hand. “You take the blue pill, the story ends, you wake up in y… -
Transformational Life Hacks
This Has To Be One Of The Best Ways For Living The Best Life You Can Possibly Live. An online search on life hacks will invariably provide you with multiple links to ways of increasing your productivi… -
This Too Shall Pass
The Blessing And The Curse Of Impermanence. There is a story of a king who sought wisdom. Eventually he heard of a man living in a remote part of the kingdom who had a reputation for great wisdom. The… -
The Stories We Tell Ourselves
The Stories That We Tell Ourselves Are A Matter Of Choice. The stories we tell ourselves form the backcloth and the tapestry of our lives. These narratives shape and define our sense of self and how a… -
Who Do You Trust?
Trust Is A Choice And It Creates Your Reality. Nothing in life is permanent and nothing lasts forever. Every relationship ends in tears. Somebody leaves or somebody dies. This is how things are. This… -
How Positive Asymmetry Can Transform Your Life
Finding This Hidden Value Is The Key To A Huge Upside. Most of us spend our lives working to linear rules where there is an equal relationship between effort and result. Work hard you get a result. Wo… -
5 Tips For Developing A Growth Mindset
How To Be There When Preparation Meets Opportunity. The growth mindset holds the core belief that your intelligence and personality is something you can develop and that you can expand and grow. The p… -
Lost For Words? Experience Truth
If you can't think of the right thing to say, say nothing.
The concepts we employ, the categorizations we apply and the words we choose and use to articulate a direct experience put us in a double bi… -
Belief - A Terrible Tyrant Or An Empowering Servant
To Believe Or Not To Believe - That Is The Question. Nothing has inflicted more suffering on humanity than its beliefs and dogmas. The religious and cultural beliefs that sanction and allow or impose… -
How To Avoid Herd Mentality
How To Stay Far From The Madding Crowd. Herd mentality describes how we make decisions based on what everyone else is thinking and doing. It is also known as a mob or crowd mentality. We follow the he… -
Are You Trying To Teach A Cat To Bark?
We Suffer Every Time We Believe A Thought That Argues With What Is. When your mind is perfectly clear, what is, is what you want. If you want reality to be different than it is, you might as well try… -
The Value Of Slack
Slack Is The Capacity To Effect Change And The Resource To Deal With The Unexpected. Many organizations are obsessed with efficiency and try to ensure that every resource is fully utilized to its full… -
6 Tips To Help You Stop Trying Too Hard
Less and less do you need to force things, until you arrive at non-action. We are brought up to believe that we get what we want in life as a result of what we do, by the actions that we take, that ha… -
Your Point Of Focus
Is It Helping You Or Hindering You? Your point of focus is the central point of your attention. Your point of focus is a binary choice and the result of that choice is critical to your happiness, your… -
Your Secret For Success
To accomplish any success in life, you must identify accurately what it is that you hunger for, thirst for and long for. The purpose of this article is to look at the foundation of the secret for succ… -
Can You Stop Thinking?
Discover The Joy Of A Quiet Mind Can you stop thinking? According to Zen Buddhist priest Shunmyo Masuno the art of not thinking about anything is not easy to achieve. Even practising monks find it dif… -
How To Build Self Confidence
What Is Holding You Back From Pursuing Your Dreams? Your self confidence is based on ability. Your self confidence is all about your level of trust, faith and belief in your abilities [your power] and… -
How To Overcome Fear And Perform Powerfully
Learning how to overcome fear and perform powerfully is a skill that we all need to acquire as fear of failure affects most of us at some point in our lives. Whilst it is natural to feel a level of fe… -
Dance Into The Light And Stop Chasing Your Tail
You are creating hell on earth when you are too subsumed within your underlying and deeply embedded negative energetic states. To counter your dissatisfactions and misery you initially go unconscious… -
Change Your Language To Change How You Feel
How you mentally characterize a situation has a profound impact on how you respond to it emotionally. There is a direct causal link between how you think about a situation, the words you use to articu… -
How To Live In A Participatory Universe
What good is a universe without somebody around to look at it? What Conclusions Can We Draw About Living In A Participatory Universe? The inescapable conclusion is that the world and the universe (wit… -
4 Keys To Working With The Energy Of Life
The journey home to the energy of life does not consist in arriving at a new destination it is a coming home to one’s self. [1] How are you feeling, what is your dominant emotion right now? [2] Where… -
How To Deal With The Challenges Of Playing The Long Game
You have to be willing to suffer today in order to reap enormous rewards tomorrow. The long game is so hard to play because it involves taking a series of very small steps, that people rarely see, per… -
If You Don't Take Care Of You, Who Will?
How To Build Self Esteem In my early life and until my early twenties I suffered from a deep inferiority complex and low self esteem. Forty years later and, said in sincerity and with humility, I have…