Anecdotal or personal evidence (APE’s) are in my opinion the worst symptom of clouded judgment. A few examples among dozens of others (caution: they do not in any way represent my way of thinking):
- "There are too many civil servants and they're useless. The proofis that I have a friend who’s a physician. In his lab, there are three medical assistants and they don't do anything."
- "Good evening, everyone. It's 1:30 on our [random 24-hour news channel].
Terrorism, a growing threat in our society? Just yesterday, [insert isolated news item]" - "You should drink birch sap for your bellyache. Every time I take it, my stomachache stops."
- "Vegans are hateful. Did you know a vegan tried to stab a butcher yesterday?"
- "The cops are violent. Yesterday a high school kid was beaten to the ground."
In short:
An anecdotal or personal evidence (APE) is an isolated, non-representative story used as an argument for a more general theory.
Why are APE’s inherently erroneous
APE’s are often used because they are simple to understand and conceal a much more complex truth. When you have a belief about the world, you may have several categories of data that reinforce that belief. For example, consider the following theory:
Theory : France has unemployment because it protects its unemployed too much.
(Again, my beliefs are not related to the theories I present in this article).
I have already had conversations with many, many people on precisely this subject. Sometimes my interlocutor knew the unemployment rate in France (this statistic is trumpeted by the media), but in any of these conversations did my interlocutor know:
- the cost of unemployment in France;
- the number of job opportunities in France;
- France's deficit;
- data on the hardship of work in France;
- etc, etc…
It happens to be so that the question of unemployment in France is an eminently complex problem. It is in these circumstances that I hear the highest number of APE's. How many of my relatives don't like unemployed people, because they know one or two of them who are unemployed and take holidays. Relying on such moral judgements leads to a distorted vision of the world.
Caution : I do not blame anyone for not knowing everything about everything. Simply, when one has an opinion on an issue, one must be aware of the limits of one's knowledge on the issue in order to be open to any new data/theory. Using APE’s is a petty way to misdirect the conversation and mask one's lack of knowledge.
The APE Problem
If I evaluate a theory based solely on APE’s, then I ignore many categories of data that are better suited to evaluate that theory.
Silent Evidence
The first data that APEists will systematically overlook is the silent evidence. Nassim Nicholas Taleb talks about this in his excellent book "The Black Swan".
Taleb, on the other hand, thinks it takes more than a good track record to be considered a good trader. Indeed, if finance is a very uncertain environment (which may be the case), and if a trader has a one in two chance of having a positive balance sheet for a given year. Then the trader has a 3% chance of having a good balance sheet over 5 successive years. There are about ten million traders in the world.
Therefore, if finance is an entirely random world where no one can make decisions that are better than random decisions, then there will be 300 thousand traders who were successful during the last five years on Earth. And it wouldn't have anything to do with skill.
Silent evidence is the hidden data that allows a theory to be evaluated. For example, the fact that important people get up at 5:00 in the morning on average does not mean that I increase my chances of becoming important by getting up at 5:00. In fact, do ALL people who get up at 5:00 a.m. have more success? Have people who started waking up at 5:00 a.m. in the middle of their lives seen their lives change?
When you hear an isolated number (which can be as tricky as an APE), then you have to play devil's advocate to try to find other reasons for that number. It's the only way to fight the traps.
The ‘stork effect’
The ‘stork effect’ (‘l’effet cigogne’ in french) is the name given to the confusion between correlation and causality. For the previous example, another explanation for the fact that important people get up early could be that, since they became, they have a busier schedule that forces them to get up early (reverse causality). Or they are very energetic and efficient people. This trait causes them to be successful AND to get up early (mediating variable). It is very difficult to characterize the causal link between two phenomena. The scientific methodology provides safeguards that are necessary for a good understanding of the world. For the causality between early wake-up and success, for example, a good way to study it would be to take a large, random and representative sample of people, make them get up early for a long time, and compare their success with a control group that does not change its habits.
Scientists have brought to light a very wide variety of biases and frequent errors (Simpsons' paradox, the regression towards the mean, the Berkson's paradox...) and it is very difficult even for them to successfully counter them all in their studies.
Conclusion: an impressionistic vision
I like to think of a theory about the world as an impressionist painting. I need to see the painting in its entirety to understand it. For example, I think that the financialization of the economy and the quest for profit has negative consequences for society:
- I've seen many documentaries and reports detailing the impact of profit and shareholding on the health, safety, privacy of citizens (Fed up, Cash investigation, ‘Thank you, Boss!’ , ...).
- I know many statistics on the subject (the financial crisis of 2007 cost 1600 dollars per human being on Earth,the net contribution of equity markets to the financing of the economy has become negative, , fraud and tax optimization cost France several tens of thousands of dollars..)
- The list of ecological, financial and health scandals due to ethical failures on the part of the corporate world is (very) long ...
I don't know if what I think is true or not. But the list of data I have acquired during my life on the subject leads me to believe that there is a greater chance (at least twice as much) that a less globalized and less financialized world is more sustainable than a world that is. It's a fuzzy picture, but one whose contours are a bit drawn in my mind.
Moreover, in this picture, the arguments in favour of the financialized economy have to must be present. For example, the economy as we know it has, in my opinion, contributed - on average - to improving the general well-being of humanity and to the appearance of many inventions.
All these points need to be meditated upon and structured in my mind. And together they depict parts of my impressionistic picture representing my vision of the impact of the financialization of the economy on the world.
In this painting, an APE (an isolated piece of data or an isolated anecdote) is a pixel.
Alone, it has no value. It takes hundreds of dots to start distinguishing contours (hence the need for empirical studies) and these contours are clearer and harder to erase if they are scientifically based arguments.
Moralism: The Pandemic at the origin of APE’s
We are rarely willing to open our minds and change our minds in a debate, often because ego issues come into play. Brad Blanton's work is very important in this regard, and will better address the irresistible need to cling to APE’s, the ultimate resort to ego preservation.
(to be continued)
Edit : thanks to David Bagatta for the meticulous proofreading <3