TL;DR – We often don't think thoughts entirely through, which leads to erroneous conclusions. We are too often intellectually lazy, and accept the first possible explanation. Awareness and simple techniques can aid us in understanding and countering these errors.
When we stop wondering
Does any of the following sound familiar?
- You always find your keys in the last place you look. Obviously, that is where you stop looking, because you have your keys.
- If you ask people their top-10 favorite songs, most people are not able to give you their true top-10 list. Instead, they will answer with a list of the first 10 good songs that enter their minds, put in order of preference.
- Ask people for the most important risks they feel threaten by, and they will likely give you a list of the first 10 important risks that enter their minds.
This is the easy way out, and we do this with our thoughts as well. We stop wondering about a problem when we come up with a plausible explanation.
"Good enough" explanations
My point? This is sloppy thinking. Instead of thinking our thoughts through, we end up with a “good enough” explanation. The type of explanation that is easy to swallow but lacks nuance and is often quite wrong. It's self-deceit because we are intellectually lazy.
Let's look at one concrete example, a study on immigration and its conclusions
An example: Belgian results of global views on immigration
Ipsos Public Affairs updated their “Global Views on Immigration” study in which they track attitudes of people in 24 countries on immigration. They’ve been doing this since 2011. What was interesting about the study was the way in which the media reported on it. A few headlines and quotes:
“Belgians have a very negative attitude about immigration” (http://deredactie.be/cm/vrtnieuws/binnenland/1.2410072)
“Research points to Belgium as one of the most migrant-unfriendly countries”
“Only 12% of Belgians believe that migrants have a positive impact on the development of our country”
Each of these quotes could be considered correct by looking at the underlying study, a presentation of which you can find if you follow this link (http://www.ipsos-na.com/download/pr.aspx?id=14710) And each of these quotes won’t tell but half of the story … there is another, opposite angle that could have been taken as well. A few examples:
- Belgians respondents' perception of the positive nature of the impact of migration has increased significantly, from 9% in 2011 to 12% in 2015, or a 25% increase.
- 19% of Belgian respondents believe immigration is good for the economy, up from 13% in 2013, or a more than 30% increase.
- More Belgian respondents believe that immigration makes our country a more interesting place to live (20%, up from 16% in 2013)
- As compared to 2011, 12% less of the Belgian respondents believe that immigration puts too much pressure on public services …
And I could continue in that vein for a while. That is not the point. I’m explicitly taking the opposite position based on the available information. These positions, though opposite, are as valid as the ones arrived at by the media. They may not be as catchy or "in tune with the times", but they are supported by the data.
Data is not information, knowledge nor wisdom
What we need to do is to think this through. Because data is just that, data. It is not yet information, certainly not knowledge and very far from wisdom.
Now, how can we go about solving this problem of focusing on the first, most accessible, most visible side of an issue, rather than trying to get a good grip on the whole? With data, that is rather simple: take the time to look at the whole of the entire available data set. We need to first understand the data and what it could possibly represent.
What does data mean? The brainstorming approach
But how do we get from the data to that more in-depth understanding? There are many ways, but one way is an approach often followed in good brainstorming exercises. A group working to understand a data set choses a facilitator. This facilitator asks the other group members to come up with as many ideas as possible on how to interpret the data. The participants are only allowed to generate ideas, they cannot challenge, question or give criticism. At the end of the exercise, the group will have gathered a multitude of possible explanations of the data set available to them. Examining all possible explanations can help to broaden their thinking when working with the data set.
Let's apply this to our example. Had the journalists brainstormed the possible explanations, they would have become aware that there were alternative explanations for the same data set. But are we sure they did not do it? No, but if they would've been entirely transparent, they would have at least offered that there were other explanations that could be arrived at with the same data set, which they disregarded for certain, to be explained, reasons. That is a transparent approach.
Limitations and next steps
Now, of course the approach has its limitations. For example, it does nothing to deepen thinking about a problem. Deepening occurs after the brainstorming, when you start to cluster the information gathered throughout the exercise. Let's examine this approach in a bit more detail:
By generating a large set of possible explanations for a data set, it is likely there will be clusters of possible explanations: explanations that are not entirely the same, but are very similar. Clustering these together and examining them as a group of possible explanations, the interactions between factors influencing the data set will become clearer. A narrative of causes and effects will become clearer. Hence, we are deepening our understanding.
Clustering in practice
From a practical point of view, this process will involve a lot of individual reflection but also a lot of discussion and exchanges. Which are likely to create an even better understanding.
This exchange requires any one that analyses a data set to be open as to their interpretation and the reasons why they interpret data like they do. If you cannot offer a rational explanation on why you interpret data as you do, you are only expressing a personal opinion, which is not necessarily supported by underlying fact.
Interpreting data sets of any kind requires more thinking than most people are used to doing. They see a data set and they think that their interpretation is the only possible one. But we need to be more concise in our thinking. We need to allow for alternatives to clarify our own thinking. And we need to be transparent about the basis for our conclusions. This requires an open exchange with other experts. And that is often neglected because of the need to be first with an interpretation, but necessarily to be right with an explanation.