Archive for the ‘Thinking’ Category
About a month ago The Age published an opinion piece I wrote under the title “Do you hold a Bayesian or Boolean worldview?“. I had submitted it under the title “Madmen in Authority,” and it opened by discussing two men in authority who are/were each mad in their own way – Maurice Newman, influential Australian businessman and climate denier, and Cuban dictator Fidel Castro. Both men had professed to be totally certain about issues on which any reasonable person ought to have had serious doubts given the very substantial counter-evidence.
Their dogmatic attitudes seemed to exemplify a kind of crude epistemological viewpoint I call “Booleanism,” in contrast with a more sophisticated “Bayesianism”. Here is the philosophical core of the short opinion piece:
On economic matters, Keynes said: “Practical men who believe themselves to be quite exempt from any intellectual influence, are usually the slaves of some defunct economist. Madmen in authority, who hear voices in the air, are distilling their frenzy from some academic scribbler of a few years back.”
Similarly, on matters of truth and evidence, we are usually unwittingly beholden to our background epistemology (theory of knowledge), partially shaped by unknown theorists from centuries past.
One such theory of knowledge we can call Boolean, after the 19th century English logician George Boole. He was responsible for what is now known as Boolean algebra, the binary logic which underpins the computing revolution.
In the Boolean worldview, the world is organised into basic situations such as Sydney being north of Melbourne. Such situations are facts. Truth is correspondence to facts. That is, if a belief matches a fact, it is objectively true; if not, it is objectively false. If you and I disagree, one of us must be right, the other wrong; and if I know I’m right, then I know you’re wrong. Totally wrong.
This worldview underpins Castro’s extreme confidence. Either JFK was killed by an anti-Castro/CIA conspiracy or he wasn’t; and if he was, then Castro is 100 per cent right. Who needs doubt?
An alternative theory of knowledge has roots in the work of another important English figure, the Reverend Thomas Bayes. He is famous for Bayes’ Theorem, a basic law of probability governing how to modify one’s beliefs when new evidence arrives.
In the Bayesian worldview, beliefs are not simply true or false, but more or less probable. That is, we can be more or less confident that they are true, given how they relate to our other beliefs and how confident we are in them. If you and I disagree about the cause of climate change, it is not a matter of me being wholly right and you being wholly wrong, but about the differing levels of confidence we have in a range of hypotheses.
Scientists are generally Bayesians, if not self-consciously, at least in their pronouncements. For example, the IPCC refrains from claiming certainty that climate change is human-caused; it says instead that it has 95 per cent confidence that human activities are a major cause.
Some people excel at critical thinking; others, not so much. Scientist Carl Sagan and investor Charlie Munger are oft-mentioned exemplars; my friend and colleague Paul Monk is less famous but also impressively sharp. On the other side we have… well, Homer Simpson can stand in for all those it would be rude to name.
But what makes a thinker more highly critical than others? And how can any person lift their game? This can be explored through the notion of habits. Highly critical thinkers have developed many habits which help them think more effectively. With sufficient commitment and patience, and perhaps a little coaching, such habits can be acquired by the rest of us.
This post describes seven major habits of highly critical thinkers. The list is obviously inspired by the hugely successful book about highly effective people. Whatever one might think of that book, if a similar exercise for critical thinking could have even a tiny fraction of its impact, it would be well worth undertaking.
Everybody is familiar with the term “critical thinking,” and has a reasonable working sense of what it is, but there is much disagreement about its proper definition. There’s no need to enter that quagmire here. Suffice to say that critical thinking, for current purposes, is truth-conducive thinking, i.e., thinking that leads to correct or accurate judgements. It is, in a phrase I like to use, the art of being right – or at least, of being more right more often.
But what kind of thinking conduces to truth? What is this subtle art? Back in the early seventeenth century, the philosopher Francis Bacon characterised it this way:
For myself, I found that I was fitted for nothing so well as for the study of Truth; as having a mind nimble and versatile enough to catch the resemblances of things … and at the same time steady enough to fix and distinguish their subtler differences; as being gifted by nature with desire to seek, patience to doubt, fondness to meditate, slowness to assert, readiness to consider, carefulness to dispose and set in order; and as being a man that neither affects what is new nor admires what is old, and that hates every kind of imposture.
Four hundred years later, political scientist Philip Tetlock conducted extensive and rigorous studies of hundreds of experts in the political arena, focusing on their ability to forecast. He found that the experts fell into two main groups:
One group of experts tended to use one analytical tool in many different domains; they preferred keeping their analysis simple and elegant by minimizing “distractions.” These experts zeroed in on only essential information, and they were unusually confident—they were far more likely to say something is “certain” or “impossible.” In explaining their forecasts, they often built up a lot of intellectual momentum in favor of their preferred conclusions. For instance, they were more likely to say “moreover” than “however.”
The other lot used a wide assortment of analytical tools, sought out information from diverse sources, were comfortable with complexity and uncertainty, and were much less sure of themselves—they tended to talk in terms of possibilities and probabilities and were often happy to say “maybe.” In explaining their forecasts, they frequently shifted intellectual gears, sprinkling their speech with transition markers such as “although,” “but,” and “however.”
The second group, the “foxes,” were better forecasters than the first, the “hedgehogs.” Foxy thinking, it seems, is more truth-conducive than hedgehoggery.
Two points jump out from these quotes. First, the two accounts have much in common, underneath the differences in style. The essence of critical thinking is largely stable across the centuries.
Second, they are both describing what good thinkers tend to do. Theorists of critical thinking have various ways of thinking about these tendencies; some talk of dispositions, others of virtues. Here I take what may be a novel approach and consider them as acquirable habits.
A habit is just a propensity to take actions of a certain kind in a relatively automatic or reflexive manner. And as we all know, and as elaborated in the recent book by Charles Duhigg, The Power of Habit, good habits can be cultivated, and bad habits overcome. So the goal here is to list:
- propensities to do things of certain kinds more or less automatically under appropriate circumstances; which propensities are
- possessed by highly critical thinkers much more often than by ordinary folk, and which
- help them to make more correct or accurate judgements, and
- could be picked up, or further developed, by any ordinary person with a reasonable amount of effort; with the result that
- they would themselves become more critical.
The habits described below are the kinds of things highly critical thinkers really do do. They are not merely prescriptions or guidelines which would help anyone to be more critical if anyone were disciplined or virtuous enough to follow them.
To illustrate: Blogger Shane Parrish reports that a hedge fund manager and author, Michael Maubousson, asked the Nobel-winning psychologist Daniel Kahneman what a person should do to improve their thinking. “Kahneman replied, almost without hesitation, that you should go down to the local drugstore and buy a very cheap notebook and start keeping track of your decisions.”
Now, it is plausible that keeping track of your decisions in a notebook would improve your thinking. However, it is not a habit of highly critical thinkers, at least in my experience. I don’t recall ever observing a highly critical thinker doing it, or hearing one say they do it. I don’t even do it myself, even after hearing the great Laureate’s advice (and apparently Maubousson doesn’t either).
And so to the habits themselves.
1. Judge judiciously
One of the most salient thinking traps is, in the common phrase, jumping to conclusions. Highly critical thinkers have cultivated four main habits which help them avoid this.
First, they tend to delay forming a judgement until the issue, and the considerations relevant to it, have been adequately explored, and also until any hot emotions have settled (Bacon’s “slowness to assert”).
Second, they tend to abstain altogether from making any judgement, where there are insufficient grounds to decide one way or another. They feel comfortable saying, or thinking, “I don’t know.”
Third, when they do make a judgement, they will treat it as a matter of degree, or assign a level of confidence to it, avoiding treating any non-trivial issue as totally certain.
And fourth, they treat their judgements as provisional, i.e., made on the basis of the evidence and arguments available at the time, and open to revision if and when new considerations arise.
2. Question the questionable
Much more often than ordinary folk, highly critical thinkers question or challenge what is generally accepted or assumed. Sometimes they question the “known knowns” – the claims or positions which constitute widely-appreciated truths. Other times, they target the implicit, the invisible, the unwittingly assumed.
Highly critical thinkers do not of course question everything. They are not “radical skeptics” doubting all propositions (as if this was even possible anywhere other than in philosophical speculation). Rather, they tend to be selective or strategic in their questioning, targeting claims or positions that are worth challenging, whether in some practical or intellectual sense. They are skilled in identifying or “sniffing out” the “questionable,” i.e. claims which are potentially vulnerable, and whose rejection may have important or useful implications.
3. Chase challenges
We all know that feeling of instant irritation or indignation when somebody dares to suggest we might be wrong about something. Highly critical thinkers have cultivated various habits counteracting this reaction – habits which actually lead to them being challenged more often, and benefiting more from those challenges.
For example, while we mostly seek and enjoy the company of those who share our views, highly critical thinkers make an effort to engage those of a contrary opinion, tactfully eliciting their objections. And when fielding such challenges, highly critical thinkers resist the instinct to ignore, reject or rebut. They will be found doing such seemingly perverse things as rephrasing the objections to be sure of understanding them, or even to render them even more powerful. Charlie Munger is quoted as saying “I never allow myself to have an opinion on anything that I don’t know the other side’s argument better than they do.”
Another habit of highly critical thinkers is reading widely, and especially reading from sources likely to present good quality contrary views and arguments. Finding themselves drawn to a position (e.g., that William Shakespeare of Stratford was unlikely to have been the author of the works attributed to him) highly critical thinkers seek out the best presentations of the orthodox position. In short, they strive to test or “prove” their views, rather than support or defend them.
4. Ascertain alternatives
Highly critical thinkers are always mindful that what they see before them may not be all there is. They habitually ask questions like: what other options are there? What have we missed? As opposed to/compared with what? They want to see the full range of relevant alternatives before passing judgement.
For example, when considering a difficult decision, they put extra effort into searching for – or creating – courses of action outside the standard, provided or obvious ranges. When trying to explain why something happened, they will allocated more time than most people do to expanding the range of hypotheses under consideration. In a negotiation, they seek to develop new, mutually acceptable solutions rather than “horse-trading” on existing positions.
5. Make use of methods
When considering a course of action, a critical thinker of my acquaintance, who happened to be successful banker and company director, said she always asked herself two simple questions: (1) what’s the worst thing that could happen here? and (2) what’s the best thing that could happen? The first question prompts us to search for potential drawbacks a bit more thoroughly than we might otherwise have done. The routine amounts to a rudimentary (or “fast and frugal”) risk analysis.
This example illustrates how highly critical thinkers habitually deploy suitable methods to structure their thinking and improve the conclusions. Another example: in psychology department colloquiua I used to attend, participants, after hearing a colleague present their work, would reflexively use a method I call scenario testing. This involves diligently and creatively searching for scenarios in which their colleague’s conclusions are false, even though their premises (data) are true. To the extent that plausible scenarios of this kind can be identified, the inferences from the premises to the conclusions are suspect.
There are literally scores of methods one might use. Some, like the rudimentary risk analysis mentioned above, are simple and informal, and can be quickly learned and exploited by almost anyone. Others are elaborate, technical and may require specialist training (e.g., rigorous argument mapping, or full quantitative risk analysis). Generally, the more sophisticated the method, the less widely it is used, even by the most highly critical thinkers. Every such thinker has built up their own repertoire of methods. What’s most important is not so much their particular selection, but the fact that they habitually deploy a wider range of methods, more often, than ordinary folk.
6. Take various viewpoints
Highly critical thinkers well understand that their view of a situation is unique, partial and biased, no matter how clear, compelling and objective it seems. They understand that there will always be other perspectives, which may reveal important aspects of the situation.
Of course, most people appreciate these points to some degree. The difference is that highly critical thinkers are especially keen to profit from a more complete understanding, and so have cultivated various habits of actually occupying, as best they can, those other viewpoints, so as to see for themselves what additional insights can be gained.
One such habit is trying to “stand in the shoes” of a person with whom we may have some conflict, or are inclined to criticise. Another is to adopt the persona of a person, perhaps a hypothetical person, who strongly disagrees with your views, and to argue against yourself as strongly as they would. A third (relatively rare) is to take the perspective of your future self, having found out that your current position turned out to be wholly, and perhaps disastrously, wrong. What do you see, from the future, that you are missing now?
7. Sideline the self
People tend to be emotionally attached to views. Core beliefs, such as provided by religions or ideologies, help provide identity, and the comforts of clarity and certainty. Sometimes pride binds us to positions; having publicly avowed and defended them previously, it would be humiliating to concede we were wrong. Highly critical thinkers have habits which help to sever these emotional bonds between self and beliefs, allowing the thinker to discard or modify beliefs as indifferently as a used car dealer will trade vehicles. Highly critical thinkers have in other words learned how to sideline the self, removing it from the field of epistemic play.
One habit is to avoid verbally identifying oneself with positions by using distancing locutions. Instead of saying things like “Its obvious to me that…” they will say things like “one plausible position is that”. A similar technique is to give positions names. Instead of boldly asserting that Shakespeare must have written the works, publicly committing yourself to this view, say “According to the Stratfordian view…”.
To be continued…
This post is already much longer than originally intended, but still leaves much unsaid. A few quick final points:
- The current list can’t claim to be definitive. Others may well come up with different lists.
- It is also a work in progress. I hope to elaborate each of the major habits in separate posts.
- Clearly much more could be said about the notion of a habit, and the somewhat paradoxical character of critical thinking habits, which generally involve automatically (“without thinking about it”) engaging in thinking activities.
- This list is not based on rigorous empirical research, though in places it is informed by such research. There is much scope for scientific clarification here. Tetlock’s studies provide an impressive model.
Comments are most welcome.
Apparently horses in races are almost always (98%) whipped.* The main reason is to make them go faster.
Congratulations to the scientists from the University of Sydney who won a prize for discovering that “whipping does not increase horses’ chances of finishing in the top three and that they actually run faster when they are not being hit.”
So it seems that overwhelmingly, the horse racing experts – the jockeys and trainers particularly – have for decades followed a practice which:
- Hurts their chances of winning.
- Hurts their beloved animals.
- Hurts their standing in the wider community.
Given how much money is at stake in horse racing, this is remarkable. It calls out for explanation. How can these experts have persisted for so long in such self-destructive behavior? Here are some possible explanations.
First, the idea that whipping makes a horse go faster has a certain “truthiness.” This truthiness made questioning the practice seem otiose. Further, belief in the truthy proposition became a perceptual filter through which they “saw” that whipping made horses go faster.
Second, the omnipresence of whipping prevented the possibility of observing the weak correlation between whipping and losing. If only about half of the jockeys whipped their horses, then over time people might have started to notice that the unwhipped horses tended to win a bit more often. Or at least didn’t lose more often.
Third, since everyone else was whipping, everyone naturally assumed that whipping was the “right” thing to do. Failing to whip the horse would look rather odd – especially if you didn’t win. As Mark Twain wrote: “We are creatures of outside influences; as a rule we do not think, we only imitate.”
* I don’t know anything about horse racing. This claim seems implausible, but here is my source.
For one could easily conceive of a machine that is made in such a way that it utters words, and even that it would utter some words in response to physical actions that cause a change in its organs—for example, if someone touched it in a particular place, it would ask what one wishes to say to it, or if it were touched somewhere else, it would cry out that it was being hurt, and so on. But it could not arrange words in different ways to reply to the meaning of everything that is said in its presence, as even the most unintelligent human beings can do.
While it is possible to imagine a machine obtaining a perfect score on the SAT or winning Jeopardy–since these rely on retained facts and the ability to recall them–it seems far less possible that a machine can weave things together in new ways or to have true imagination in a way that matches everything people can do, especially if we have a full appreciation of the creativity people are capable of.
Almost everyone agrees that critical thinking skills are important. Almost everyone agrees that it is worth investing effort (in education, or in workplace training) to improve these skills. And so it is rather surprising to find that there is, in the academic literature, little clarity, and even less consensus, about one of the most basic questions you’d need answered if you wanted to generate any sort of gains in critical thinking skills (let alone generate those gains cost-effectively); viz., how are critical thinking skills acquired?
Theories on this matter come in five main kinds:
- Formal Training. CT skills are simply the exercise of generic thinking power which can be strengthened by intensive training, much as general fitness can be enhanced by running, swimming or weightlifting. This approach recommends working out in some formal ‘mental gym’ such as chess, mathematics or symbolic logic as the most convenient and effective way to build these mental muscles.
- Theoretical Instruction. CT skills are acquired by learning the relevant theory (logic, statistics, scientific method, etc.). This perspective assumes that mastering skills is a matter of gaining the relevant theory. People with poor CT poor skills lack only a theoretical understanding; if they are taught the theory in sufficient detail, they will automatically be able to exhibit the skills, since exhibiting skills is just a matter of following explicit (or explicable) rules.
- Situated Cognition. CT is deeply tied to particular domains and can only be acquired through properly “situated” activity in each domain. Extreme versions deny outright that there are any generic CT skills (e.g. McPeck). Moderate versions claim, more plausibly, that increasingly general skills are acquired through engaging in domain-specific CT activities. According to the moderate version general CT skills emerge gradually in a process of consolidation and abstraction from particular, concrete deployments, much as general sporting skills (e.g., hand-eye coordination) are acquired by playing a variety of particular sports in which those general skills are exercised in ways peculiar to those sports.
- Practice sees CT skills as acquired by directly practicing the general skills themselves, applying them to many particular problems within a wide selection of specific domains and contexts. The Practice perspective differs from Formal Training in that it is general CT skills themselves which are being practiced rather than formal substitutes, and the practice takes place in non-formal domains. It differs from Situated Cognition in that it is practice of general skills aimed at improving those general capacities, rather than embedded deployment of skills aimed at meeting some specific challenge within that domain.
- Evolutionary Psychology views the mind as constituted by an idiosyncratic set of universal, innate, hard-wired cognitive capacities bequeathed by natural selection due to the advantages conferred by those capacities in the particular physical and social environments in which we evolved. The mind does not possess and cannot attain general-purpose CT skills; rather, it can consolidate strengths in those particular forms or patterns of thinking for which evolution has provided dedicated apparatus. Cultivating CT is a matter of identifying and nurturing those forms.
Formal training is the oldest and most thoroughly discredited of the perspectives. It seems now so obvious that teaching latin, chess, music or even formal logic will have little or no impact on general critical thinking skills that it is hard to understand now how this idea could ever have been embraced. And we also know why it fails: it founders on the rock of transfer. Skills acquired in playing chess do not transfer to, say, evaluating political debates. Period.
Theoretical Instruction has almost as old a philosophical pedigree as Formal Training. It has been implemented in countless college critical thinking classes whose pedagogical modus operandi is to teach students “what they need to know” to be better critical thinkers, by lecturing at them and having them read slabs out of textbooks. Token homework exercises are assigned primarily as a way of assessing whether they have acquired the relevant knowledge; if they can’t do the exercises, what they need is more rehearsing of theory. As you can probably tell from the tone of this paragraph, I believe this approach is deeply misguided. The in-depth explanation was provided by philosophers such as Ryle and Heidegger who established the primacy of knowledge-how over knowledge-that, of skills over theory.
Current educational practice subscribes overwhelmingly (and for the most part unwittingly) to the moderate version of Situated Cognition. That is, we typically hope and expect that students’ general CT skills will emerge as a consequence of their engaging in learning and thinking as they proceed through secondary and especially tertiary education studying a range of particular subjects. However, students generally do not reach levels of skill regarded as both desirable and achievable. As Deanna Kuhn put it, “Seldom has there been such widespread agreement about a significant social issue as there is reflected in the view that education is failing in its most central mission—to teach students to think.” In my view the weakness of students’ critical thinking skills, after 12 or even 16 years of schooling, is powerful evidence of the inadequacy of the Situated Cognition perspective.
There may be some truth to the Evolutionary Psychology perspective. However in my view the best argument against it is the fact that another perspective – Practice – actually seems quite promising. The basic idea behind it is very simple and plausible. It is a truism that, in general, skills are acquired through practice. The Practice perspective simply says that generic critical thinking skills are really just like most other skills (that is, most other skills that are acquired, like music or chess or trampolining, rather than skills that are innate and develop naturally, like suckling or walking).
In our work in the Reason Project at the University of Melbourne we refined the Practice perspective into what we called the Quality (or Deliberate) Practice Hypothesis. This was based on the foundational work of Ericsson and others who have shown that skill acquisition in general depends on extensive quality practice. We conjectured that this would also be true of critical thinking; i.e. critical thinking skills would be (best) acquired by doing lots and lots of good-quality practice on a wide range of real (or realistic) critical thinking problems. To improve the quality of practice we developed a training program based around the use of argument mapping, resulting in what has been called the LAMP (Lots of Argument Mapping) approach. In a series of rigorous (or rather, as-rigorous-as-possible-under-the-circumstances) studies involving pre-, post- and follow-up testing using a variety of tests, and setting our results in the context of a meta-analysis of hundreds of other studies of critical thinking gains, we were able to establish that critical thinking skills gains could be dramatically accelerated, with students reliably improving 7-8 times faster, over one semester, than they would otherwise have done just as university students. (For some of the detail on the Quality Practice hypothesis and our studies, see this paper, and this chapter.)
So if I had to choose one theory out of the five on offer, I’d choose Practice. Fortunately however we are not in a forced-choice situation. Practice is enhanced by carefully-placed Theoretical Instruction. And Practice can be reinforced by Situated Cognition, i.e. by engaging in domain-specific critical thinking activities, even when not framed as deliberate practice of general CT skills. As one of the greatest critical thinkers said in one of the greatest texts on critical thinking:
“Popular opinions, on subjects not palpable to sense, are often true, but seldom or never the whole truth. They are a part of the truth; sometimes a greater, sometimes a smaller part, but exaggerated, distorted, and disjoined from the truths by which they ought to be accompanied and limited.”
Daddy, is this true? If there were giants, then a football to them would be the same size as a pea to us.
That was, word for word, a completely out-of-the blue utterance by our 6 year old daughter, Lillian.
Her “if…then” construct is what is known as a counter-factual conditional – If [something that is not in fact true] then [something else that is not in fact true], and to correctly construct such conditionals you use subjunctive verb forms (were… would be…).
I’m amazed that somehow, without any explicit instruction at all, Lillian can spontaneously express counterfactual conditionals with perfectly grammatical sentences of complex construction. (In saying this, I’m not implying that I think Lillian is somehow especially advanced. I assume she’s showing a normal developmental progression. It is that progression which is amazing.)
But what really amazes me is that somehow, without instruction, she’s acquired the conceptual capacity to talk about the truth value of counterfactual conditions. (Aside: in some of our workshops we teach professionals such as intelligence analysts to try to avoid talking (i.e. thinking) about truth of conditionals, where there are regular alternatives, since it is so much more cognitively demanding than thinking about regular statements.)
No wonder linguists have argued that humans have a kind of innate capacity for language acquisition.
[repost of a shorter version posted on another blog]