Archive for the ‘Teaching’ Category

I’ve had the following abstract accepted for a presentation at a conference in December at the University of Melbourne, Higher Education Research & the Student Learning Experience in Business.

A Pragmatic Definition of Critical Thinking for Business

This presentation will lay out a pragmatic definition of critical thinking.  It doesn’t purport to be the definitive characterization of what critical thinking is. Rather, it is offered as a convenient framework for understanding the nature and scope of critical thinking, which may be useful for purposes such as developing a dedicated subject in critical thinking for business, improving the teaching of critical thinking within existing subjects, or evaluating the effectiveness of a business course in developing critical thinking.

The definition is constructed around five commitments:

    • First, the essence of critical thinking is correct or accurate judgement. That is, to think critically is to think in ways that are conducive to being “more right more often” when making judgements.
    • Second, “being more right more often” can be achieved through the skillful application of general thinking methods or techniques.
    • Third, these techniques range on a spectrum from the simple and easily acquired to technical methods which require special training.
    • Fourth, for all but the simplest of methods, there are degrees of mastery in application of these techniques.
    • Fifth, there are many different kinds of judgements made in business, including decision making, prediction, estimation, (causal) explanation, and attribution of responsibility. For each major type of judgement, there are typical pitfalls, and a range of critical thinking methods which can help people avoid or compensate for those pitfalls.

These commitments enable us to define a kind of three-dimensional chart representing the critical thinking competency of any individual. Along one (categorical) axis is the various kinds of judgements (decision making, etc.). Another axis represents the spectrum from simple through to advanced critical thinking methods. Particular methods can then be placed in appropriate “boxes” in the grid defined by these axes. A person will have a degree of mastery of the methods in each box; this can be represented on a third dimension. A person’s critical thinking competency is thus a distinctive “landscape” formed by the varying levels of mastery.

This characterisation is tailoring, for business, a more general pragmatic approach to understanding critical thinking.  About a year ago I developed this approach in preparation for a workshop in the US on development of a test of critical thinking for intelligence analysts; my role in the workshop was to lay out a general framework for understanding what critical thinking is.   That approach was described in a manuscript Dimensions of Critical Thinking.

I’m also supporting a team from the University of Sydney Business School, who have had the following abstract accepted:

Evaluating critical thinking skill gains in a business subject

Helen Parker, Leanne Piggott, Lyn Carson
University of Sydney Business School
Tim van Gelder
University of Melbourne and Austhink Consulting

Critical thinking (CT) is one of the most valued attributes of business school graduates, and many business school subjects claim to enhance it. These subjects frequently implement pedagogical strategies of various kinds aimed at improving CT skills. Rarely however are these efforts accompanied by any rigorous evaluation of CT skill gains. But without such evaluation, it is difficult to answer questions such as:

    • Are our students’ CT skills in fact improving? By how much?
    • Are those skills improving more than they would have even without our special CT instruction?
    • Are the marginal gains worth the cost?
    • Are our attempts to improve our instruction from semester to semester making any difference?

These kinds of questions are particularly relevant to the University of Sydney Business School, which has an entire subject dedicated to improving CT (BUSS5000 – Critical Thinking in Business), enrolling some 800 students per semester. Consequently, in 2013, the Business School embarked on a large-scale, multi-year evaluation program. The evaluation is based on pre- and post-testing using an independent objective test (the Halpern Critical Thinking Assessment), whose coverage overlaps with the range of critical thinking skills taught in the subject. This presentation will give an overview of the approach it has adopted. It will discuss some of the challenges and pitfalls in the testing process, and how to interpret results. Finally, it will present data and insights from the first semester of full-scale evaluation. The session should be of interest to anyone interested in evaluating CT skills, or more generally in how business school education can enhance CT.

There’s an obvious complementarity between these two topics.


Read Full Post »

Fernando Leal and colleagues at the University of Guadalajara have released Introducción a la Teoría de la Argumentación, an integrated selection of pieces intended to assist students and their teachers to focus on argumentation when reading and writing academic texts.

The section Parte II: La téchnica de mapeo de argumentos (argument mapping) contains three pieces emerging from work at the University of Melbourne and Austhink:

  • A translation of my article Teaching Critical Thinking: Some Lessons from Cognitive Science
  • A translation of Enhancing our Grasp of Complex Arguments, by Paul Monk and me, a big-picture view of why complex arguments are cognitively challenging and how argument mapping can help.  It has been available as a manuscript since 2004 and been well-received so we are very happy to see it finally appear in print.
  • A new chapter  by Claudia María Álvarez Ortiz, ¿El estudio de le filosofia mejora las habilidades de pensamiento crítico? which extracts some core material from her MA thesis Does Philosophy Improve Reasoning Skills?.  This is the first proper publication of the very important meta-analysis of studies of gains in critical thinking at college.

Well-designed and attractively produced, the appearance of the volume is a significant development in critical thinking pedagogy and theory, particularly in the Spanish-speaking Americas.   Regrettably the language barrier will be a major hurdle to recognition and uptake in the Anglosphere.  Perhaps somebody should undertake a translation of the whole volume into English?

Read Full Post »

Almost everyone agrees that critical thinking skills are important.  Almost everyone agrees that it is worth investing effort (in education, or in workplace training) to improve these skills.   And so it is rather surprising to find that there is, in the academic literature, little clarity, and even less consensus, about one of the most basic  questions you’d need answered if you wanted to generate any sort of gains in critical thinking skills (let alone generate those gains cost-effectively); viz., how are critical thinking skills acquired?

Theories on this matter come in five main kinds:

  • Formal Training. CT skills are simply the exercise of generic thinking power which can be strengthened by intensive training, much as general fitness can be enhanced by running, swimming or weightlifting.  This approach recommends working out in some formal ‘mental gym’ such as chess, mathematics or symbolic logic as the most convenient and effective way to build these mental muscles.
  • Theoretical Instruction. CT skills are acquired by learning the relevant theory (logic, statistics, scientific method, etc.).  This perspective assumes that mastering skills is a matter of gaining the relevant theory.  People with poor CT poor skills lack only a theoretical understanding; if they are taught the theory in sufficient detail, they will automatically be able to exhibit the skills, since exhibiting skills is just a matter of following explicit (or explicable) rules.
  • Situated Cognition. CT is deeply tied to particular domains and can only be acquired through properly “situated” activity in each domain.  Extreme versions deny outright that there are any generic CT skills (e.g. McPeck).  Moderate versions claim, more plausibly, that increasingly general skills are acquired through engaging in domain-specific CT activities.  According to the moderate version general CT skills emerge gradually in a process of consolidation and abstraction from particular, concrete deployments, much as general sporting skills (e.g., hand-eye coordination) are acquired by playing a variety of particular sports in which those general skills are exercised in ways peculiar to those sports.
  • Practice sees CT skills as acquired by directly practicing the general skills themselves, applying them to many particular problems within a wide selection of specific domains and contexts.  The Practice perspective differs from Formal Training in that it is general CT skills themselves which are being practiced rather than formal substitutes, and the practice takes place in non-formal domains.  It differs from Situated Cognition in that it is practice of general skills aimed at improving those general capacities, rather than embedded deployment of skills aimed at meeting some specific challenge within that domain.
  • Evolutionary Psychology views the mind as constituted by an idiosyncratic set of universal, innate, hard-wired cognitive capacities bequeathed by natural selection due to the advantages conferred by those capacities in the particular physical and social environments in which we evolved.  The mind does not possess and cannot attain general-purpose CT skills; rather, it can consolidate strengths in those particular forms or patterns of thinking for which evolution has provided dedicated apparatus.  Cultivating CT is a matter of identifying and nurturing those forms.

Formal training is the oldest and most thoroughly discredited of the perspectives.   It seems now so obvious that teaching latin, chess, music or even formal logic will have little or no impact on general critical thinking skills that it is hard to understand now how this idea could ever have been embraced.   And we also know why it fails: it founders on the rock of transfer.  Skills acquired in playing chess do not transfer to, say, evaluating political debates.  Period.

Theoretical Instruction has almost as old a philosophical pedigree as Formal Training.  It has been implemented in countless college critical thinking classes whose pedagogical modus operandi is to teach students “what they need to know” to be better critical thinkers, by lecturing at them and having them read slabs out of textbooks.   Token homework exercises are assigned primarily as a way of assessing whether they have acquired the relevant knowledge; if they can’t do the exercises, what they need is more rehearsing of theory.   As you can probably tell from the tone of this paragraph, I believe this approach is deeply misguided.  The in-depth explanation was provided by philosophers such as Ryle and Heidegger who established the primacy of knowledge-how over knowledge-that, of skills over theory.

Current educational practice subscribes overwhelmingly (and for the most part unwittingly) to the moderate version of Situated Cognition.  That is, we typically hope and expect that students’ general CT skills will emerge as a consequence of their engaging in learning and thinking as they proceed through secondary and especially tertiary education studying a range of particular subjects.  However, students generally do not reach levels of skill regarded as both desirable and achievable.  As Deanna Kuhn put it, “Seldom has there been such widespread agreement about a significant social issue as there is reflected in the view that education is failing in its most central mission—to teach students to think.”  In my view the weakness of students’ critical thinking skills, after 12 or even 16 years of schooling, is powerful evidence of the inadequacy of the Situated Cognition perspective.

There may be some truth to the Evolutionary Psychology perspective.  However in my view the best argument against it is the fact that another perspective – Practice – actually seems quite promising.   The basic idea behind it is very simple and plausible.   It is a truism that, in general, skills are acquired through practice.   The Practice perspective simply says that generic critical thinking skills are really just like most other skills (that is, most other skills that are acquired, like music or chess or trampolining, rather than skills that are innate and develop naturally, like suckling or walking).

In our work in the Reason Project at the University of Melbourne we refined the Practice perspective into what we called the Quality (or Deliberate) Practice Hypothesis.   This was based on the foundational work of Ericsson and others who have shown that skill acquisition in general depends on extensive quality practice.  We conjectured that this would also be true of critical thinking; i.e. critical thinking skills would be (best) acquired by doing lots and lots of good-quality practice on a wide range of real (or realistic) critical thinking problems.   To improve the quality of practice we developed a training program based around the use of argument mapping, resulting in what has been called the LAMP (Lots of Argument Mapping) approach.   In a series of rigorous (or rather, as-rigorous-as-possible-under-the-circumstances) studies involving pre-, post- and follow-up testing using a variety of tests, and setting our results in the context of a meta-analysis of hundreds of other studies of critical thinking gains, we were able to establish that critical thinking skills gains could be dramatically accelerated, with students reliably improving 7-8 times faster, over one semester, than they would otherwise have done just as university students.   (For some of the detail on the Quality Practice hypothesis and our studies, see this paper, and this chapter.)

So if I had to choose one theory out of the five on offer, I’d choose Practice.  Fortunately however we are not in a forced-choice situation. Practice is enhanced by carefully-placed Theoretical Instruction.  And Practice can be reinforced by Situated Cognition, i.e. by engaging in domain-specific critical thinking activities, even when not framed as deliberate practice of general CT skills.   As one of the greatest critical thinkers said in one of the greatest texts on critical thinking:

“Popular opinions, on subjects not palpable to sense, are often true, but seldom or never the whole truth. They are a part of the truth; sometimes a greater, sometimes a smaller part, but exaggerated, distorted, and disjoined from the truths by which they ought to be accompanied and limited.”

Read Full Post »

This evening  I was fortunate* to meet Greg Hunt, Federal Shadow Minister for Climate Change, Environment and Water.  I mentioned how in 2003 I had used an opinion piece he had written in an exercise for undergraduate students.  The exercise involved creating a map of his argument.  He was, naturally, curious to see what such a map would look like.

* Update in 2013 – I’m now a bit embarrassed to have written that.  Greg Hunt has turned out to be (if indeed he wasn’t all along) the worst kind of politician, using lies and spin to defend the morally indefensible in craven pursuit of political power.

Background: In the leadup to the (second) Iraq war, one of the hot topics of debate was whether the proposed invasion was legal in international law.  In February 2003, a group of 43 Australian legal heavyweights published Coalition of the Willing? Make that War Criminals, arguing bluntly that the war would be illegal and that its architects (Bush, Howard, Major) would be war criminals.   One of the ringleaders in this piece was Hilary Charlesworth, who had been one of Greg Hunt’s teachers at the University of Melbourne Law School.

Greg Hunt took on the task of responding publicly.  In March 2003 he published Yes, This War is Legal.

At the time I was teaching critical thinking in the Faculty of Arts at the University of Melbourne, using the method that we developed there, which was heavily based on argument mapping and required lots of practice mapping “real world” arguments.  I could think of no topic more timely, contentious and important than the legality of the upcoming war – and conveniently we had 800 word presentations of the arguments on each side.  So it made an ideal exercise in which these “best and brightest” young students could try out their emerging argument mapping skills.

For the record, I found that these students, among the most elite in the Australian educational system, were, for the most part, unable to ascertain the actual structure of the arguments presented on either side, even after having had many weeks of argument mapping training.  They could get a rough sense of the arguments, but discerning the precise logical shape demanded considerably more expertise than they had at that time.  Consequently, they were unable to properly evaluate the arguments; most ended up siding with the position they already favoured at an emotional or ideological level.  This is just an illustration of a quite general phenomenon; on matters of any complexity, the actual arguments are simply not comprehendable by most people.  And this of course is in large part because our standard means of presenting those arguments (e.g., in 800 word written opinion pieces in the newspaper) pose immense interpretative challenges.  The problem is not so much that people are stupid, but that the task given to them is far too difficult.

Anyway, here, in bCisive 2 format, is my own rendition, in argument map, of Greg Hunt’s case:


[click on image to view full-size version]

I’m not endorsing this argument.  What the map does is lay it out transparently, which lays the foundation for careful critique.  You can see at a glance such basic features as how many lines of argument there are; which points have been supported, and which merely asserted; where key assumptions lurk, waiting to be exposed; and so on.

Read Full Post »

That’s the title the editor gave to a letter I had published in the Education Age (21 May 07), commenting on an opinion piece by my University of Melbourne colleague Marty Ross.  Since they don’t make the letters to Education Age available online, I’m putting it up here.

Marty’s piece generally was very good.  He and I have no deep disagreement; but this letter puts into relief one point where we’d differ at least in emphasis. 

According to Marty Ross, the purpose of teaching mathematics is “training in logical thinking, learning to reason about anything.”  But mathematics is a poor way to achieve this goal, and there are better reasons to study the “queen of the sciences”. 

Ross is recycling an idea as old as Euclid himself, for we can find in Plato the view that reasoning is like a muscle, which can be strengthened through training in some formal discipline such as mathematics or grammar. 

The view is plausible, even appealing, but it is misguided.  It violates a key insight of research into cognitive skills, known as the “problem of transfer”: skills learned in one situation “transfer” to other situations far less than we would expect.  Studying mathematics may help students learn to prove theorems, but quite different kinds of reasoning are usually needed in everyday life and the workplace.  

There is a better way.  General “informal” reasoning skills can be taught directly, i.e., as skills in their own right.  In recent years, some remarkably effective methods for doing this have emerged.   Crudely, to Ross I would say: if you want to improve general reasoning, why not teach general reasoning?  Why teach something else, in the forlorn hope that some general reasoning skills will result? 
Unfortunately, effective direct approaches are not widely used.  In addition to its mauling of mathematics, a major problem in the Victorian education system is its failure to systematically teach the fourth R, reasoning. 

Lets not try to make mathematics carry a burden it cannot bear.  Lets instead teach mathematics to give all students at least a chance to appreciate the profundity, and beauty, of the most magnificent achievements of the human intellect.    Meanwhile, lets teach general informal reasoning skills using more direct and effective approaches.

Read Full Post »

“Just Some Guy” wrote today:

I recently stumbled on an excellent online article authored by yourself entitled “Teaching Critical Thinking“. I was wondering if you could take a moment of your valuable time to suggest a couple of books on the subject. I would like improve my critical thinking skills so I suppose the focus sought would be adult learner skill(s) acquisition with emphasis on techniques and (lots of) practice. I have been trying to develop said skills on my own (without much success). I would really like to have find a proven program to apply. As you know there is tons of information available online however I am getting lost trying to sort out all the wheat from chaff. Thank you in advance for your consideration.

I used to be a regular academic, and one reason for heading off in a different direction was the experience most academics know all too well, which is that you’ll slave for months on a paper, have it published, and then… nothing happens. It seems you may as well not have bothered. So it is gratifying when some paper you wrote, and which seemed to have vanished without a trace, starts to get picked up, read, and perhaps even appreciated. In the case of the paper mentioned above, in past month I’ve heard that it is the subject of a faculty discussion group at the University of Pittsburgh (where I did my PhD), and read by administrators at a startup university campus in Singapore. Now it seems to have helped Just Some Guy. Maybe it was worth the effort that went into it.

Anyway, regarding JSG’s query, in workshops I used to hand out brief annotated “further reading” list. Here it is:

There are hundreds of books on thinking and how to improve it, ranging from airport junk to turgid academic treatises. Here is a short list of some of the best, focusing on critical thinking. All are accessible, entertaining, and contain many valuable insights. Listed in alphabetical order, so don’t necessarily start at the top.

Cialdini, R. B. (1984). Influence: The Psychology of Persuasion. New York: William Morrow and Co. Classic, eye-opening description of the tricks, ruses and deceptions others use to manipulate us into doing what they want.

Giere, R. N. (1996). Understanding Scientific Reasoning (4th ed.). Fort Worth: Holt, Rinehart and Winston, Inc. Very clear overview of the fundamentals of scientific reasoning. Basic literacy in scientific methodology.

Heuer, R. J. (1999). Psychology of Intelligence Analysis. Center for the Study of Intelligence, CIA. Although intended primarily to assist intelligence analysts, there is a lot of good stuff here, on both the descriptive (how our minds work) and normative (rules for better thinking) sides. Plus, available free online!

Kepner, C. H., & Tregoe, B. B. (1997). The New Rational Manager. Princeton: Princeton Research Press. These are the people who first brought “critical thinking” to the business world and built out of it a multinational consulting firm. Very practical orientation.

Minto, B. (1996). The Minto Pyramid Principle: Logic in Writing, Thinking and Problem Solving.  Minto Books International Limited (www.barbaraminto.com). [Note: this edition supersedes the earlier edition, published by Pearson.]  Barbara Minto was a McKinsey consultant and editor; this book is now the “Bible” in this area for major consulting firms. Some profound truths about good thinking and communication, cast in a way which makes sense for folks in the business community.

Myers, D. G. (2002). Intuition: Its Powers and Perils. New Haven: Yale University Press. “Europe in ten days” tour of the ways intuitive thinking can go wrong, according to serious psychologists. Pretty exhaustive coverage, but most of it will just wash over you.

Paul, R. W., & Elder, L. (2002). Critical Thinking: Tools for Taking Charge of Your Professional and Personal Life. Upper Saddle River, New Jersey: Financial Times Prentice Hall. Paul and Elder are prominent critical thinking instructors. This book packages their insights as practical tools for personal and professional life. Stresses psychological and ethical issues, though often becomes a bit too “pop psychology”.

Piatelli-Palmarini, M. (1994). Inevitable Illusions: How Mistakes of Reason Rule our Minds. New York: Wiley. Very readable introduction to some of the most famous cognitive biases and blindspots. More diagnosis than therapy.

Salmon, M. (1989). Introduction to Logic and Critical Thinking (2nd ed.). San Diego: Harcourt Brace Jovanovich. The best of the standard undergraduate textbooks. A bit dull, but very sound.

Spence, G. (1995). How to Argue and Win Every Time. New York: St. Martin’s Press. Written by a criminal attorney who (according to the dust jacket) never lost a case. If you can look beyond the very “American” style, there is much wisdom here. It is a treatise in the art of rhetoric, but it is principled rhetoric rather than mendacious sophistry.

Whyte, J. (2004) Crimes Against Logic. McGraw-Hill. A short introduction to “fallacies,” i.e., common patterns of bad reasoning. Whyte runs through about a dozen, but there are dozens of others. Witty, fast-moving and brief.

Read Full Post »

On the AILACT list, Michael Scriven wrote:

Mark got in a dig about ‘speed reasoning’ my most popular course; perhaps I should mention that the first thing I say in the first session is, there’s no royal road to speed reasoning, you just have to become good at plain old slow reasoning first, and then do it a thousand more times, and you’ve mastered speed reasoning. BUT WE CAN HELP with the first part, by giving you a nice bunch of tools, beginning with argt structuring, plus a number of templates for patterns to spot as problematic, plus some neat ways to counter those, and now let’s see how that works in ten subject matter fields, and then we’ll test you on five other ones to see if you’ve ‘got the point’ See… it’s easy to improve!

As it happens, that’s a pretty good description of the pedagogical approach in our “Critical Thinking: The Art of Reasoning” subject at the University of Melbourne. Though I’d emphasize that we don’t just begin with argument structuring, we use argument structuring in diagrammatic form (i.e., argument mapping) throughout the subject.

Read Full Post »