So says the outgoing Governor of the state of Victoria, David de Kretser.  Or at least, this is what he was reported as saying in today’s Age.

I can hardly he believe he really said it.  There are many (how many? I don’t know – but heaps) of websites presenting evidence-based information.   Here are just a few which come to mind quickly:

• Real Climate – “Climate science from climate scientists”
• Climate Progress – the legendary blog relentlessly fighting the good fight;
• Skeptical Science – a wealth of evidence-based information, including detailed responses to standard “denialist” arguments, at three levels of scientific detail, and available on an iPhone/Pad app;
• Our very own CSIRO’s website section on climate change;
• The IPCC.

Supposing de Kretser both said it and believed it, his strange assertion calls out for some kind of explanation.   Here’s a couple which seem plausible to me.

1.  de Kretser doesn’t actually surf the web very much.  He doesn’t read online.  He is of the generation that hardly uses computers very much, let alone dwells in the digisphere.  de Kretser thinks there is nothing out there because he hasn’t ventured out there to look.

2. de Kretser is sort of aware that there is at least some good stuff out there.  But he’s working backwards from the fact, seemingly inexplicable to him, that there is so much ignorance, delusion, and apathy in the population.  He tends to believe that when people are exposed to good information, they change their mind accordingly.  Since vast numbers of Australians don’t know and don’t care about climate change, they can’t have been exposed to information.  So there must be a lack of good information.  Maybe we should have a good website!

But this is naive.  It is naive about  individual psychology and how beliefs form and change.  And it is naive about the forces at work in society whose effect (only sometimes deliberate) is to distract, disinform, and confuse.   Possibly, intelligent and ethical scientists such as himself are exposed to, interested in, and form their beliefs on the basis of, good information.  But people such as himself are a tiny minority.

Sir, we don’t need more websites.  We have plenty already, and websites on their own are nearly useless in dealing with the kind of challenges we have – not just the primary challenge of dealing with climate change itself, but the tactical challenge of inducing appropriate change in people’s minds and behaviors.  Please devote your considerable capacities and influence to activities with real impact, not the shifting of pixels on the digital decks of a sinking civilisation.

The section Parte II: La téchnica de mapeo de argumentos (argument mapping) contains three pieces emerging from work at the University of Melbourne and Austhink:

• A translation of my article Teaching Critical Thinking: Some Lessons from Cognitive Science
• A translation of Enhancing our Grasp of Complex Arguments, by Paul Monk and me, a big-picture view of why complex arguments are cognitively challenging and how argument mapping can help.  It has been available as a manuscript since 2004 and been well-received so we are very happy to see it finally appear in print.
• A new chapter  by Claudia María Álvarez Ortiz, ¿El estudio de le filosofia mejora las habilidades de pensamiento crítico? which extracts some core material from her MA thesis Does Philosophy Improve Reasoning Skills?.  This is the first proper publication of the very important meta-analysis of studies of gains in critical thinking at college.

Well-designed and attractively produced, the appearance of the volume is a significant development in critical thinking pedagogy and theory, particularly in the Spanish-speaking Americas.   Regrettably the language barrier will be a major hurdle to recognition and uptake in the Anglosphere.  Perhaps somebody should undertake a translation of the whole volume into English?

Theories on this matter come in five main kinds:

• Formal Training. CT skills are simply the exercise of generic thinking power which can be strengthened by intensive training, much as general fitness can be enhanced by running, swimming or weightlifting.  This approach recommends working out in some formal ‘mental gym’ such as chess, mathematics or symbolic logic as the most convenient and effective way to build these mental muscles.
• Theoretical Instruction. CT skills are acquired by learning the relevant theory (logic, statistics, scientific method, etc.).  This perspective assumes that mastering skills is a matter of gaining the relevant theory.  People with poor CT poor skills lack only a theoretical understanding; if they are taught the theory in sufficient detail, they will automatically be able to exhibit the skills, since exhibiting skills is just a matter of following explicit (or explicable) rules.
• Situated Cognition. CT is deeply tied to particular domains and can only be acquired through properly “situated” activity in each domain.  Extreme versions deny outright that there are any generic CT skills (e.g. McPeck).  Moderate versions claim, more plausibly, that increasingly general skills are acquired through engaging in domain-specific CT activities.  According to the moderate version general CT skills emerge gradually in a process of consolidation and abstraction from particular, concrete deployments, much as general sporting skills (e.g., hand-eye coordination) are acquired by playing a variety of particular sports in which those general skills are exercised in ways peculiar to those sports.
• Practice sees CT skills as acquired by directly practicing the general skills themselves, applying them to many particular problems within a wide selection of specific domains and contexts.  The Practice perspective differs from Formal Training in that it is general CT skills themselves which are being practiced rather than formal substitutes, and the practice takes place in non-formal domains.  It differs from Situated Cognition in that it is practice of general skills aimed at improving those general capacities, rather than embedded deployment of skills aimed at meeting some specific challenge within that domain.
• Evolutionary Psychology views the mind as constituted by an idiosyncratic set of universal, innate, hard-wired cognitive capacities bequeathed by natural selection due to the advantages conferred by those capacities in the particular physical and social environments in which we evolved.  The mind does not possess and cannot attain general-purpose CT skills; rather, it can consolidate strengths in those particular forms or patterns of thinking for which evolution has provided dedicated apparatus.  Cultivating CT is a matter of identifying and nurturing those forms.

Formal training is the oldest and most thoroughly discredited of the perspectives.   It seems now so obvious that teaching latin, chess, music or even formal logic will have little or no impact on general critical thinking skills that it is hard to understand now how this idea could ever have been embraced.   And we also know why it fails: it founders on the rock of transfer.  Skills acquired in playing chess do not transfer to, say, evaluating political debates.  Period.

Theoretical Instruction has almost as old a philosophical pedigree as Formal Training.  It has been implemented in countless college critical thinking classes whose pedagogical modus operandi is to teach students “what they need to know” to be better critical thinkers, by lecturing at them and having them read slabs out of textbooks.   Token homework exercises are assigned primarily as a way of assessing whether they have acquired the relevant knowledge; if they can’t do the exercises, what they need is more rehearsing of theory.   As you can probably tell from the tone of this paragraph, I believe this approach is deeply misguided.  The in-depth explanation was provided by philosophers such as Ryle and Heidegger who established the primacy of knowledge-how over knowledge-that, of skills over theory.

Current educational practice subscribes overwhelmingly (and for the most part unwittingly) to the moderate version of Situated Cognition.  That is, we typically hope and expect that students’ general CT skills will emerge as a consequence of their engaging in learning and thinking as they proceed through secondary and especially tertiary education studying a range of particular subjects.  However, students generally do not reach levels of skill regarded as both desirable and achievable.  As Deanna Kuhn put it, “Seldom has there been such widespread agreement about a significant social issue as there is reflected in the view that education is failing in its most central mission—to teach students to think.”  In my view the weakness of students’ critical thinking skills, after 12 or even 16 years of schooling, is powerful evidence of the inadequacy of the Situated Cognition perspective.

There may be some truth to the Evolutionary Psychology perspective.  However in my view the best argument against it is the fact that another perspective – Practice – actually seems quite promising.   The basic idea behind it is very simple and plausible.   It is a truism that, in general, skills are acquired through practice.   The Practice perspective simply says that generic critical thinking skills are really just like most other skills (that is, most other skills that are acquired, like music or chess or trampolining, rather than skills that are innate and develop naturally, like suckling or walking).

In our work in the Reason Project at the University of Melbourne we refined the Practice perspective into what we called the Quality (or Deliberate) Practice Hypothesis.   This was based on the foundational work of Ericsson and others who have shown that skill acquisition in general depends on extensive quality practice.  We conjectured that this would also be true of critical thinking; i.e. critical thinking skills would be (best) acquired by doing lots and lots of good-quality practice on a wide range of real (or realistic) critical thinking problems.   To improve the quality of practice we developed a training program based around the use of argument mapping, resulting in what has been called the LAMP (Lots of Argument Mapping) approach.   In a series of rigorous (or rather, as-rigorous-as-possible-under-the-circumstances) studies involving pre-, post- and follow-up testing using a variety of tests, and setting our results in the context of a meta-analysis of hundreds of other studies of critical thinking gains, we were able to establish that critical thinking skills gains could be dramatically accelerated, with students reliably improving 7-8 times faster, over one semester, than they would otherwise have done just as university students.   (For some of the detail on the Quality Practice hypothesis and our studies, see this paper, and this chapter.)

So if I had to choose one theory out of the five on offer, I’d choose Practice.  Fortunately however we are not in a forced-choice situation. Practice is enhanced by carefully-placed Theoretical Instruction.  And Practice can be reinforced by Situated Cognition, i.e. by engaging in domain-specific critical thinking activities, even when not framed as deliberate practice of general CT skills.   As one of the greatest critical thinkers said in one of the greatest texts on critical thinking:

“Popular opinions, on subjects not palpable to sense, are often true, but seldom or never the whole truth. They are a part of the truth; sometimes a greater, sometimes a smaller part, but exaggerated, distorted, and disjoined from the truths by which they ought to be accompanied and limited.”

Marty’s piece generally was very good.  He and I have no deep disagreement; but this letter puts into relief one point where we’d differ at least in emphasis. 

According to Marty Ross, the purpose of teaching mathematics is “training in logical thinking, learning to reason about anything.”  But mathematics is a poor way to achieve this goal, and there are better reasons to study the “queen of the sciences”. 

Ross is recycling an idea as old as Euclid himself, for we can find in Plato the view that reasoning is like a muscle, which can be strengthened through training in some formal discipline such as mathematics or grammar. 

The view is plausible, even appealing, but it is misguided.  It violates a key insight of research into cognitive skills, known as the “problem of transfer”: skills learned in one situation “transfer” to other situations far less than we would expect.  Studying mathematics may help students learn to prove theorems, but quite different kinds of reasoning are usually needed in everyday life and the workplace.  

There is a better way.  General “informal” reasoning skills can be taught directly, i.e., as skills in their own right.  In recent years, some remarkably effective methods for doing this have emerged.   Crudely, to Ross I would say: if you want to improve general reasoning, why not teach general reasoning?  Why teach something else, in the forlorn hope that some general reasoning skills will result? 
   
Unfortunately, effective direct approaches are not widely used.  In addition to its mauling of mathematics, a major problem in the Victorian education system is its failure to systematically teach the fourth R, reasoning. 

Lets not try to make mathematics carry a burden it cannot bear.  Lets instead teach mathematics to give all students at least a chance to appreciate the profundity, and beauty, of the most magnificent achievements of the human intellect.    Meanwhile, lets teach general informal reasoning skills using more direct and effective approaches.

Mark got in a dig about ‘speed reasoning’ my most popular course; perhaps I should mention that the first thing I say in the first session is, there’s no royal road to speed reasoning, you just have to become good at plain old slow reasoning first, and then do it a thousand more times, and you’ve mastered speed reasoning. BUT WE CAN HELP with the first part, by giving you a nice bunch of tools, beginning with argt structuring, plus a number of templates for patterns to spot as problematic, plus some neat ways to counter those, and now let’s see how that works in ten subject matter fields, and then we’ll test you on five other ones to see if you’ve ‘got the point’ See… it’s easy to improve!

As it happens, that’s a pretty good description of the pedagogical approach in our “Critical Thinking: The Art of Reasoning” subject at the University of Melbourne. Though I’d emphasize that we don’t just begin with argument structuring, we use argument structuring in diagrammatic form (i.e., argument mapping) throughout the subject.

It’s been a fascinating time, and I’m not bored with it yet. Still, on occasion I also wonder what it would be like to teach some other subject, a subject where logical thinking is a tool rather than the focus, and where argument mapping can be evaluated not for its impact on critical thinking skill gains, but rather for its benefits in supporting learning and inquiry.

In particular, given my background in philosophy, what would it be like to teach – say – Introduction to Philosophy, incorporating argument mapping?

A prior question is – how would argument mapping actually be used in such a subject?

Some early adopters have already been leading the way in terms of introducing argument mapping into general philosophy. For example there’s David Spurrett in Durban, South Africa, who for a number of years has been using Reason!Able, Rationale’s predecessor, in various subjects.

Looking at what people like Dave have been doing, and trying to imagine what I would probably do, here are some ideas:

1. Use Rationale during preparation of lectures to create graphics of the main arguments under consideration to include in Powerpoint overheads.
2. Or better still, display “live” argument maps during lectures from within an argument mapping application such as Rationale. That way, you can easily modify the maps to bring out points during the class, for example in response to students’ questions or suggestions.
3. More radically, a whole lecture could be presented from within the argument mapping application. Rationale, for example, has an infinitely extendable workspace, on which you can include as many maps as you like; you can zoom and scroll as needed to bring the relevant material to the forefront. Sticky notes can be used to incorporate material which doesn’t naturally fit into an argument map.
4. Instead of just requiring students to read some text before a class, have them try to map the core argument(s) being presented. This gives them a task with a goal, which will help them engage with the text better. (They’ll find this activity very hard at first, but with your guidance they’ll gradually get better at it.)
5. Require students to work up an argument map of their own argument or arguments they’re presenting in their essays. Have them hand the map in with the essay. This will (a) lead them to be much more explicit about what exactly their argument is; (b) give them a logical skeleton on which to hang their essay; and (c) give you a fast way of understanding what they are trying to argue (and, in many cases, whether they are hopelessly confused).

That’s just a few ideas; no doubt there are lots of other good ways to use argument mapping. After all, philosophy (at least in the “Anglo-American analytic” tradition) is heavily focused on understanding and evaluating arguments, even at introductory levels (e.g., Descartes’ classic arguments for the distinctness of mind and body). If we’ve got a better way of displaying and manipulating argument structures, it must surely find many uses.

We worked on this problem for about five years at the University of Melbourne. We wanted a method for improving critical thinking skills which demonstrably achieves substantial results.

I’ll add now that we wanted a method which reliably acheives these results, i.e, gets them year after year, and in a variety of different contexts.

We think that we succeeded in this. The Reason method, as we called it, achieved gains of about 0.8 standard deviations semester after semester with University of Melbourne students.

We’ll soon be releasing a comprehensive review of the empirical literature (a “meta-analysis”) which compares our results with results found in other studies. Looking at the charts of the data, there certainly appears to be something special about the Reason method.

Increasingly, teachers and researchers around the world are doing their own studies to see if they can obtain the kind of good results showing up in our studies.

That’s great. It is crucial that the results be independently tested and (we hope) verified.

However, any independent attempt at replication should attempt to recreate the essential ingredients of the method being tested.

If you do a study which drops some of the essential ingredients, it doesn’t tell us much about the method.

Unfortunately that’s what we’re seeing. Again and again, attempts by third parties to find out if the Melbourne argument-mapping-based method “really works” don’t really test that method. They drop out a crucial ingredient – and then, usually and predictably, they find that they don’t achieve the same good results.

There are two crucial ingredients to the method we devised.

The first and most important is practice. Lots of practice. The Reason method, in a nutshell, is an application of the Ericsson theory that high levels of skill in any field come from lots of “deliberate practice.” Our idea (hardly very original or brilliant) is that the same will be true of reasoning or critical thinking skills.

The second ingredient comes out of that notion “deliberate.” Basically, deliberate means “good quality”. The challenge we confronted was – how do you get students to do LOTS of “good quality” practice?

Our insight was that you could improve the quality of practice students are doing by putting that practice into a good environment. In particular, we came to believe that argument mapping is a far better context for practice of reasoning skills than typical “prose”-based contexts (such as the typical university lecture, discussion section, book etc. which makes little or no use of argument diagrams).

So the second crucial ingredient in our method is using argument mapping. Specifically, doing one’s practice using argument mapping software.

Let me repeat that for emphasis. The method consists of

1. LOTS of practice
2. using argument mapping software

We used to call the method, in more technical language, “DPAM” – which stood for “deliberate practice using argument mapping.” Not very catchy.

Yanna Rider, one of the Austhink team, came up with the much better acronym LAMP. The method is Lots of Argument Mapping Practice.

Now, whether or not you use argument mapping, doing LOTS of practice is going to demand of a lot of institutional resources. Crudely put, it is going to take a lot of time and effort from staff, or time and effort from a lot of staff.

That’s a problem. Educational institutions are usually stretched pretty thin already, and putting MORE resources into some teaching exercise is a “big ask.”

What they are more often looking for is some way to get results with LESS resources.

So, what we usually find in independent attempts to “replicate” our results is that, when you look closely at how the method is being implemented, the focus is on argument mapping. the LOTS of PRACTICE has been downplayed or ignored.

Our prediction of course is that, to the extent that you don’t do the practice – the extent that you do AM rather than the full LAMP – you won’t get results as strong as ours.

The good news in all of this is that it is possible to achieve substantial gains. But you have to be prepared to do what it takes.

If you find out about an argument mapping study whose results were less impressive than ours, ask – were they really doing enough? Or does it look like they were hoping that argument mapping is some kind of magic bullet?

We believe that an argument-mapping based method is more efficient than other methods, because it offers a better quality of practice. So, for the same amount of resources or practice, you’ll get better results. But if the amount of practice your students are doing is negligible, the results will also be negligible.

One of our goals is to help educational institutions have their students do lots of argument-mapping-based practice without imposing significant extra resource requirements on those institutions. So a teacher can use the LAMP method without creating a lot of extra work for herself.