How do you help your students to achieve really worthwhile gains in critical thinking skills?

We worked on this problem for about five years at the University of Melbourne. We wanted a method for improving critical thinking skills which demonstrably achieves substantial results.

I’ll add now that we wanted a method which reliably acheives these results, i.e, gets them year after year, and in a variety of different contexts.

We think that we succeeded in this. The Reason method, as we called it, achieved gains of about 0.8 standard deviations semester after semester with University of Melbourne students.

We’ll soon be releasing a comprehensive review of the empirical literature (a “meta-analysis”) which compares our results with results found in other studies. Looking at the charts of the data, there certainly appears to be something special about the Reason method.

Increasingly, teachers and researchers around the world are doing their own studies to see if they can obtain the kind of good results showing up in our studies.

That’s great. It is crucial that the results be independently tested and (we hope) verified.

However, any independent attempt at replication should attempt to recreate the essential ingredients of the method being tested.

If you do a study which drops some of the essential ingredients, it doesn’t tell us much about the method.

Unfortunately that’s what we’re seeing. Again and again, attempts by third parties to find out if the Melbourne argument-mapping-based method “really works” don’t really test that method. They drop out a crucial ingredient – and then, usually and predictably, they find that they don’t achieve the same good results.

There are two crucial ingredients to the method we devised.

The first and most important is practice. Lots of practice. The Reason method, in a nutshell, is an application of the Ericsson theory that high levels of skill in any field come from lots of “deliberate practice.” Our idea (hardly very original or brilliant) is that the same will be true of reasoning or critical thinking skills.

The second ingredient comes out of that notion “deliberate.” Basically, deliberate means “good quality”. The challenge we confronted was – how do you get students to do LOTS of “good quality” practice?

Our insight was that you could improve the quality of practice students are doing by putting that practice into a good environment. In particular, we came to believe that argument mapping is a far better context for practice of reasoning skills than typical “prose”-based contexts (such as the typical university lecture, discussion section, book etc. which makes little or no use of argument diagrams).

So the second crucial ingredient in our method is using argument mapping. Specifically, doing one’s practice using argument mapping software.

Let me repeat that for emphasis. The method consists of

  1. LOTS of practice
  2. using argument mapping software

We used to call the method, in more technical language, “DPAM” – which stood for “deliberate practice using argument mapping.” Not very catchy.

Yanna Rider, one of the Austhink team, came up with the much better acronym LAMP. The method is Lots of Argument Mapping Practice.

Now, whether or not you use argument mapping, doing LOTS of practice is going to demand of a lot of institutional resources. Crudely put, it is going to take a lot of time and effort from staff, or time and effort from a lot of staff.

That’s a problem. Educational institutions are usually stretched pretty thin already, and putting MORE resources into some teaching exercise is a “big ask.”

What they are more often looking for is some way to get results with LESS resources.

So, what we usually find in independent attempts to “replicate” our results is that, when you look closely at how the method is being implemented, the focus is on argument mapping. the LOTS of PRACTICE has been downplayed or ignored.

Our prediction of course is that, to the extent that you don’t do the practice – the extent that you do AM rather than the full LAMP – you won’t get results as strong as ours.

The good news in all of this is that it is possible to achieve substantial gains. But you have to be prepared to do what it takes.

If you find out about an argument mapping study whose results were less impressive than ours, ask – were they really doing enough? Or does it look like they were hoping that argument mapping is some kind of magic bullet?

We believe that an argument-mapping based method is more efficient than other methods, because it offers a better quality of practice. So, for the same amount of resources or practice, you’ll get better results. But if the amount of practice your students are doing is negligible, the results will also be negligible.

One of our goals is to help educational institutions have their students do lots of argument-mapping-based practice without imposing significant extra resource requirements on those institutions. So a teacher can use the LAMP method without creating a lot of extra work for herself.