Feeds:
Posts
Comments

Archive for the ‘Reasoning’ Category

One lesson of the terrible Black Saturday fires in Victoria was that lines of communication can break down, with tragic consequences. Information which may have been available to some did not reach and so could not inform the decisions of those who had to act.

The recently-released interim report of the Victorian Bushfires Royal Commission is in many ways an impressive achievement, produced on a tight schedule.

Ironically, however, the report is itself marred by broken lines of communication. In this case, they are the threads of argument relating the mass of information provided in the chapters with the Commission’s general recommendations.

For example, the Commission has in effect concluded that the controversial “stay or go” policy should be retained, though in modified form (Rec. 7.1). But what exactly is the case for retaining this policy as opposed, for example, to “leave early,” which they acknowledge is “unquestionably the safest course”?

Presumably the Report’s authors understand the case well enough, but nowhere have they succinctly and transparently laid it out. Relevant parts are scattered through Chapter 7, but considerable effort needed to pull them out and reconstruct them into a coherent whole.

The Commission has given us a kind of logical jigsaw puzzle, but where many of the pieces don’t belong to the puzzle at all and others are missing entirely.

What is the standard or benchmark against which this complaint is being made?

Simply this: that for any significant conclusion or recommendation, the ordinary reader ought to be able to tell, reliably and without inordinate effort,

(a) what are the main lines of argument for or against it, and for or against any salient alternative;

(b) what is the detailed evidence backing up each of those lines of arguments

(c) why the recommended option is on balance more strongly supported.

The Report consistently fails to meet this mundane standard. It is, in fact, a document only a lawyer could love.

Deficiencies of this kind are common in official reports, but they can have very serious consequences.

First, and most importantly, the Commission may have made some errors of judgement. We trust the Commission to base every recommendation on due consideration of all relevant arguments. If it had those arguments clearly before its collective mind, then there should be no trouble presenting them in easily digestible form in the Report. Conversely, if the Commission does not or cannot present them succinctly and transparently, then what confidence can we have that they were ever properly considered?

Second, recommendations may fail to be adopted. A case that cannot be identified or understood cannot be properly appreciated, and the recommendation may be unjustly rejected or neglected.

Third, the Commission may suffer in the court of public opinion. Those inclined to disagree with a recommendation, unable to divine the Commission’s sweet logic, will suspect it of stupidity, ignorance, malice, cowardice or perhaps even involvement in some conspiracy.

As it happens, the Victorian government has indicated that it will be accepting all the Commission’s recommendations.  However it is not at all clear that it did so because it clearly understood the Commission’s reasoning and recognized the compelling force of the arguments.  Rather, acceptance may have been driven primarily by political imperatives.   And it may in fact be accepting some ill-justified recommendations.

Granted, the interim report is only a staging point in the Commission’s process. We can hope – and justifiably expect – that the final report will be much improved in this regard.

Fortunately the problem should not be difficult to fix. It does not entail drastic changes to the form or substance of the report. Rather, the simple addition of “case summaries” would largely do the trick.

Every significant conclusion or recommendation ought to have associated with it a succinct and transparent summary of the reasoning on which it is based, a summary satisfying the basic requirements listed above. It is not crucial where the case summary appears; it could be a in sidebar in the chapter body, directly alongside the recommendation, or in an appendix. What is crucial is that it be easily found, and easily followed.

Simple “user testing” will show whether the case summary approach is working. Show a draft report to a handful of ordinary readers and simply ask them to find and explain the basis for any given recommendation or conclusion.

Australia has just been experiencing record-smashing winter temperatures, consistent with dire predictions of global warming and its impacts. We should anticipate that the bushfire conditions of February 2009 will recur frequently. The conclusions of the Victorian Bushfires Royal Commission will be of national significance for decades to come.

Given the enormous investment of public resources in the Commission, the extra cost and effort required to address this problem of logical transparency would be negligible.  An appropriate commitment to logical housekeeping in the final report may help to save resources, property and many lives later on.

Advertisements

Read Full Post »

I have recently been involved in a series of tutorials with a team from a law firm, involving mapping the arguments found in Federal or High Court judgements in the area of competition law.   For example we mapped the arguments presented by Chief Justice Gibbs of the High Court in the case Castlemaine Tooheys Ltd v Williams and Hodgson Transport Pty Ltd (1986) 68 ALR 376.  Here is the map in Rationale format:

CastlemaineTooheys

Figure 1.  View full size image, or the Rationale file.

Mapping these judgements is quite hard work.   This is due in part to the complexity of the arguments and their often turning on subtle conceptual distinctions.  However the difficulty is also largely due to the manner of expression, i.e. the way in which judges present complex legal arguments in traditional legal prose.  In a number of ways, this style of writing makes it very challenging to determine what the arguments are, at least at the level of clarity and rigour required for a well-developed argument map.

Sparse use of logical language

When we we present arguments in prose, we can use logical language to clearly indicate the logical connections between the parts of the argument.  So for example I might say

A brewer cannot sell beer on the condition that the buyer engage a particular transport firm for delivery.  The reason is that this would amount to exclusive dealing.

Here I have used the locution “The reason is” to help the reader understand that what follows is an argument for the preceding proposition.

However if you look closely at the Gibbs judgement, you’ll find that there is surprisingly little logical language of this kind.   Argument pieces are provided, but with little clear and explicit flagging of their logical relationships.  For example:

It is even more clear that there was no exclusive dealing of the kind mentioned in s.47(7).  The appellant did not refuse to supply the beer to any retailer.  It was already ready to supply any retailer, either by delivering it from the brewery to North Queensland, or by allowing it to be picked up from a regional depot.

The phrase “it is even more clear that” does suggest that the following proposition (there was no exclusive dealing…) is a point being established or supported.  The following sentences constitute (in part) the argument for that proposition.  Note however that those sentences are simply stated immediately after the proposition to be established.  There is no explicit verbal flag that they constitute an/the argument.   You, the reader, are supposed to just “see” that.  The cue is contextual (proximity) rather than verbal.  Gibbs could have said something like “This is because…” – but he didn’t; he relied on non-verbal cues instead.  The trouble is that non-verbal cues to logical relationships are generally quite weak and ambiguous, and impose a heavy interpretative burden – and hence cognitive load – on the reader.

Diverse or unclear logical language

Even when logical language is used, there are innumerable different forms, with subtle differences in meaning.   Readers would have a much easier time identifying complex arguments if there were standard, unambiguous verbal conventions for signposting logical relations.  Unfortunately, there are not.  Of course there are some locutions that are used more often than others and are relatively unambiguous – e.g. “It follows that”.  But you also find obscure or archaic expressions like “does not gainsay”, and use of ambiguous terms like “however” and “but”.  Both of these common expressions can be functioning as argument structure indicators, but might be playing quite different role.

Missing premises

Another problem is that only some pieces of the argument are actually presented in the prose.  The rest are left unstated, with the expectation that the reader will be able to fill in the gaps.

Consider again the passage quoted above.  What’s missing – at the very least – from this presentation of the argument is something like

Exclusive dealing of the kind mentioned in s.47(7) would require the appellant to refuse to supply beer.

Gibbs is assuming that the reader will understand that some proposition of this kind is part of the argument.  In this case it may seem a reasonable enough assumption.  However we know in general that people find it very hard to identify these missing parts, and when they try, there is huge variation in what they come up with.   In other words, it is dangerous to assume that readers will properly “read in” the missing pieces.  Even if they can do it correctly, it often takes signficant mental effort (though it is worth noting that very often, in simple cases, filling in the missing pieces is seemingly effortless).

Missing intermediate pieces (“leaps of logic”)

There are at least two kinds of missing pieces.  The first, just illustrated, is unstated co-premises; in other words, the argument is enthymematic.  The other kind are intermediate steps in the argument.  Here, in Rationale format, is an alternative rendering of the argument, making clear that between the general conclusion Gibbs is drawing (no exclusive dealing) and the  particular fact offered by way of proof (no refusal to supply beer) there is really an intermediate level of argument.

Implicit_intermediate_level

Figure 2.  Click on the thumbnail to view full-size version.

In other words, judges often expect readers to fill in missing pieces both alongside and between the pieces that are provided.

In fact, the first premise at the intermediate level had been provided, but many pages  before.  It was not (re)stated in the immediate context of the argument, though it did appear in the text, in a rather remote location, with many other pieces of the larger argument appearing in between.   Which brings up the next point…

Ordering

Consider an argument structure of the kind displayed in Figure 1.  Suppose you had to present that argument in ordinary prose.   Ordinary prose is essentially just a “linear” sequence of sentences – one sentence after another – though of course we can use some formatting to break up the monotony.    An issue you would be forced to address is: in what order do you present the pieces of the argument?

There is a systematic way to map a hierarchical structure into a sequence.  In fact there are many such ways.  However, every such way necessarily involves placing items adjacent in the sequence when they are not adjacent in the hierarchy.  This is simply a topological fact, with an interesting consequence for presenting arguments in prose.  It means that pieces of the argument with no direct relationship to each other must end up adjacent to each other in the text (though there may be other material, not part of the pure argument structure, separating them).  In other words, the prose will “bounce around” the argument structure.

puzzledWhen we study real examples of presentation of complex arguments in prose, such as the Gibbs judgement, and we look carefully at the order of presentation of pieces, we find that the author “bounces around” even more than is required mathematically.  In other words, in terms purely of ordering, the pieces are all “jumbled up”, almost as if pieces of a jigsaw had been thrown in a bag and drawn out one by one.

Of course, there will generally be some kind of logic in the madness; the author is trying to present the pieces in an order which, with the surrounding text, will help the reader understand the argument.  The issue is whether the author has done as well as he could in this.   Other things being equal, the more the “dis-order” or jumbling, the more trouble the reader will have in reassembling the hierarchical structure.

 

Intermingling with other material – and purposes

Another problem is that a written judgement is not solely concerned with transparently presenting the argument.   First, there is other material needing to be presented.  For example, there are the facts of the case, including both the facts that are immediately involved in the argument as well as other information providing important context.  Also there will be related issues which the judge may feel should be discussed, even though they are not part of the core arguments.

The trouble is that the presentation of the argument is intermingled with all this other material.  This means that the reader must pull apart what is argument, what is background, what is peripheral, etc..

On a related point, the judge has purposes in addition to transparent presentation of argument.  There is a certain amount of rhetoric involved in presenting a judgement, since the goal is to persuade others such as the parties to the case, their lawyers, and other judges.  Further, most judges would like to think that their written judgements are well-crafted, readable, perhaps somewhat entertaining, and possibly even literary, in a legal fashion.   It turns out that these goals are somewhat in tension with the goal of presenting the argument transparently.   It is certainly possible to present complex arguments in prose in such a way as a suitably trained reader can see, without inordinate effort or doubt, exactly what the argument is.  The problem is that such text is not pleasant to read, and it may not be as persuasive as more mellifluous discourse.   It requires a true master of argumentative writing to simultaneously reward the reader’s taste for literature or entertainment, and their desire to understand a complex argument with crystal clarity.

Argument Structure vs Essence

I’ve been writing as if judges’ primary ambition, in drafting written judgements, is to transparently display a complex argument structure.  But often when reading these judgements it seems that the authors are  more concerned to convey the “conceptual essence” of the argument – the key insight(s) such that, if the reader get that insight, then they’ve grasped the real heart of the argument.  The key that unlocks the case.   How exactly that essence is embedded in a larger structure of argument may not be so important.   So you find judges “talking around” that key insight or argument step, doing multiple “takes” on the presenting the point so that if the reader doesn’t quite get it one way, they’ll get it another, gaining understanding through a kind of triangulation.

Clearly conveying the conceptual essence of the case is of course a very important thing to do.  However it that is what the judge is primarily trying to do, then the effort devoted to this may be at the expense of, or even interfere with, transparent display of the rest of the case.

Summing up

Judges use written judgements to convey the complex set of arguments supporting their decision.  However it is difficult to extract the arguments from those written judgements, at the level of clarity and rigour demanded by good-quality argument mapping.   This difficulty is due in large part to various aspects of traditional legal prose.

  • Judges make surprisingly little use of verbal indicators of logical structure, and often use obscure or vague indicators
  • Judges present only some pieces of the arguments, expecting the reader to fill in the rest
  • The pieces necessarily appear in the text in a “disrupted” order, compared with their proper relationships in the argument structure
  • When producing their written judgements, judges have multiple purposes in addition to clearly conveying a complex structure; and the argument is intermingled, in the text, with other material
  • Judges may be more focused on conveying the conceptual essence of the argument than the full argument structure.

These observations are based on a fairly small sample – a handful of judgements in the current round of tutorials, plus my occasional experience over the past few decades  grappling with similar legal writings.  Still, I’m confident that the factors listed would be in play in most legal argumentative writing, and indeed almost any time an author attempts to convey a complex argument in prose.

If this is right, then if we’re faced with the challenge of presenting a complex argument in prose, we can help our readers by:

  • Making generous use of logical structure indicators, and trying to use a limited range of relatively standard, unambiguous ones
  • Explicitly stating more pieces of the argument
  • Trying to present the pieces in as coherent an order as possible, given the logical relationships among the pieces
  • Being aware of one’s purposes, and trying to avoid compromising the clear expression of the argument by other purposes
  • Disentangling the presentation of the argument from presentation of other material
  • Not neglecting overall argument structure while conveying the conceptual essence.

Find this interesting? Sign up for occasional emails about new posts or other news

Read Full Post »

Slides from a presentation at an intelligence & security seminar in Canberra last week.

Thanks to Brett Peppler for getting me the gig.

Read Full Post »

Maybe this post should’ve been called “Why judges should be paid more.”

Simon Lewis alerted me to the written judgment of Justice Ronald Sackville in the case Seven Network Limited v News Limited, otherwise known as the C7 case, or “Kerry Stokes against the world.”

This is a monster (1200 pages, 76mb in rtf format) document, itself the tip of the iceberg consisting of a far more monstrous legal case. The first chapter is a commentary on the case itself and its challenges. Some highlights:

“This case is an example of what is best described as ‘mega-litigation’…Mega-litigation, if it proceeds to finality, often generates very long judgments. Regrettably, this is a prime example.”

“The hearing occupied 120 sitting days…The burden on the Court was not limited to the 120 hearing days… ” Nevertheless “The hearing in the present case was considerably shorter than it might have been.”

One factor which reduced the length of the hearing was the extensive use of an “electronic courtroom.” “It would have been virtually impossible to conduct the trial without the use of modern technology.”

the volume of closing written submissions filed by the parties was truly astonishing” – but “The written submissions are only a minor component of the ‘paper’ burden in a case like this.”

“What is surprising is the sheer amount of money that has been devoted to a single case…the litigation has cost the parties collectively a staggering sum, amounting to nearly $200 million…In my view, the expenditure of $200 million (and counting) on a single piece of litigation is not only extraordinarily wasteful but borders on the scandalous.”

“I directed the parties to prepare an agreed chronology and encouraged them to agree on a template for written submissions. However, the responses illustrate that parties to mega-litigation are often able effectively to ignore (albeit politely) directions made by the court, if they consider that their forensic interests will be advanced by doing so.”

“The fundamental difficulty facing a court hearing mega-litigation, however, is that the parties may decide, for whatever reason, to engage in a full-blown forensic battle in which almost every barely arguable issue is examined in depth. In these circumstances, the best efforts of the court to limit the scope of the dispute may amount to very little.”

“No doubt courts must endeavour to control mega-litigation more efficiently.”

“the boards and shareholders of public companies embroiled in litigation of this kind need to take a more critical and sustained interest in the proceedings…If there is one lesson to emerge from this case, it is that even the largest and best-resourced corporations owe it to their shareholders, if not to the general public, to think very carefully before committing themselves irrevocably to mega-litigation.”

the length of written submissions may not be a true reflection of their worth. Very detailed submissions, despite their length, can of course be most helpful in clarifying the issues in dispute and in analysing the complex factual and legal questions requiring resolution. But this is not necessarily so.”

“the parties had not structured their Closing Submissions by reference to an agreed list of topics that had been handed up in court towards the conclusion of the evidence…by and large, they had decided to ignore the ‘agreed’ list of topics. They had taken this course notwithstanding my understanding, derived from discussions in court, that the list would provide a template for the written submissions and, in all probability, for the judgment.”

From a letter to the parties: “Quite apart from their length, I must confess to being surprised about some aspects of the submissions. At the risk of stating the obvious, part of the art of advocacy is to make it easy for the decision-maker to understand what issues need to be resolved and to explain clearly, cogently and concisely how and why the crucial issues should be resolved in favour of a particular party. To leave the Judge, if not completely at large, then without a reliable working compass in a vast sea of factual material, is not a technique calculated to advance a party’s case. This.. is because the cogency and persuasiveness of submissions depends on the ability of the Judge to follow them and to isolate the critical legal and factual issues upon which a case is likely to turn’.”

“Writing a judgment in a case such as this is an extremely onerous task. In part, this is due to the sheer volume of material that must be read, absorbed and analysed. The onerous nature of the task increases in proportion to the complexity of the legal and factual issues requiring resolution. In my view, only those who have undertaken a task of this character and magnitude can appreciate how relentless and indeed stressful it can be.”

mega-litigation requires the judge to be given every assistance that modern information technology can provide…in future, the setting up and co-ordination of electronic databases in mega-litigation must be carried out under the direct supervision of the Court, not the parties. Moreover, the process must be directed from the outset to meeting the judgment writing needs of the judge. ”

“The conclusion I have reached is that Seven has not succeeded in any of the many causes of action in which it has relied.” Note: this is amazing. They spent 100 million dollars fighting a legal case and didn’t succeed on a single point.

“There is a particular risk associated with mega-litigation that (happily for all concerned, but particularly for me) has not (yet) eventuated in these proceedings. The completion of the trial and the timely preparation of a judgment are contingent upon the trial judge surviving in reasonable health for the entirety of the proceedings…I asked at a pre-trial directions hearing whether the parties in the present case had considered insuring against the risk of judicial death or infirmity. “

Two of these issues – the failure of the lawyers to present their arguments in a manner easily comprehended by the judge, and the need for “every assistance that modern information technology can provide” are the ones of most interest to me and I will address them in a subsequent post.

Read Full Post »

 Now available – the final version of my paper prepared in connection with the conference Graphic and Visual Representations of Evidence and Inference in Legal Settings in January this year.  The paper is now called The Rationale for Rationale™.

Read Full Post »

 A new Rationale user working on a PhD thesis emailed the following:

I finished my comps in March and have been working to nail down my dissertation topic since. I have too many interests and little discipline so it’s been daunting. Notably, I sat down last week with rationale and decided to map out what I was thinking and feeling. I used the reasoning tools to nail down my main argument, the assertions I am inclined to make in support of that argument, and then what I know (or believe) supports those. Trying to not get bogged down, I next skipped to basis statements that helped me sort out which of these things I know are supported in the literature, which I need to do original logic on, which I need to test using a game model, and which I need to support using case studies. And finally – after months of circling, I went to the text panel and got the skeleton of a précis. Spent three more days cleaning up and thinking, and then as of this morning I sent those 4 pages off to a prospective adviser to start a conversation.

It might also be useful later in the process, articulating and evaluating what you take to be your core arguments.

If you’re writing a thesis, or some other elaborate piece of argumentative prose, then its a good idea to try mapping your arguments just to test whether you really know what they are.

If you actually have any substantial arguments, and if you are truly clear about what they are, mapping them should be a trivial exercise – just whacking claims into boxes and putting those boxes where they belong in the logical hierarchy.

However, it almost never is a trivial exercise.  We are, in fact, often quite deluded about the extent to which we really understand our own arguments.   Of course often we’re aware that we’re not fully on top of the arguments.  The more interesting point here is that, most of the time, when we think we know exactly what they are, we’re laboring under a kind of illusion of clarity.  There’s nothing like the demand to lay out the arguments in a map (well, a map observing the core principles of good argument mapping) to puncture the illusion.

The amount of effort you find you need to put in to get a tolerably good map of your arguments is a measure of the lack of clarity you have about those arguments.

(This assumes that you’re using a tool, like Rationale, which reduces to almost nothing the mechanics of producing an argument map diagram.)

Read Full Post »

That’s the title the editor gave to a letter I had published in the Education Age (21 May 07), commenting on an opinion piece by my University of Melbourne colleague Marty Ross.  Since they don’t make the letters to Education Age available online, I’m putting it up here.

Marty’s piece generally was very good.  He and I have no deep disagreement; but this letter puts into relief one point where we’d differ at least in emphasis. 

According to Marty Ross, the purpose of teaching mathematics is “training in logical thinking, learning to reason about anything.”  But mathematics is a poor way to achieve this goal, and there are better reasons to study the “queen of the sciences”. 

Ross is recycling an idea as old as Euclid himself, for we can find in Plato the view that reasoning is like a muscle, which can be strengthened through training in some formal discipline such as mathematics or grammar. 

The view is plausible, even appealing, but it is misguided.  It violates a key insight of research into cognitive skills, known as the “problem of transfer”: skills learned in one situation “transfer” to other situations far less than we would expect.  Studying mathematics may help students learn to prove theorems, but quite different kinds of reasoning are usually needed in everyday life and the workplace.  

There is a better way.  General “informal” reasoning skills can be taught directly, i.e., as skills in their own right.  In recent years, some remarkably effective methods for doing this have emerged.   Crudely, to Ross I would say: if you want to improve general reasoning, why not teach general reasoning?  Why teach something else, in the forlorn hope that some general reasoning skills will result? 
   
Unfortunately, effective direct approaches are not widely used.  In addition to its mauling of mathematics, a major problem in the Victorian education system is its failure to systematically teach the fourth R, reasoning. 

Lets not try to make mathematics carry a burden it cannot bear.  Lets instead teach mathematics to give all students at least a chance to appreciate the profundity, and beauty, of the most magnificent achievements of the human intellect.    Meanwhile, lets teach general informal reasoning skills using more direct and effective approaches.

Read Full Post »

« Newer Posts - Older Posts »