Feeds:
Posts
Comments

A new draft of What Do We Think?  Divining the Public Wisdom to Guide Sustainability Decisions is now available.

Download PDF

What do we need?

If the transition to sustainability requires the public wisdom, and if we currently have no practical and effective mechanism for ascertaining that wisdom, then we need to develop something better.

What would such a mechanism look like?  Here’s a wish list:

  • It would generate public wisdom in the fullest sense – i.e. the collective considered opinion based on large-scale deliberation.
  • It would be generating that wisdom on all major issues, including new issues as soon as they arise.
  • It would make that wisdom available to anyone at any time.
  • It would by inclusive in the sense of providing a practical opportunity for any interested citizen to participate, and would in fact involve participation of numerous and diverse members of the public.
  • It would be politically neutral and completely independent of control by government, corporates or any other powerful interest group.

A National Virtual Forum

Surveying the wish list, it is obvious that any mechanism capable of delivering the goods would have to be internet-based. It would have to be, in other words, a kind of national virtual forum (NVF).

No such forum exists today.  The good news however is that a NVF plausibly could exist.   As everyone knows, the internet hosts innumerable forums already; many are focused on serious social, economic and political issues, and support deliberation that is often of surprisingly high quality.  While it is de rigueur to sneer at the quality of online discussion, and indeed much of it is rubbish, we should at the same time acknowledge that every day literally thousands of Australians jump online and vigorously debate the major issues of the day.

Further, and more profoundly, there is the fact that internet-based environments or systems have been proven capable of synthesizing collective intelligence or wisdom of various kinds.  Wikipedia, prediction markets, Amazon.com, and Stack Overflow are all well known examples.  To be sure, none of these generate collective rational consensus of the kind expected from the NVF.  How exactly that form of collective intelligence will be assembled or extracted is a major design challenge.  But important precedents do exist, and they do more than just prove that collective intelligence can be generated: they provide a wealth of insights and hints for the development of a NVF.

Challenges

A NVF would clearly face numerous major obstacles.  In my view, these are best regarded as challenges to be overcome rather than fatal objections to the whole exercise.   Here are four, with brief hints as to how they might be tackled.

  1. Critical Mass.  The NVF will have to attract many and diverse participants.  To do this, first and foremost the NVF must be easily accessible – simple to use and available via any major channel (website, mobile apps, etc.).  It must be thoroughly and effectively integrated with social media (Twitter, etc.).   “Gamification” techniques will help deepen participants’ engagement.  Finally, a major media alliance will situate the NVF in the public’s attention (similar to, say, the Oursay cooperation with The Age).
  2. Representativeness.  For its outputs to count as the wisdom of the public as a whole, the participants would need to be sufficiently similar to the public – i.e., to statistically represent the public.  On the face of it, this will be a problem if the NVF has an open-door approach, allowing its participants to self-select.  Despite this various strategies can be used to approximate and enhance representativeness, approaching full representativeness as a kind of limit case.  For example, assuming there are demographics on participants of a known degree of reliability, and a sufficiently large and diverse set of participants, it would be possible to select suitable subsets of participants to form the pool for the purposes of computing group wisdom.
  3. Gaming.  If it builds any kind of momentum, the NVF will become a target for “gaming” (e.g. astroturfing) as groups attempt to manipulate the outputs to suit their own interests.  This problem can never by fully solved, but could be handled adequately.  The problem of distinguishing genuine from bogus participation is similar to the problem of distinguishing genuine email from spam, and Google has shown that this can be done remarkably well.
  4. Credibility/Influence.  The main point of setting up the NVF is to help governments make the best decisions.  For this to work, governments would have to take the NVF outputs seriously.  I’m optimistic that this problem would start to solve itself just insofar as the NVF achieves critical mass and delivers its intended output – not because governments will be virtuous and do the right thing but rather because they will inevitably start to respond out of pure pragmatic political self-interest.  If the genuine considered opinion of the public on a major issue is available, and if it diverges significantly from the public attitude as expressed in the polls, then it will constitute another kind of political cudgel which can be used by either the government or the opposition.

Conclusion – Now is the time to start

Clearly, establishing a NVF of the kind described would be no mean feat.  Yet as I’ve argued, we need such a thing if we’re to make a smooth, timely transition to sustainability.

It is high time we had practical and effective mechanisms for knowing what the public really thinks on the major issues affecting it.   The ubiquity and sophistication of the internet and the systems built upon it provide us the opportunity to realise this democratic ideal.

The NVF proper will not be built in a day or even a year.  Rather, it will evolve in a serious of stages, incrementally approaching the full vision.

Eight years (from here to 2020) is probably a reasonable time-frame within which something worthwhile could be achieved.  Remember that Twitter is less than eight years old, and has already played a key role in democratic movements worldwide (e.g. North Africa).


This post is the third part of a draft chapter What Do We Think? Identifying the Public Wisdom to Guide Sustainability Decisions, in preparation for the volume 20/20 Vision for a Sustainable Society, being put together by the Melbourne Sustainable Society Institute.

Previous:

One of the main points of this chapter is that to make the transition to sustainability in a safe and timely manner we need to identify the public wisdom on sustainability issues.

Why?

First, because it will help governments to make the decisions we need them to make. The public wisdom can give the government the kind of “mandate” or authority it needs to tackle divisive issues and make tough decisions, even when doing so may be going against the tide of public opinion as measured by the polls.

Knowing the public wisdom on the matter may also help swing public opinion.

This potential benefit was behind Julia Gillard’s recommendation, in the 2010 election campaign, that a 150-person Citizen’s Assembly be convened to develop some rational consensus around climate change policy.  She apparently believed that identifying the considered opinon of the public on the issue would help break the political impasse.  The proposal had some intrinsic merit but was, as Lyn Carson has described, ridiculed from many directions, with such rejection driven by many different agendas and misunderstandings.   One lesson of this episode is that we need institutions and mechanisms capable of articulating the collective wisdom without requiring any support or approval from the powers that be (see below).

Second, because on many issues, the public wisdom would be best guide to the truth of the matter.  If we’re serious about making the right decision, then we must find out what the public really thinks.

Consider an issue like whether we should have more large dams to better manage scarce water resources.   This simple-sounding question sits on top of a complex web of issues, involving not just factual and technical matters but diverse competing interests and conflicting values.  Certainly many individual experts and interest groups are highly knowledgeable about particular aspects, and their input should be given due consideration.  However such folks always have a particular perspective; they see only their part of the larger elephant.  The wider the involvement – the more diverse and comprehensive the selection of participants – the more chance that all the relevant information can be brought to the decision, and the relevant interests and values recognised and accommodated.

Note that I’m not claiming that the public’s considered opinion is the best guide on any complex matter.  Many issues clearly are matters of specialist expertise, and the general public is in no position assess the merits of different theories.  An obvious example is the science of climate change.  Only the body of climate scientists has the knowledge and competence to settle the scientific issues.  Neither laypeople individually nor the public as a whole have any business trying to make up their own minds on this topic.

However on major sustainability issues there are no individuals or special groups in a uniquely privileged position to discern the truth.  These decisions are matters of interests and values as much as they are matters of knowledge or expertise, and the Australian people are the relevant authorities on what their interests and values are.

But we don’t know what the public wisdom is

The problem with the public wisdom is that we almost never know what it is.  That is, on any given major issue, we don’t know what the collective considered opinion is.  Indeed, that wisdom usually doesn’t even exist, in the sense that nothing has been done to put it together.

We do have many windows onto public opinion, but they’re all either ineffective (don’t deliver public wisdom) or impractical (too cumbersome and expensive).

For example standard opinion polls, for reasons described above, don’t tell us what the public wisdom is.

Well-designed surveys are a step up from standard opinion polls (Reser).  However, these surveys provide little opportunity for the respondents to engage in any sustained reflection, individually or collectively.   They are just more sensitive ways of identifying the attitudes people happen to have.  Ideally, such surveys would play a much larger role than they currently do in the gauging of public opinion.   However they can’t identify the public wisdom, in the sense described above.

Deliberative polling, and why it is not enough

By far the best mechanism we currently have for ascertaining the public wisdom is deliberative polling.  It is summarised on the Stanford Center for Deliberative Democracy website as follows:

A random, representative sample is first polled on the targeted issues. After this baseline poll, members of the sample are invited to gather at a single place for a weekend in order to discuss the issues. Carefully balanced briefing materials are sent to the participants and are also made publicly available. The participants engage in dialogue with competing experts and political leaders based on questions they develop in small group discussions with trained moderators. Parts of the weekend events are broadcast on television, either live or in taped and edited form. After the deliberations, the sample is again asked the original questions. The resulting changes in opinion represent the conclusions the public would reach, if people had opportunity to become more informed and more engaged by the issues.

Over the past few decades, dozens of deliberative polls have been conducted around the world.  In Australia has had a handful, on topics such as republicanism and reconciliation.

Australia would benefit greatly if deliberative polls were held much more often, and if their results were more influential in major decisions.

However, deliberative polling, in its standard form at least, can’t meet the need to deliver the collective wisdom for the purpose of guiding timely decision making on major sustainability issues.

The critical problem is that deliberative poll is a cumbersome exercise and is costly to stage.  This has a number of consequences:

  • There aren’t enough of them.  The large cost is one major reason there have been so few deliberative polls since the idea was first propounded over two decades ago.  It may be that the frequency of deliberative polls is increasing, which is a surely a good thing, but deliberative polling currently and for the foreseeable future can address only a fraction of the issues which properly ought to be guided by public wisdom.
  • They take a long time to set up.  It can take six months or more to set up and run a deliberative poll.  The time from conception – the moment when it is recognised that having a DP on a certain topic would desirable – is far longer.  And of course most deliberative polls that have been conceived simply haven’t been run (yet).
  • Once run, they’re finished.  The public wisdom identified in the deliberative poll is frozen in time.   It becomes outdated and irrelevant as circumstances and information change.

The latter two points may not be such a problem for relatively timeless issues such as whether Australia should become a republic, but they constitute a severe drawback when decisions need to be made quickly on issues which are heavily shaped by circumstances arising at a particular moment in history.

For example, should Australia adopt the Gillard governments carbon pricing scheme?  This is not the general issue of e.g. whether Australia should take action on climate change, and whether it should institute an emissions trading scheme.  Rather it is whether a particular plan should be adopted at a particular historical juncture.  The debate is raging as this is being written, and ideally we would be able to divine the collective wisdom right now.   A deliberative poll on the topic would be great, but it isn’t happening, and practically speaking couldn’t happen for many months.  By the time a deliberative poll was staged, it may well be too late.

A more philosophical quibble with deliberative polling is that, as standardly conducted, it doesn’t deliver public wisdom in the fullest sense.   The primary output of the deliberative poll is the poll results – i.e. a tabulation of individual opinions.  Granted, these individual opinions have become more considered through quality deliberation, and are thus worthy of more respect than the attitudes tapped by standard opinion polls.  However there has been no deeper aggregation of individual judgement into a coherent collective viewpoint.   It is as if the IPCC reports were to consist of an exit poll of climate scientists’ beliefs, rather than a carefully drafted and agreed statement.


This post is the second part of a draft chapter What Do We Think? Identifying the Public Wisdom to Guide Sustainability Decisions, in preparation for the volume 20/20 Vision for a Sustainable Society, being put together by the Melbourne Sustainable Society Institute.

Previous:

Coming up:

Australia is patently unsustainable in many ways, and so will have to change.  Will this change be wisely and pro-actively managed?  Or will it be forced on us in unwelcome, disruptive and possibly catastrophic ways?

Wise management will require governments at all levels to make lots of difficult decisions, and to make them expeditiously.

In this decision making, public opinion is a critical constraint.

For example, there is a good case for road use pricing to manage our unsustainable dependence on use of private vehicles.  Yet this option is instantly dismissed by both major political parties, fearing a public backlash – no matter how ill-informed, short-sighted or self-serving that public reaction may be.  Meanwhile our cities become increasingly gridlocked, with escalating economic, health and environmental costs.

Simply put, unless we can improve the relationship between government decision making and public opinion, we’re going to “hit the wall” in numerous respects.

Of course, the importance of public opinion has hardly been lost on sustainability advocates.  There has already been, and continues to be, lots of good work in this area – particularly as regards climate change.  Considerable insight has been gained on topics such as how opinions are formed, how they are related to behavior, and how they can be influenced.

As part of this effort, we must also develop better ways to find out what the public opinion is, i.e. what the public actually thinks.

But what’s the problem?  Don’t we already know pretty much what the public thinks, from the endless stream of opinion polls? And isn’t the problem in fact that there is too much monitoring of public opinion, and that governments are too sensitive to it?

Its true that public opinion, in the standard sense – what might be called the public attitude – is in oversupply.

What we almost never know is the considered opinion of the public – the public wisdom.

Public attitude versus public wisdom

Public opinion, as we usually understand it, is the kind of information generated by the familar polls run by organisations such as Morgan and Gallup and delivered as fodder to the mainstream media.

The public wisdom, by contrast, is the collective, considered opinion of the public.  It is what the public as a whole would think if it were able to think seriously about the matter, i.e. become well-informed, reflect carefully, and somehow pool their thoughts into a coherent position.  Thinking seriously in this way requires collective deliberation, i.e. constructive discussion and debate.

Public opinion falls a long way short of public wisdom.  In his book When The People Speak, notable theorist of democracy James Fishkin has pointed to a number of problems with public opinion:

  • Respondents are generally ill-informed; indeed they will usually be rationally ignorant on the topic.
  • Individuals’ attitudes are subject to manipulation by powerful forces pursuing their own agendas, e.g. major corporates resisting progressive tax reforms.
  • The opinions elicited in standard polls may be artificially manufactured by the polling process itself, i.e. may not reflect any real attitude held by the respondents but rather are generated on the spot in response to the polling process and are shaped by that process.

To which I would add: the respondents will generally not have engaged in any serious deliberation (on their own, or with others) on the issue, and the polling process provides no opportunity for such deliberation.

In short, standard opinion polls give us a distorted snapshot of the attitudes the respondents happen to have at that moment – not a fair reflection of what they (would) think about the issue.

To compound matters, standard polling processes do nothing more than tabulate individual opinions.  They don’t synthesize or aggregate the viewpoints of the respondents into a common or collective position, as would be required for genuine “wisdom of the crowd.”

For an example of genuine collective wisdom, consider the reports of the Intergovernmental Panel on Climate Change (IPCC).  These are generated by means of an elaborate process, involving much high-quality deliberation, in which exceptionally well-informed scientists pool and refine their knowledge, coming up with an agreed expression of what their community as a whole believes.


This post is the first part of a draft chapter What Do We Think? Identifying the Public Wisdom to Guide Sustainability Decisions, in preparation for the volume 20/20 Vision for a Sustainable Society, being put together by the Melbourne Sustainable Society Institute.

Coming up:

Apparently horses in races are almost always (98%) whipped.*  The main reason is to make them go faster.

Congratulations to the scientists from the University of Sydney who won a prize for discovering that “whipping does not increase horses’ chances of finishing in the top three and that they actually run faster when they are not being hit.”

So it seems that overwhelmingly, the horse racing experts – the jockeys and trainers particularly – have for decades followed a practice which:

  1. Hurts their chances of winning.
  2. Hurts their beloved animals.
  3. Hurts their standing in the wider community.

Given how much money is at stake in horse racing, this is remarkable.  It calls out for explanation.  How can these experts have persisted for so long in such self-destructive behavior?   Here are some possible explanations.

First, the idea that whipping makes a horse go faster has a certain “truthiness.”  This truthiness made questioning the practice seem otiose.  Further, belief in the truthy proposition became a perceptual filter through which they “saw” that whipping made horses go faster.

Second, the omnipresence of whipping prevented the possibility of observing the weak correlation between whipping and losing.   If only about half of the jockeys whipped their horses, then over time people might have started to notice that the unwhipped horses tended to win a bit more often.  Or at least didn’t lose more often.

Third, since everyone else was whipping, everyone naturally assumed that whipping was the “right” thing to do.   Failing to whip the horse would look rather odd – especially if you didn’t win.   As Mark Twain wrote: “We are creatures of outside influences; as a rule we do not think, we only imitate.”

* I don’t know anything about horse racing.  This claim seems implausible, but here is my source.

Much of what Austhink does these days is concerned with “collective wisdom” – the knowledge that a group as a whole has.   As Surowiecki famously pointed out, when the conditions are right, the wisdom of the group can be superior to that of the individuals making it up.

However finding out what that collective wisdom is – identifying, assembling or articulating it – is often an interesting challenge.  Various methods or approaches have been developed, suited to various sorts of groups and types of knowledge.

Not surprisingly, the quality or “grade” of the collective wisdom generated by these various methods can differ markedly.

Here is one way to distinguish some main grades:

  1. Grade 1 (the lowest grade) results simply from statistical enumeration of individual opinions, as happens in, for example, a standard opinion poll, plebiscite or election.   The “wisdom of the crowd” identified by an opinion poll is just the majority opinion.
  2. Grade 2 still results from statistical enumeration of individual opinions, but those opinions have been improved by some appropriate collective process, i.e. they have benefited from some relevant kind of interaction in the group.  In deliberative polling, for example, the collective wisdom is the result of a poll taken after a collective deliberative process in which individuals are presumed to benefit from deliberating with each other.   The group opinion, as reflected in the result of the poll, is better than Grade 1 just insofar as the individual opinions are better as a result of the process.  (Note that there is a whole cluster of interesting issues to do with whether, and under what conditions, group deliberative processes do lead to improved opinions.)
  3. In Grade 3, the collective wisdom is not just aggregated or enumerated individual opinion, but results from some kind of synthesis of individual opinions.  One of the simplest forms this can take is just an averaging, as for example in the well-known Galton “guess the weight of the ox” scenario (as described by Surowiecki).   A more interesting type of collective wisdom at this level is price in some kind of market, including a prediction market.
  4. In Grade 4,  the collective wisdom is a synthesis of individual opinions, plus the collective opinion is endorsed by all or at least most of the individuals.   The outcome of a prediction market, for example, doesn’t make Grade 4 because the current price is deemed by most participants to be “wrong”; some regard it as too high (hence they’re not buying) and others see it as too low (hence they’re not selling).  However IPCC reports do appear to be Grade 4 on this scheme.  They result from a elaborate collaborative process of drafting (synthesizing the information and views of the participating scientists – views which might themselves be improved by the process) and the result is generally endorsed by those scientists.

I doubt that the classification scheme described above is the best/ideal scheme.  How could it be improved?

Indeed, maybe through a collaborative process we could with a scheme which itself consists of “AAA-grade” collective  wisdom (in its own terms).  Wouldn’t that be neat?

In a recent post to his excellent blog, Kailash Awati writes (and I quote at length)

As regular readers of this blog will know, I am a fan of dialogue mapping,  a conversation mapping technique pioneered by Jeff Conklin. Those unfamiliar with the technique will find a super-quick introduction here.  Dialogue mapping uses a visual notation called issue based information system (IBIS) which I have described in detail in this post.  IBIS was invented by Horst Rittel as a means to capture and clarify facets of   wicked problems – problems that are hard to define, let alone solve.  However, as I discuss in the paper, the technique also has utility in the much more mundane day-to-day business of managing projects.

In essence, IBIS provides a means to capture questions,  responses to questions and arguments for and against those responses. This, coupled with the fact that it is easy to use, makes it eminently suited to capturing conversations in which issues are debated and resolved. Dialogue mapping is therefore a great way to surface options, debate them and reach a “best for group” decision in real-time. The technique thus has many applications in organizational settings. I have used it regularly in project meetings, particularly those in which critical decisions regarding design or approach are being discussed.

Early last year I used the technique to kick-start a data warehousing initiative within the organisation I work for. In the paper I use this experience as a case-study to illustrate some key aspects and features of dialogue mapping that make it useful in project discussions.  For completeness I also discuss why other visual notations for decision and design rationale don’t work as well as IBIS for capturing conversations in real-time. However, the main rationale for the paper is to provide a short,  self-contained introduction to the technique via a realistic case-study.

Most project managers would have had to confront questions such as “what approach should we take to solve this problem?” in situations where there is not enough information to make a sound decision. In such situations, the only recourse one has is to dialogue – to talk it over with the team, and thereby reach a shared understanding of the options available. More often than not, a  consensus decision emerges from such dialogue.  Such a decision would be based on the collective knowledge of the team, not just that of an individual.  Dialogue mapping provides a means to get to such a collective decision.

And now for an unabashed commercial plug: Austhink Consulting provides “on demand” facilitation using dialogue mapping and related techniques for decision making and wicked problem “solving”.

Note:  this post first appeared on another blog back in 2008.  I’m reposting it here now because this blog is its natural home and because its main points appear to need frequent reiterating… 

________________________________

I’ve long had a suspicion that, just as knowledge management generally isn’t concerned with knowledge, so “business intelligence” is not really concerned with intelligence. Rather, in both cases, they’re primarily concerned with the management of information; the hard “knowledge” or “intelligence” part is usually left for the user. This is not to deny that knowledge management and business intelligence are useful; of course they are, in their own ways. It is just that there is a gap between their somewhat grand self-designations and the somewhat more mundane reality of what they do.

At BI Questions Blog, there is a video of Timo Elliott giving an interesting overview of business intelligence (BI) and what contemporary BI suites such as Business Objects can deliver in this area. It is quite an eye-opener in many ways, and well worth watching.

It gives a nice opportunity to elaborate the point about BI skipping over the “I” or intelligence part.

Here is TE describing what BI is fundamentally about:

So what do we mean by business intelligence success? Well at a very high level it means getting four things right:

  1. First, you need to be able to tame information chaos. You’ve got lots of different information in lots of different systems, structured data, unstructured data, documents, emails, there’s the web out there; so bringing all that information into one coherent structure so you can start doing something with it.
  2. Of course having information alone is useless; you need to turn that into insight. You need tools that actually let you look around and drill into the data and report it out to all of the people in the information who need it.
  3. The third thing you need to do is turn that insight into action. Looking at a report is useless. Something has to change in the business in order to get any value. Information has to be actionable. So to help you turn insight into action we provide a suite of business applications, in particular for the office of finance, to help you with financial planning, budgeting, consolidation, profitability and cost analysis – a whole suite of tools that really let you start doing things with the information.
  4. Now so far we could get through all of those stages and have optimum performance, we could have fantastic figures; but unfortunately we might get fantastic figures the way, say, Enron did. So we need one more block, which is governance, risk, compliance…

The accompanying visual is this:

But there’s something missing from this picture. In non-trivial or non-routine cases, you can’t (or shouldn’t) skip directly from insight to action. Insight, in TE’s description of it, appears to be a richer, more synthesized, more accessible form of information; it is what you’ve got when you’ve used their tools to “look around and drill into the data and report it out.” Between insight, in this sense, and action there have to be processes of assessing, deliberating, integrating, weighing, and choosing – in short, there has to be decision.

Decision making is the crucial bridge between information (even quality information, i.e. insight) and action.

To illustrate with an example drawn from a later part of TE’s presentation: using a nifty Business Objects component, he shows a chart of Disney corporate revenues over a few years. He then puts on the same chart a line showing the performance of the US economy over the same period; and he then shows the difference between the two:

Impressive! If you look closely, it seems that Disney does poorly in Q1 each year. It certainly does seem that we’ve instantly got some greater insight.

But what follows from this insight? What should Disney do? Close down in Q1 each year? Increase advertising in Q1 – or Q4? Lower prices? Hire more staff? Fire more staff? Nothing?

Before any action, you’d have to decide which action was most appropriate in the circumstances. The insight we obtained (and no doubt numerous others we could get from our wonderful BI suite) would surely help. But insights, no matter how penetrating or how numerous, don’t dictate any particular decision. The decision is generally made through a deliberative, usually collaborative process in which insights are translated into arguments and arguments are assessed and weighed.

This deliberative decision process, so pervasive in business, that is missing not only from the TE box graphic above but, it seems, from the whole BI mindset.

I’m reminded of the famous cartoon of two scientists:

Just as there are complex formula either side of the crucial “step two” on the blackboard, so business intelligence suites, it seems, provide technical power either side of the “miracle” of human deliberative decision.

What business intelligence suites (and knowledge management systems) seem to lack is any way to make the thinking behind core decision processes – the “step 2″ in moving from information to action – more explicit.

_____________________________

Timo Elliott responded:

Thanks for watching the presentation! I completely agree that BI needs more support for collaboration and the way people really make decisions. The only thing I’d argue with is where to put it — rather than being “between” insight and action, I think it’s essential all the way along the spectrum… You need to collaborate/decide what information is relevant, what insights are the correct ones, what actions are most appropriate, what controls to put in place, etc… Regards, Timo

Q: Can argument mapping be used in strategic planning?

A: Of course! – because strategic planning involves complex arguments, and argument mapping can help whenever you have to deal with complex arguments.

However to move beyond that sort of trite proclamation, it is useful to have concrete examples of how argument mapping can enhance a strategic planning process.

Austhink recently providing mapping expertise for a major Australian organisation developing its strategic outlook for a nominated date of 2030. In order to do detailed planning, leading to major decisions such as investing many billions of dollars in human resources and equipment, it had to first develop a conception of what its “operating environment” would be in 2030 and how the organisation would be able to achieve competitive advantage in that environment. The team developing this conception had drafted a document laying it out, including seven hypotheses as to how the organisation would be able to achieve advantage, with arguments to support the hypotheses. Necessarily these hypotheses and arguments were quite abstract, intended as they were to cover a wide range of scenarios.

Parenthetically, it is worth emphasizing how difficult this task is. We all know how rapidly the world is changing in all sort of respects (technology, geopolitics, climate etc.), and how unpredictable that change is. The more you try to say anything reasonably definite and useful about the 2030s, the more they appear to be hidden in a dense fog of uncertainty. Yet this organisation – like so many others – can’t just throw up its hands. It has to make conceptual and predictive commitments with very high stakes, for the organisation itself and indeed far beyond it.

Having developed a draft strategic conception, the organisation is now putting it through a fairly elaborate process of “stress testing”. This raises the question – how do you “put to the test” sets of arguments relating to highly abstract and intrinsically speculative propositions? Their idea, in essence,was to

  1. Articulate the arguments with as much clarity and rigor as possible
  2. With the help of a broad selection of domain experts, in a series of workshops, identify strengths and weaknesses, including
    - Gaps – places where key arguments are missing, or more substantiation is needed;
    - Assumptions – especially “hidden” assumptions, i.e. ones you haven’t realized you’ve been making;
    - Objections and challenges
  3. Use the findings to guide further development of the thinking

Developing good-quality argument maps in complex, murky territory is a challenging business. It involves getting sufficient clarity about what the issues are, and what arguments you have, and how they “hang together,” to be able to represent those issues and arguments in diagrams following the rules of argument mapping – which are really just fundamental principles of good logical thinking. It is inevitably an iterative process, with each draft resolving some matters but opening others for exploration.

In what follows, I’ll briefly recap this iterative process for just one of the seven argument maps we developed.  (Sorry that the illustrations are unreadable – this is deliberate to preserve confidentiality.)

As is typically the case, the arguments as we first encountered them were presented in standard prose:

I’ve discussed elsewhere how difficult it is to identify complex arguments in standard prose presentations, even when those arguments have been developed and written out by the sharpest of legal minds. In this case we were unsurprised to encounter the usual sorts of problems:

  • Arguments pertaining to a particular hypothesis were scattered in various places around the document and interspersed with other not-directly-related material.
  • The arguments were difficult to pin down, often because they were largely implicit.
  • The arguments were easy to misunderstand, if indeed one didn’t miss them altogether.
  • Consequently it was difficult to evaluate the arguments (i.e., judge with any confidence how effectively they supported the hypothesis).

In the first workshop with domain experts, we used real-time facilitated argument mapping with bCisive in an attempt to pin down and elaborate the main arguments, resulting in:

Many useful ideas had come out, but as you can see from the wide flat layout, were still struggling to find an appropriate overall structure. At this stage the map is poorly organised and missing a lot, but at least we could see more clearly what we had and how one thing supposedly relates to another.

We took the maps from the first workshop away and did some reworking, relying mostly on our generic argument mapping expertise (and only a little on commonsense and general knowledge of the domain). What emerged was a basic structure with more coherence, simplicity, and even elegance:

The overall structure is starting to emerge. Now we can distinguish between the higher level (more general, abstract) arguments and their lower-level supporting arguments. This “macro” is the structural “coat hanger” on which the rest can hang. This basic structure was now stable through the remaining iterations.

Aside: this was consistent with what I think of as one of the more profound insights I’ve derived from my years of experience with argument mapping: that complex arguments have a “true” form, a form which is (a) determined by the fundamental principles of good thinking meshing with the underlying reality of the issues, and (b) which uncoverable by patient reworking of the argument under the “rules” or guidelines of argument mapping.

During second workshop, a small number of valuable additions were made to the map:

But more importantly, participants used a “grouputer” system to jot down lots of additional ideas, which we took away and sorted and integrated into another reworked version of the map:

What we can now see emerging is a richer and more articulated sense of the case bearing on the hypothesis. We can clearly see both major lines of supporting argument. We know which claims have been supported and which have not. We can see key objections or warnings (little red blobs in the graphic above). We can see numerous places where unstated assumptions are lurking.

A map like this positions us well to make a provisional judgement as to how well the hypothesis (the main contention in the map) is supported. It also helps one see the numerous things one could do to further elaborate the thinking and develop greater confidence in that judgement. From the standpoint afforded by this map, it is clear that the arguments as originally presented simply couldn’t be properly evaluated. When you have only a very fuzzy sense of what the arguments are, you can have at best only a fuzzy sense of whether they are any good. You are then more likely to be guided by prejudice, bias, habit, instinct or “conventional wisdom”.

My friend and colleague Paul Monk last night delivered a speech to the Australian Institute of International Affairs, “Keeping the Insulation from Coming Undone: American Debt, Wall Street and Looming Realities.”

An excerpt:

A United States in sound economic order and sound political order would always have been able to absorb any conceivable blow that al Qaeda could have dealt it. The United States, however, is not in sound economic or political order. It has delivered a series of mighty blows to itself, all on its own, and they are far from having been fully absorbed as we gather here this evening. The image of the World Trade Center  twin towers crumbling as the sun shone over New York was unforgettably dramatic, but the realities of the US economy being shaken to its very foundations by fiscal mismanagement, corporate corruption and the sheer stupidity of Wall Street bond traders over the past decade should actually disturb us a great deal more. The increasingly polarized and demagogical character of politics in the United States; the dominance of bully pulpit propaganda and irresponsible rhetoric over serious and principled debate are profoundly troubling.

 

Older Posts »

Follow

Get every new post delivered to your Inbox.