Feeds:
Posts
Comments

Archive for the ‘Wisdom of Crowds’ Category

[This post is in response to a request from a colleague for “the best thing to introduce someone to YourView”.]

The goal of the YourView project was to develop a platform for identifying collective wisdom, focusing on major public issues, aiming to remedy some of the defects of democracy.  The platform was a deliberative aggregator, i.e. it supported large-scale deliberation and its aggregated outputs reflected that deliberation.  The project was active from around 2011 to 2014, with its moment of glory being when the platform was used as part of Fairfax Media’s coverage of the 2013 Federal Election.

The YourView platform is still alive and can be explored, though there has been no real activity on the site since 2014.  There is a collection of pages and links about the YourView project.

The best theoretical overview is Cultivating Deliberation for Democracy, particularly the second half.  The “The Zone” interview by Fairfax’s Michael Short is a good read.

Advertisements

Read Full Post »

Prediction markets can be a remarkably effective way to divine the wisdom of crowds.

Prediction markets of course only work for predictions – or more generally for what I call “verifiable” questions.   A verifiable question is one for which it is possible, at some point, to determine the answer definitively.  For example, predicting the winner of the Oscar for best picture.   This is what allows the prediction market to determine how much each player wins or loses.

The problem is that many issues we want addressed are not verifiable in this sense.

For example, decisions.  Would it be better to continue to negotiate with Iran over its nuclear ambitions, or should a military strike be launched?   We can speculate and debate about this, but we’ll never know the answer for sure, because one path will be taken, and the other never taken, and so we’ll never know what would have happened had we taken the other path.

Wouldn’t it be good if we had something like a prediction market, but which works for non-verifiable issues?

Amazon.com book ratings are an interesting case.   Whether a book is or is not a good one is certainly a non-verifiable issue.   Yet Amazon has created a mechanism for combining the views of many people into a single collective verdict, e.g. 4.5 stars.   At one level the system is just counting votes; Amazon users vote by choosing a numerical star level, and Amazon averages these.   But note that Amazon’s product pages also allow users to make comments, and reply to comments; and these comment streams can involve quite a lot of debate.   It is plausible that, at least sometimes, a user’s vote is influenced by these comments.   So the overall rating is at least somewhat influenced by collective deliberation over the merits of the book.

Amazon’s mechanism is an instance of a more general class, for which I’ve coined the term “deliberative aggregator“.   A deliberative aggregator has three key features:

  1. It is some kind of virtual forum, thereby allowing large-scale, remote and asynchronous participation.
  2. It supports deliberation, and its outputs in some way depend on or at least are influenced by that deliberation.  (That’s what makes it “deliberative.”)
  3. It aggregates data of some kind (e.g. ratings) to produce a collective viewpoint or judgement.

YourView is another example of a deliberative aggregator.   Yourview’s aggregation mechanism (currently) is to compute the “weighted vote,” i.e. the votes of users weighted by their credibility, where a user’s credibility is a score, built up over time, indicating the extent to which, in their participation on YourView, they have exhibited “epistemic virtues,” i.e. the general traits of good thinkers.

Many other kinds of deliberative aggregators would be possible.   An interesting theoretical question is: what is the best design for a deliberative aggregator?  And more generally: what is the best way to discern collective wisdom for non-verifiable questions?

Read Full Post »

I have a short paper appearing next month in the Journal of Public Deliberation.  A preview is available here.  Below is a precis.

In its first half, “Cultivating Deliberation for Democracy” discusses the failure of “deliberation technologies” to substantially improve public deliberation in either quantity or quality.   To be sure, new technologies have made possible massive quantities of deliberation of a very public kind (e.g. in public forums such as comments in the New York Times).  However those technologies are not specifically deliberation technologies.  Nothing about them is specifically tailored to support deliberation as opposed to other forms of public conversation.  Meanwhile, deliberation technologies properly so-called – including my own previous efforts – have notably failed to be adopted by the public at large.  I explain this by pointing out the obvious: people don’t like to be “boxed in” by the kinds of constraints typically provided in deliberation technologies.

The second half gives an overview of the YourView project.  YourView is a deliberation technology, but tries to take a rather different approach, aiming to cultivate rather than construct quality public deliberation. YourView provides a forum in which participants can vote and comment on major public issues.  What makes YourView distinctive is that it attempts to determine the “collective wisdom” of the participants.  It does this by calculating, for each participant, a “credibility” score, using data generated through their participant and others’ responses.   In more philosophical terms, YourView attempts to determine the extent to which a participant is exhibiting various “epistemic virtues” such as open-mindedness.  Credibility scores are useful in two ways.  First, they enable YourView to calculate the collective wisdom by weighting contributions by credibility.   Second, they drive more, and more thoughtful, engagement on the site, because high credibility translates to status and (in some ways) power in the YourView forum.

 

Read Full Post »

Two perceptive comments on the state of democracy in Australia, from yesterday’s Age.  First Barry Jones laments the dismal state of political discourse:

I have been heavily involved in politics all my adult life and the current national situation, both in the government and opposition, is a low point, the lowest I can recall – even the dark days of 1955, 1966, 1975 and 1996. It seems to get worse every week.  The 2010 federal election was the worst in my memory because there was no debate about ideas, simply an exchange of slogans and mantras (”Stop the boats!”).  The word ”because”, leading to an attempt at reasoned explanation, seemed to have fallen out of the political lexicon. We observed an infantilisation of debate, for example on refugees and climate change.  There is good reason to expect that the 2013 election will be even more depressing. I have lost count of the number of exchanges I have had with voters in Melbourne streets where they express their dismay to me about the state of politics, on both sides. Some burst into tears.

This reiterates a point in an Age opinion piece back in July:

Tony Abbott’s approach to the carbon tax debate is illustrative of a general collapse in the quality of rational discourse. The proposed carbon tax, a very complex issue, is being attacked with ruthless simplicity, ”Stop a toxic tax, based on a lie.” Is there a second sentence in this argument? Has the word ”because” fallen out of the political vocabulary?

Then James Button, pointing towards an explanation:

Strangely, the information age seems to have made grasping the truth of things harder. The shrinking of the broad base of political parties, their failure to tell stories that inspire and ring true, the increasing lack of penetration of the serious media, the rarity of deep analysis told in a compelling way, the 60-second YouTube videos that portray Julia as robotic or Kevin as a knockabout bloke who swears a bit much, the distrust and distractedness of we the people – they all promote misunderstanding. They are death to an engaged politics.

Neither however has much to say about how to fix or ameloriate the problem.  They seem resigned to it, or perhaps hope that describing the problem will somehow help turn things around.  In this they are like Lindsay Tanner, whose Sideshow was an entertaining book-length treatment of this territory but was also short on solutions (see the tepid final chapter).

Button suggests that “the information age” is to blame, at least partly, for the degradation of politics.  Yet it is also the only place where countervailing forces are likely to arise.  And indeed we are seeing a sudden proliferation of new forms of democratic engagement (Duval, Next Generation Democracy; Shirky, Here Comes Everybody).

Our own fledgling effort in this area, YourView, is aimed at (among other things) helping interested citizens to “grasp the truth of things” by making the key arguments on major public issues easily accessible, allowing citizens to express their view, and identifying the “wisdom of the crowd.”

Read Full Post »

A new draft of What Do We Think?  Divining the Public Wisdom to Guide Sustainability Decisions is now available.

Download PDF

Read Full Post »

What do we need?

If the transition to sustainability requires the public wisdom, and if we currently have no practical and effective mechanism for ascertaining that wisdom, then we need to develop something better.

What would such a mechanism look like?  Here’s a wish list:

  • It would generate public wisdom in the fullest sense – i.e. the collective considered opinion based on large-scale deliberation.
  • It would be generating that wisdom on all major issues, including new issues as soon as they arise.
  • It would make that wisdom available to anyone at any time.
  • It would by inclusive in the sense of providing a practical opportunity for any interested citizen to participate, and would in fact involve participation of numerous and diverse members of the public.
  • It would be politically neutral and completely independent of control by government, corporates or any other powerful interest group.

A National Virtual Forum

Surveying the wish list, it is obvious that any mechanism capable of delivering the goods would have to be internet-based. It would have to be, in other words, a kind of national virtual forum (NVF).

No such forum exists today.  The good news however is that a NVF plausibly could exist.   As everyone knows, the internet hosts innumerable forums already; many are focused on serious social, economic and political issues, and support deliberation that is often of surprisingly high quality.  While it is de rigueur to sneer at the quality of online discussion, and indeed much of it is rubbish, we should at the same time acknowledge that every day literally thousands of Australians jump online and vigorously debate the major issues of the day.

Further, and more profoundly, there is the fact that internet-based environments or systems have been proven capable of synthesizing collective intelligence or wisdom of various kinds.  Wikipedia, prediction markets, Amazon.com, and Stack Overflow are all well known examples.  To be sure, none of these generate collective rational consensus of the kind expected from the NVF.  How exactly that form of collective intelligence will be assembled or extracted is a major design challenge.  But important precedents do exist, and they do more than just prove that collective intelligence can be generated: they provide a wealth of insights and hints for the development of a NVF.

Challenges

A NVF would clearly face numerous major obstacles.  In my view, these are best regarded as challenges to be overcome rather than fatal objections to the whole exercise.   Here are four, with brief hints as to how they might be tackled.

  1. Critical Mass.  The NVF will have to attract many and diverse participants.  To do this, first and foremost the NVF must be easily accessible – simple to use and available via any major channel (website, mobile apps, etc.).  It must be thoroughly and effectively integrated with social media (Twitter, etc.).   “Gamification” techniques will help deepen participants’ engagement.  Finally, a major media alliance will situate the NVF in the public’s attention (similar to, say, the Oursay cooperation with The Age).
  2. Representativeness.  For its outputs to count as the wisdom of the public as a whole, the participants would need to be sufficiently similar to the public – i.e., to statistically represent the public.  On the face of it, this will be a problem if the NVF has an open-door approach, allowing its participants to self-select.  Despite this various strategies can be used to approximate and enhance representativeness, approaching full representativeness as a kind of limit case.  For example, assuming there are demographics on participants of a known degree of reliability, and a sufficiently large and diverse set of participants, it would be possible to select suitable subsets of participants to form the pool for the purposes of computing group wisdom.
  3. Gaming.  If it builds any kind of momentum, the NVF will become a target for “gaming” (e.g. astroturfing) as groups attempt to manipulate the outputs to suit their own interests.  This problem can never by fully solved, but could be handled adequately.  The problem of distinguishing genuine from bogus participation is similar to the problem of distinguishing genuine email from spam, and Google has shown that this can be done remarkably well.
  4. Credibility/Influence.  The main point of setting up the NVF is to help governments make the best decisions.  For this to work, governments would have to take the NVF outputs seriously.  I’m optimistic that this problem would start to solve itself just insofar as the NVF achieves critical mass and delivers its intended output – not because governments will be virtuous and do the right thing but rather because they will inevitably start to respond out of pure pragmatic political self-interest.  If the genuine considered opinion of the public on a major issue is available, and if it diverges significantly from the public attitude as expressed in the polls, then it will constitute another kind of political cudgel which can be used by either the government or the opposition.

Conclusion – Now is the time to start

Clearly, establishing a NVF of the kind described would be no mean feat.  Yet as I’ve argued, we need such a thing if we’re to make a smooth, timely transition to sustainability.

It is high time we had practical and effective mechanisms for knowing what the public really thinks on the major issues affecting it.   The ubiquity and sophistication of the internet and the systems built upon it provide us the opportunity to realise this democratic ideal.

The NVF proper will not be built in a day or even a year.  Rather, it will evolve in a serious of stages, incrementally approaching the full vision.

Eight years (from here to 2020) is probably a reasonable time-frame within which something worthwhile could be achieved.  Remember that Twitter is less than eight years old, and has already played a key role in democratic movements worldwide (e.g. North Africa).


This post is the third part of a draft chapter What Do We Think? Identifying the Public Wisdom to Guide Sustainability Decisions, in preparation for the volume 20/20 Vision for a Sustainable Society, being put together by the Melbourne Sustainable Society Institute.

Previous:

Read Full Post »

One of the main points of this chapter is that to make the transition to sustainability in a safe and timely manner we need to identify the public wisdom on sustainability issues.

Why?

First, because it will help governments to make the decisions we need them to make. The public wisdom can give the government the kind of “mandate” or authority it needs to tackle divisive issues and make tough decisions, even when doing so may be going against the tide of public opinion as measured by the polls.

Knowing the public wisdom on the matter may also help swing public opinion.

This potential benefit was behind Julia Gillard’s recommendation, in the 2010 election campaign, that a 150-person Citizen’s Assembly be convened to develop some rational consensus around climate change policy.  She apparently believed that identifying the considered opinon of the public on the issue would help break the political impasse.  The proposal had some intrinsic merit but was, as Lyn Carson has described, ridiculed from many directions, with such rejection driven by many different agendas and misunderstandings.   One lesson of this episode is that we need institutions and mechanisms capable of articulating the collective wisdom without requiring any support or approval from the powers that be (see below).

Second, because on many issues, the public wisdom would be best guide to the truth of the matter.  If we’re serious about making the right decision, then we must find out what the public really thinks.

Consider an issue like whether we should have more large dams to better manage scarce water resources.   This simple-sounding question sits on top of a complex web of issues, involving not just factual and technical matters but diverse competing interests and conflicting values.  Certainly many individual experts and interest groups are highly knowledgeable about particular aspects, and their input should be given due consideration.  However such folks always have a particular perspective; they see only their part of the larger elephant.  The wider the involvement – the more diverse and comprehensive the selection of participants – the more chance that all the relevant information can be brought to the decision, and the relevant interests and values recognised and accommodated.

Note that I’m not claiming that the public’s considered opinion is the best guide on any complex matter.  Many issues clearly are matters of specialist expertise, and the general public is in no position assess the merits of different theories.  An obvious example is the science of climate change.  Only the body of climate scientists has the knowledge and competence to settle the scientific issues.  Neither laypeople individually nor the public as a whole have any business trying to make up their own minds on this topic.

However on major sustainability issues there are no individuals or special groups in a uniquely privileged position to discern the truth.  These decisions are matters of interests and values as much as they are matters of knowledge or expertise, and the Australian people are the relevant authorities on what their interests and values are.

But we don’t know what the public wisdom is

The problem with the public wisdom is that we almost never know what it is.  That is, on any given major issue, we don’t know what the collective considered opinion is.  Indeed, that wisdom usually doesn’t even exist, in the sense that nothing has been done to put it together.

We do have many windows onto public opinion, but they’re all either ineffective (don’t deliver public wisdom) or impractical (too cumbersome and expensive).

For example standard opinion polls, for reasons described above, don’t tell us what the public wisdom is.

Well-designed surveys are a step up from standard opinion polls (Reser).  However, these surveys provide little opportunity for the respondents to engage in any sustained reflection, individually or collectively.   They are just more sensitive ways of identifying the attitudes people happen to have.  Ideally, such surveys would play a much larger role than they currently do in the gauging of public opinion.   However they can’t identify the public wisdom, in the sense described above.

Deliberative polling, and why it is not enough

By far the best mechanism we currently have for ascertaining the public wisdom is deliberative polling.  It is summarised on the Stanford Center for Deliberative Democracy website as follows:

A random, representative sample is first polled on the targeted issues. After this baseline poll, members of the sample are invited to gather at a single place for a weekend in order to discuss the issues. Carefully balanced briefing materials are sent to the participants and are also made publicly available. The participants engage in dialogue with competing experts and political leaders based on questions they develop in small group discussions with trained moderators. Parts of the weekend events are broadcast on television, either live or in taped and edited form. After the deliberations, the sample is again asked the original questions. The resulting changes in opinion represent the conclusions the public would reach, if people had opportunity to become more informed and more engaged by the issues.

Over the past few decades, dozens of deliberative polls have been conducted around the world.  In Australia has had a handful, on topics such as republicanism and reconciliation.

Australia would benefit greatly if deliberative polls were held much more often, and if their results were more influential in major decisions.

However, deliberative polling, in its standard form at least, can’t meet the need to deliver the collective wisdom for the purpose of guiding timely decision making on major sustainability issues.

The critical problem is that deliberative poll is a cumbersome exercise and is costly to stage.  This has a number of consequences:

  • There aren’t enough of them.  The large cost is one major reason there have been so few deliberative polls since the idea was first propounded over two decades ago.  It may be that the frequency of deliberative polls is increasing, which is a surely a good thing, but deliberative polling currently and for the foreseeable future can address only a fraction of the issues which properly ought to be guided by public wisdom.
  • They take a long time to set up.  It can take six months or more to set up and run a deliberative poll.  The time from conception – the moment when it is recognised that having a DP on a certain topic would desirable – is far longer.  And of course most deliberative polls that have been conceived simply haven’t been run (yet).
  • Once run, they’re finished.  The public wisdom identified in the deliberative poll is frozen in time.   It becomes outdated and irrelevant as circumstances and information change.

The latter two points may not be such a problem for relatively timeless issues such as whether Australia should become a republic, but they constitute a severe drawback when decisions need to be made quickly on issues which are heavily shaped by circumstances arising at a particular moment in history.

For example, should Australia adopt the Gillard governments carbon pricing scheme?  This is not the general issue of e.g. whether Australia should take action on climate change, and whether it should institute an emissions trading scheme.  Rather it is whether a particular plan should be adopted at a particular historical juncture.  The debate is raging as this is being written, and ideally we would be able to divine the collective wisdom right now.   A deliberative poll on the topic would be great, but it isn’t happening, and practically speaking couldn’t happen for many months.  By the time a deliberative poll was staged, it may well be too late.

A more philosophical quibble with deliberative polling is that, as standardly conducted, it doesn’t deliver public wisdom in the fullest sense.   The primary output of the deliberative poll is the poll results – i.e. a tabulation of individual opinions.  Granted, these individual opinions have become more considered through quality deliberation, and are thus worthy of more respect than the attitudes tapped by standard opinion polls.  However there has been no deeper aggregation of individual judgement into a coherent collective viewpoint.   It is as if the IPCC reports were to consist of an exit poll of climate scientists’ beliefs, rather than a carefully drafted and agreed statement.


This post is the second part of a draft chapter What Do We Think? Identifying the Public Wisdom to Guide Sustainability Decisions, in preparation for the volume 20/20 Vision for a Sustainable Society, being put together by the Melbourne Sustainable Society Institute.

Previous:

Coming up:

Read Full Post »

Older Posts »