Archive for the ‘Business’ Category

I’ve had the following abstract accepted for a presentation at a conference in December at the University of Melbourne, Higher Education Research & the Student Learning Experience in Business.

A Pragmatic Definition of Critical Thinking for Business

This presentation will lay out a pragmatic definition of critical thinking.  It doesn’t purport to be the definitive characterization of what critical thinking is. Rather, it is offered as a convenient framework for understanding the nature and scope of critical thinking, which may be useful for purposes such as developing a dedicated subject in critical thinking for business, improving the teaching of critical thinking within existing subjects, or evaluating the effectiveness of a business course in developing critical thinking.

The definition is constructed around five commitments:

    • First, the essence of critical thinking is correct or accurate judgement. That is, to think critically is to think in ways that are conducive to being “more right more often” when making judgements.
    • Second, “being more right more often” can be achieved through the skillful application of general thinking methods or techniques.
    • Third, these techniques range on a spectrum from the simple and easily acquired to technical methods which require special training.
    • Fourth, for all but the simplest of methods, there are degrees of mastery in application of these techniques.
    • Fifth, there are many different kinds of judgements made in business, including decision making, prediction, estimation, (causal) explanation, and attribution of responsibility. For each major type of judgement, there are typical pitfalls, and a range of critical thinking methods which can help people avoid or compensate for those pitfalls.

These commitments enable us to define a kind of three-dimensional chart representing the critical thinking competency of any individual. Along one (categorical) axis is the various kinds of judgements (decision making, etc.). Another axis represents the spectrum from simple through to advanced critical thinking methods. Particular methods can then be placed in appropriate “boxes” in the grid defined by these axes. A person will have a degree of mastery of the methods in each box; this can be represented on a third dimension. A person’s critical thinking competency is thus a distinctive “landscape” formed by the varying levels of mastery.

This characterisation is tailoring, for business, a more general pragmatic approach to understanding critical thinking.  About a year ago I developed this approach in preparation for a workshop in the US on development of a test of critical thinking for intelligence analysts; my role in the workshop was to lay out a general framework for understanding what critical thinking is.   That approach was described in a manuscript Dimensions of Critical Thinking.

I’m also supporting a team from the University of Sydney Business School, who have had the following abstract accepted:

Evaluating critical thinking skill gains in a business subject

Helen Parker, Leanne Piggott, Lyn Carson
University of Sydney Business School
Tim van Gelder
University of Melbourne and Austhink Consulting

Critical thinking (CT) is one of the most valued attributes of business school graduates, and many business school subjects claim to enhance it. These subjects frequently implement pedagogical strategies of various kinds aimed at improving CT skills. Rarely however are these efforts accompanied by any rigorous evaluation of CT skill gains. But without such evaluation, it is difficult to answer questions such as:

    • Are our students’ CT skills in fact improving? By how much?
    • Are those skills improving more than they would have even without our special CT instruction?
    • Are the marginal gains worth the cost?
    • Are our attempts to improve our instruction from semester to semester making any difference?

These kinds of questions are particularly relevant to the University of Sydney Business School, which has an entire subject dedicated to improving CT (BUSS5000 – Critical Thinking in Business), enrolling some 800 students per semester. Consequently, in 2013, the Business School embarked on a large-scale, multi-year evaluation program. The evaluation is based on pre- and post-testing using an independent objective test (the Halpern Critical Thinking Assessment), whose coverage overlaps with the range of critical thinking skills taught in the subject. This presentation will give an overview of the approach it has adopted. It will discuss some of the challenges and pitfalls in the testing process, and how to interpret results. Finally, it will present data and insights from the first semester of full-scale evaluation. The session should be of interest to anyone interested in evaluating CT skills, or more generally in how business school education can enhance CT.

There’s an obvious complementarity between these two topics.


Read Full Post »

In a murder trial, to prove that a defendant was guilty the prosecution must establish that:

  • The victim was killed;
  • The defendant unlawfully caused the death of the victim; and
  • The defendant did so with “malice aforethought.”

These are known as the elements of the crime.   Elements have been defined for many legal actions.  They are the major things which all must be proved in order to establish the prosecution’s (or plaintiff’s) case.

We can by analogy think of the elements of a major business decision.   In a board context, where typically management will recommend an option for the board to approve or reject,  these are the major things that management must establish to the board’s satisfaction.  The recommended option must be:

  1. Strategically sound;
  2. Financially sound;
  3. Operationally sound;
  4. Prudentially sound (i.e., acceptable from a risk perspective)
  5. Ethically sound; and
  6. Legally sound.

These are the minimal conditions; a decision satisfying these conditions would be reasonable or defensible.  It may not be the best decision.  For that, an additional element point must be established:

  • The option is on balance better than any other relevant option across elements 1-6.

These elements are often not independent matters.  For example, what counts as financially sound will depend on the organisation’s strategy (and ultimately of course on its purpose).

Each element must in turn be established by argumentation, governed by standard principles of clarity and rigor.

Associated with each element are a series of critical questions. Addressing the critical questions generates supporting arguments or objections.   For example, critical questions for strategic soundness might include:

  • Is there a wider corporate strategy pertaining to this type of decision?
  • If so does the proposed option align with that strategy?
  • If there is no wider strategy, or no alignment, is this option nevertheless strategically defensible?

The elements listed above are generic, suiting a typical major business decision.  However, just as different crimes have their distinctive sets of elements, so particular categories of business decision would have tailored sets of elements.

The “elements of the case” approach can be used in a variety of ways.  It can for example guide the development and structure of the board paper and presentation in which the recommendation is advanced.  Or, it might be used within the board meeting to structure attention and discussion.

Read Full Post »

Two recent articles in the business press underscore the importance of unearthing and challenging the assumptions which are shoring up your inferences.  They also provide fascinating insights more generally into organisational decision, a topic which always becomes more interesting when things go badly wrong.

The first, Potash: The deal that didn’t have to die appeared in the Canadian newspaper The Globe and Mail in the aftermath of the failed attempt by mining behemoth BHP Billiton to acquire Potash Corp.   Potash Corp makes a lot of money digging up – you guessed it – potash in Sasketchewan and selling it around the world.  Potash Corp is part of a cartel, the existence of which helps keep the Saskatchewan state coffers replenished.

As authors McNish, Bouw and Reguly state, in the lead-up to the bid,

[BHPB CEO] Mr. Kloppers had every reason to believe that the takeover odds were on his side. Potash prices were still climbing out of the cyclical basement, and few other companies in the world could match BHP’s financial heft to top the bid. A government-backed company from China probably could, but no Chinese entity has ever tried to make such a large acquisition in a developed, democratic country. Even better, the widely held Saskatchewan company had no blocking shareholders. The Canadian government had never rejected a foreign takeover of a major resource company, and just three years ago allowed Rio Tinto PLC to take over Alcan Inc. of Montreal in a deal of similar size.

Focusing on one aspect of this, it appears that BHPB were reasoning that if the Canadian government had never rejected a foreign takeover of a major resource company, then they’d be unlikely to disapprove the takeover by BHPB of Potash Corp.   Which seems a perfectly reasonable argument.

Of course there is an unstated premise in this argument, viz., that Potash Corp is a major resource company.  This seems so obvious that it is hardly worth mentioning – and ipso facto hardly worth challenging.  But often the critical weakness in a case can be disguised as a truism even after we take away the cloak of invisibility.   Sometimes the highest level of critical scrutiny is not challenging the dubious but daring to question the obvious.

The real force of the assumption “Potash Corp is a major resource company” was the idea that Potash Corp was just another major resource company, just like all the previous ones that had been taken over.  That, in other words, there was nothing special about Potash Corp, in Canada in 2010, that would prevent it being taken over just like others before it.

But that assumption was wrong.  Potash Corp was in fact, at the time, quite special due to the close connection with Saskatchewan state funding, and the special connection of Saskatchewan politicians to the Canadian federal government.

As Andrew Mackenzie, a key BHPB executive, conceded after the failure of the takeover bid: “We didn’t grasp how significant potash is to Saskatchewan.”

If the authors are correct, this “failure to grasp” – a kind of unchallenged assumption – was the critical error leading to a failed bid.  Some of the consequences: immediate waste of some $350 million dollars, loss of potential billions in future profits from selling Canadian potash, embarrassment for BHPB, and a major headache: what now should it do with the mountains of money it is making due the resource boom?

The second article appeared in today’s edition of the Melbourne paper The Age.  In AXA deal could bring $2.6bn headache David Symons recounts how some financial sharks spotted a design weakness in superannuation giant AXA’s offerings and tricked AXA into granting them the right to exploit that weakness mercifully for decades to come – hence the $2.6 billion dollar lawsuit.  The details are of course somewhat complex but on Symons’ account AXA management – apparently lurching from one feat of incompetence to another – repeatedly made assumptions which turned out false.  For example:

“anxious to resurrect a faltering product, AXA ignored the embedded option risk. AXA management had referred the special terms to the legal department and the board product committee, but not to the chief actuary. The three-day option was considered a tweak that didn’t require costing. Nothing could have been further from the truth. AXA had effectively written a multibillion dollar series of puts over the equity index.”

Quite a different domain, but the same basic point: sometimes an unquestioned assumption comes back to bite you in a very costly way.

Of course in both these cases we have the wonderful benefit of 20/20 hindsight.  It is relatively easy to spot a false assumption after its falsity has caused a calamity.  It is much harder to expose all the assumptions one is making at the time of decision, and to know which of those many assumptions need to given especially rigorous scrutiny.

One of the great benefits of argument mapping, rigorously applied, is its utility in the former challenge, i.e. exposing hidden assumptions.

Before closing it is worth noting another kind of assumption in play in the AXA case.  The aforementioned sharks clearly devoted much of their exquisite intellectual power to figuring out how to exploit the financial system so as to attract to themselves vast amounts of money of unearned and undeserved money.  Rather than doing anything productive, anything that might generate tangible value for society, these knaves are prepared to follow their greed to the point where they might even ruin an enormous institution (cf Equitable Life), with presumably masses of shareholders and policy holders and perhaps even taxpayers to take the hurt.   They are assuming that this is reasonable, legitimate conduct.   That assumption may not come back to bite them in this particular case.  But when large parts of society unconsciously and unconscionably adopt that assumption, it can be disastrous for almost everyone – as Ireland’s current problems illustrate.

Read Full Post »

A common decision making trap is thinking more data = better decision – and so, to make a better decision, you should go out and get more data.  

Let’s call this the datacentric fallacy.  

Of course there are times when you don’t have enough information, when having more information (of the right kind) would improve the decision, and when having some key piece of information would make all the difference.  

Victims of datacentrism however reflexively embark on an obsessive search for ever more information.  They amass mountains of material in hope that they’ll stumble across some critical piece, or critical mass, that will suddenly make clear what the right choice is.  But they are usually chasing a mirage.  

In their addiction to information, what they’re neglecting is the thinking that makes use of all the information they’re gathering.  

As a general rule, quality of thinking is more important than quantity of data.  Which means that you’ll usually be better rewarded by putting any time or energy you have available for decision making into quality control of your thinking rather than searching for more/better/different information.

Richards Heuer made this point in his classic Psychology of Intelligence Analysis.  Indeed he has a chapter on it, called Do You Really Need More Information? (Answer – often, no.  In fact it may hurt you.) 

A similar theme plays out strongly in Phil Rosenzweig’s The Halo Effect… and the Eight Other Business Delusions That Deceive Managers. Rosenzweig provides a scathing critique of business “classics” such as In Pursuit of Excellence, Good to Great and Built to Last, which purport to tell you the magic ingredients for success.  

He points out how in such books  the authors devote much time and effort to boasting about the enormous amount of research they’ve done, and the vast quantities of data they’ve utilised, as if the sheer weight of this information will somehow put their conclusions beyond question.  

Rosenzweig points out that it doesn’t matter how much data you’ve got if you think about it the wrong way.  And think about it the wrong way they did, all being victims of the “halo effect” (among other problems).  In these cases, they failed to realise that the information they were gathering so diligently had been irretrievably corrupted even before they got to it.  

Another place you can find datacentrism  running rampant is in the BI or “business intelligence” industry.  These are the folks who sell software systems for organising, finding, massaging and displaying data in support of business decision making.   BI people tend to think decisions fall automatically out of data, and so presenting more and more data in ever prettier ways is the path to better decision making.

Stephen Few, in his excellent blog Visual Business Intelligence, has made a number of posts taking the industry to task for this obsession with data at the expense of insightful analysis.  

The latest incidence of datacentrism to come my way is courtesy of the Harvard Business Review.  I’ve been perusing this august journal in pursuit of the received wisdom about decision making in the business world.   In a recent post, I complained that the 2006 HBR article How Do Well-Run Boards Make Decisions? told us nothing very useful about how well-run boards make decisions.  

I was hoping to be more impressed by the 2006 article The Seasoned Executive’s Decision Making Style.  The basic story here is that decision making styles change as you go up the corporate ladder, and if you want to continue climbing that ladder you’d better make sure your style evolves in the right way.  (Hint: become more “flexible.”) 

In a sidebar, the authors make a datacentric dash to establish the irrefutablity of their conclusions:

For this study, we tapped Korn/Ferry International’s database of detailed information on  more than 200,000 predominantly North American executives, managers, and business professionals in a huge array of industries and in companies ranging from the Fortune 100 to startups. We examined educational backgrounds, career histories, and income, as well as standardized behavioral assessment profiles for each individual. We whittled the database down to just over 120,000 individuals currently employed in one of five levels of management from entry level to the top.  We then looked at the profiles of people at those five levels of management. This put us in an excellent position to draw conclusions about the behavioral qualities needed for success at each level and to see how those qualities change from one management level to another.

120,000.  Wow. 

They continue:

These patterns are not flukes. When we computed standard analyses of variance to determine whether these differences occurred by chance, the computer spit out nothing but zeroes, even when the probability numbers were worked out to ten decimal points.  That means that the probability of the patterns occurring by chance is less than one in 10 billion. Our conclusion: The observed patterns come as close to statistical fact (as opposed to inference) as we have ever seen.

This seems too good to be true.   Maybe their thinking is going a bit off track here?  

I ran the passage past a psychologist colleague who happens to be a world leader in statistical reform in the social sciences, Professor Geoff Cumming of Latrobe University.   I asked for his “statistician’s horse sense” concerning these impressive claims.  He replied [quoted here with permission]:

P-value purple prose! I love it!

Several aspects to consider. As you know, a p value is Prob(the observed result, or one even more extreme, will occur|there is no true effect). In other words, the conditional prob of our result (or more extreme), assuming the null hypoth is true.

It’s one of the commonest errors (often made, shamefully, in stats textbooks) to equate that conditional prob with the prob that the effect ‘is due to chance’. The ‘inverse probability fallacy’. The second last sentence is a flamboyant statement of that fallacy. (Because it does not state the essential assumption ‘if the null is true’.)

An extremely low p value, as the purple prose is claiming, often in practice (with the typical small samples used in most research) accompanies a result that is large and, maybe, important. But it no way guarantees it. A tiny, trivial effect can give a tiny p value if our sample is large enough. A ‘sample’ of 120,000 is so large that even the very tiniest real effect will give a tiny p. With such large datasets it’s crazy even to think of calculating a p value. Any difference in the descriptive statistics will be massively statistically significant. (‘statistical fact’)

Whether such differences are large, or important, are two totally different issues, and p values can’t say anything about that. They are matters for informed judgment, not the statistician. Stating, and interpreting, any differences is way more important than p-p-purple prose! 

So their interpretation of their data – at least, its statistical reliability – amounts to a “flamboyant statement” of “one of the commonest errors.” Indeed according to Geoff it was “crazy to even think of” treating their data this way.  

The bulk of their article talks about the kinds of patterns they found, and maybe their main conclusions hold up despite the mauling of the statistics.  Maybe.   Actually I suspect their inferences have even more serious problems than committing the inverse probability fallacy – but that’s a topic for another time.  

In sum, beyond a certain point, the sheer volume of your data or information matters much less than thinking about it soundly and insightfully.  Datacentrism, illustrated here, is a kind of intellectual illness which privileges information gathering – which is generally relatively easy to do – over thinking, which is often much harder.

Read Full Post »

Everyweek decisions

In a nutshell: describes the distinctive character of decisions managers or executives must make on a weekly basis.

It is a curious fact about the English language that the second most common word (“of”) occurs about half as often as the most common word (“the”); the third most common word occurs 1/3 as often as the most common word; and this pattern holds pretty well for the first 1000 words or more.

This pattern, known as Zipf’s Law, or more generally a power law distribution, occurs for a suprisingly diverse range of phenomena, from cities (the bigger they get, the fewer there are of that size) to earthquakes to price variations.

I don’t know of any any measurements to verify this, but it seems plausible that such a pattern would hold for business decisions as well. These decisions can be rated in terms of their level of significance, ranging from the trivial or everyday (Where should we go for lunch? Use “Yours sincerely” or “Cheers”?) to the truly momentous (Should we pay bribes to Saddam Hussein to sell more wheat?). In a given period, the typical manager or executive would make a great many everyday decisions, very few momentous ones, and a generous handful of “mid-sized” decisions.

Making everyday decisions would generally take only moments or at most minutes; momentous decisions might take months or years. Somewhere in between are decisions that the typical decision maker would make regularly but not everyday; she might make a few a week, and each would be thought about over a period of days or weeks. We might call these “everyweek” decisions (as opposed to everyday decisions).


Examples might be:

  • Who should we hire for this job? (Should we fire Jones?)
  • Should we revamp the current website, commission a new one – or what?
  • Should we pay for a booth at the XX tradeshow?
  • What legal structure should our US subsidiary have?

Everyweek decision problems tend to have the following properties:

  • They involve considering a number of options, and “weighing up” the considerations bearing on these options.
  • The weighing-up is done intuitively, i.e., without using any strict rule or calculation.
  • Each option has various pros and cons.
  • The pros and cons are heterogeneous, i.e., very different in nature.
  • The pros and cons are generally qualitative; it is impracticable or even nonsensical to put numbers on them.
  • The pros and cons may need to be backed up by evidence.
  • Indeed, the pros and cons are often disputable; we may need to consider the arguments for or against them, and the debate may get quite complex.
  • Generally not all the options, or pros and cons, are immediately apparent; it will take effort to “uncover” or generate them.
  • Everyweek decision problems are made under time pressure.
  • The time pressure rules out doing lots of research or investigation; we have to “make do” what what knowledge or beliefs we have already, or can access quickly and easily
  • Everyweek decisions are collaborative, i.e., there are multiple people involved in thinking through the decision.
  • These people may be remote, i.e. far away and in a different time zone
  • The decision maker is “accountable” for the decision.
  • In particular, the decision maker may have to justify the decision, i.e., explain the thinking behind it

It seems that the great majority of the non-trivial decisions the typical business decision maker has to make are “everyweek” in scale and nature.

I’m hoping, in subsequent posts, to discuss a number of issues related to everyweek decisions, such as:

  1. How everyweek decisions are generally made;
  2. What problems or “pain points” there are in everyweek decision making;
  3. The lack of good tools to support everyweek decision making
  4. The direction we (Austhink) are taking in addressing these issues.

Read Full Post »

 The Telegraph is carrying a piece, At the end of the day, you’ve given 110 per cent, which mocks clichés – and implicitly, those who use them.  The piece contains ten or so winning entries in a competition to cram the most cliches into a short text.

Interestingly, the word “cliché” doesn’t appear in the piece – they use “infuriating phrase” instead.  Maybe there’s a subtle difference there.

It is mildly amusing, though all the cleverness and jollity soon becomes a bit tiresome.

Not so long ago, I would have read the piece with the attitude that one as reader is presumed to have – a kind of smug superiority.  Of course *I* wouldn’t use these clichés – only the dull, the vulgar, the crass, the stupid would rely on such banal and overworked turns of phrase.

However I feel a little different now.  Having been doing far more “business communication” – writing, and especially conversing – than I ever used to do, I find myself relying on clichés more than ever before.  Is this because my brain is atrophying the longer I spend away from the intellectual realms of philosophy and cognitive science?  Perhaps.

But I think there might be something else at work.  Communication is only in part a matter of sending information, contained in the meaning of one’s words, to another person.  It is also about establishing a kind of rapport – conversing with them rather than talking to them.  In that “conversing with”, clichés are very useful.  They are standard moves from a common repertoire, allowing conversants to synchronize their thoughts and attitudes.  Sure, instead of saying “we’ll be giving it 110%” you could say something like “we’ll be working like untenured academics” but the very originality of such a phrase is likely to throw some sand in the conversational gears.

There is a useful analogy with that universal business cliché, the standard handshake.  Such a dull way of greeting somebody!  Why not, instead, try shaking their hand side-to-side, or with one’s fingers clenched, or with a wet hand, or… or hold their arm, stroke their hair, touch their nose… Try any of these more imaginative alternatives, and you’ll instantly create the perception that you are at the very least a bit odd.  You’ll seriously impair your chances of a successful business relationship. People want to know that they’ll be able to “play the business game” with you, by standard rules, not your creative and unpredictable rules. Shaking hands in a more or less normal way is just an opening signal that you’re interested to see the game go well.

So give clichés a break.  There’s something to be said for them.

Read Full Post »