One of the stories that’s making the rounds right now concerns Adam Orth, a (former) Creative Director at Microsoft who caused a ruckus by cramming his foot in his Twittermouth. He did so while weighing in on a potential “always on, always connected to the Internet” nature of Microsoft’s next Xbox console. The gist of his high crimes is that he supposedly looked down on those complaining about such a console feature, downplaying the significance of places where Internet connectivity is reliable and/or affordable.
Here’s the widely circulated screen caps of the conversation:
Setting aside the actual rumor, what I find interesting about this situation is a certain cognitive bias that seems to be on display in these comments, as well as in many of the surrounding debates I’ve seen on the ‘net. Orthy’s estimation of the number of people who have stable Internet connections and who should thus “#dealwithit” is probably influenced by the availability heuristic. Or rather, the UNavailability heuristic.
The availability heuristic is an old mental foible that I’ve written about before, and the short version is that the easier it is to remember of examples of something, the more prevalent, frequent, or large we think it is. But interestingly, the opposite is also true: will tend to think something is less prevalent or less frequent when it’s harder to remember examples of it. Various things make an event or a condition easier to remember, but they often include experiencing it ourselves, having seen it recently, or knowing someone who has.
My favorite example of this (un)availability heuristic is a hack created by University of California Los Angeles professor Craig Fox (2006) to boost his student class evaluations. Before completing the evaluation forms, Fox asked half the students to suggest 2 ways to improve the class (an easy task), then asked the other half to suggest 10 ways to improve it (a much harder task). Those who go quickly got stumped on the road to coming up with 10 improvements gave Fox higher course evaluation ratings than the others. Why? Because they misinterpreted the difficulty of recalling so many flaws as evidence that there were few flaws at all.
And so it is when most of us try to estimate how many potential Xbox owners have reliable, unmetered internet connection. Live in a city like San Francisco where you and all your friends have great connectivity, you’re more likely to underestimate how big a problem an always-connected device would be because it’s hard to think of times when you’ve had trouble getting online. On the flipside, if you have a sister who lives in a rural area with only spotty DSL or a friend residing in a country where they pay for Internet access by the minute, you’ll probably overestimate it. For stuff like this, don’t trust your gut. It doesn’t think in terms of statistics. It thinks in terms of stories and experiences.
Oh, also, don’t post to Twitter, Facebook, or any other public place about your employer or its products. That’s a bad idea, too.
REFERENCES
Fox, C. (2006). The availability heuristic in the classroom: How soliciting more criticism can boost your course ratings. Judgment and Decision Making, 1(1), 86-90.
Incidentally, the person he’s having a conversation with: Manveer Heir, is (or was) as esteemed game developer. He’s given many talks on game design and has been involved with many AAA games.
Agree completely with the thrust of the article — people constantly reference their personal experience or anecdotal evidence, in all sorts of arguments, as if it has any bearing on statistical data. A common one I see is “This one guy had his work pirated, but it led to a ton of exposure and he actually ended up making a lot of money on it, so piracy is actually GOOD for independent creators.” — an argument that’s decimated as soon as you introduce any actual data to the discussion.
Nitpick: second to last paragraph – wouldn’t you be likely to *underestimate* “how big a problem an always-connected device would be” if you live in SF, and *overestimate* it if you’re thinking of places where internet is spotty? The current text indicates the reverse.
Whoops, good catch! Fixed.
Relevant academic paper: “How Many People Do You Know in Prison?” by Zheng et al. ( http://www.stat.columbia.edu/~gelman/research/published/overdisp_final.pdf )
Thanks for the link. Seems like a different phenomenon, but still interesting.