Hunting for wisdom of crowds applications

The wisdom of crowds is all the rage lately. Or so it would seem, since it's talked about so much. But as it turns out, many self-proclaimed wisdom of crowds applications are actually leveraging crowdsourcing or collective intelligence, which are different organisms (see my previous post for an explanation of the difference between the wisdom of crowds and crowdsourcing, and Henry Jenkin’s post for the difference between crowd wisdom and collective intelligence).

(img src: Oracle Blogs)

All the buzz about crowd wisdom implies that it’s sitting atop the Peak of Inflated Expectations on Gartner’s hype cycle (see above), a cycle representing five phases that emerging technologies go through. (Note: I’m taking liberty to consider the wisdom of crowds a technology, in that it is something that can be used to achieve a desirable goal, though this might be a discussion to have here...) The Peak of Inflated Expectations is a phase in which a frenzy of publicity generates over-enthusiasm and unrealistic expectations about a technology, resulting in some successful applications but more failed ones. We could wish that technologies just ascend the Slope of Enlightenment already, but the Peak of Inflated Expectations is actually a great place to experiment with possible applications of a technology, and the subsequent Trough of Disillusionment is a great place to sober up about the applications for which a technology is actually useful. (Besides, these phases have awesome names.) So, with all due respect to the Peak, it’s only via experimentation that we discover what works and what doesn’t. Hence the Peak’s importance as part of the cycle emerging technologies go through. And hence widespread experimentation with the wisdom of crowds as it sits atop the Peak.

Again, or so it would seem. Because, when I looked around to see what other wisdom of crowds applications were around, I didn’t find to many. Which gives me the impression that we aren’t experimenting so much yet – we think we’re experimenting, but we’re oftentimes conflating crowd wisdom with other organisms. A case in point is FinancialPuzzle's comment on Lazy Man and the Money’s post about Piqqem, "this whole crowdsourcing thing is trying to work its way into every industry." So, I'd actually venture to say that the wisdom of crowds as a technology is not at the Peak, but somewhere closer to the Technology Trigger. (Crowdsourcing and collective intelligence, on the other hand, are other stories.) That said, there's still some experimentation going on, and I still wanna take a look.

In James Surowiecki's conception of it, the wisdom of crowds is only applicable to quantifiable, objective data. Which means Wikipedia, Digg, and even most of the examples in We Are Smarter Than Me, a book ostensibly dedicated to the wisdom of crowds, don’t make the grade. Yahoo!, CBS Sportsline, and other major newsie sites already display aggregate user results, but they don't explicitly apply crowd wisdom to coming up with those results, which is what I'd consider a wisdom of crowds application. And according to Wikipedians, crowd wisdom applications fall into three categories: prediction markets, Delphi methods, and traditional opinion polls. But I found mostly predictions markets. Here's a sampling:

• Betfair Betfair applies the wisdom of crowds to predicting future events from Horse Racing to Sports to traditional gambling. Falling under the category of predictions markets, Betfair is the is the world's largest online betting site, with around $28 billion traded in 2007. Users make 'Back' bets (normal bets on a selection to win) and 'Lay' bets (bets on the opposite side of the Back, against the selection), enabling Betfair to aggregate bets, and gauge crowd sentiment. Note that Betfair is similar to other predictions markets like Predictify, which is based on deterministic, verifiable questions concerning future events, but differs from those like Long Bets that pit only two users against each other or don’t aggregate the opinions of many.
• Hollywood Stock Exchange The Hollywood Stock Exchange, or HSX, is a predictions market applying the wisdom of crowds to predict Hollywood-related events. Players use play money to buy and sell "shares" of actors, directors, upcoming films, and film-related options. According to Wikipedians, "because trading directly affects the prices of the securities — purchasing enough shares of a stock causes its price to rise, and selling causes its price to fall — and because the ultimate value of a moviestock is based on the film's box office, stock prices act as box office predictions. For example, if a particular moviestock trades at 'H$40.00', the market is predicting that the movie will gross US$40 million at the box office in the first four weekends of wide release." In 2007, for example, HSX players correctly predicted 32 of the 39 major-category Oscar nominees and 7 out of 8 top-category winners.
• NewsFutures Another predictions market oriented towards news and without real money is NewsFutures, which applies the wisdom of crowds to predict future news events. Users buy and sell contracts that will pay a given amount of play money if a particular event happens in the future, and zero if it doesn't, enabling News Futures to generate consensus probabilities for news events. As a company, it is one of several helping large corporations set up private prediction markets to predict project completion dates, sales, or the market potential for new ideas (please comment on other companies if you know of them).
• Click! A Crowd-Curated Exhibition As a photography exhibit at the Brooklyn Museum that leverages crowd wisdom to evaluate art and, in turn, curate the exhibit, Click! could fall into the category of a traditional opinion poll. "Taking its inspiration from the critically acclaimed book The Wisdom of Crowds, in which New Yorker business and financial columnist James Surowiecki asserts that a diverse crowd is often wiser at making decisions than expert individuals, Click! explores whether Surowiecki’s premise can be applied to the visual arts—is a diverse crowd just as “wise” at evaluating art as the trained experts?" Ok, fine, art evaluations are not necessarily quantifiable, objective data, but I’m letting that slide for purposes of including an example other than a predictions market.
• CrowdChess Thanks to Stan Oleynik, the founder of CrowdChess, for submitting this! CrowdChess is precisely what it sounds like: a web app that applies the wisdom of crowds to playing chess. Members of the same team can introduce new moves, vote for and comment on already proposed ones, see real time game stats, and chat with each other. Again, chess moves are not necessarily quantifiable, objective data, and the ability to chat with fellow players compromises the requirement of independence for crowd wisdom to emerge, rendering CrowdChess more of a collective intelligence app than a wisdom of crowds one. But it was too awesome to leave off this list, so voilà.

Which is to say: please gimme more examples. Does anyone know of a case applying the wisdom of crowds to scientific data? What ideas do you have for crowd wisdom applications? To clarify, asking you for ideas is applying crowdsourcing to wisdom of crowd apps; if you submitted your proposed applications wiki-style, enabling the crowd to discuss and edit, it would apply collective intelligence to wisdom of crowd apps; and if you voted on whether the apps would succeed or fail, well, that would apply the wisdom of crowds to wisdom of crowds apps – letting the wise crowd predict which crowd wisdom apps would succeed vs. fail. Adorable. But we'll need a hearty selection of crowd wisdom apps to start with, so comment on ones you know of, or invent your own!