On crowdsourcing and why people contribute content online.

June 4, 2007

Wetpaint is a website for user-created wikis. You can search the site on a certain subject and if you find an existing wiki on that subject you can join it. Otherwise you can create your own. The CEO of Wetpaint, Ben Elowitz, was interviewed at Assignment Zero where he talked about Wetpaint as well as crowdsourcing in general. One of the more interesting parts of the interview dealt with the concept of “the wisdom of crowds”:

Q: Do you really think there’s wisdom in crowds? If so, what’s the clearest example you know of? What projects in particular are you impressed by?

A: I’m a big fan of “The Wisdom of Crowds.” I think some of the best examples of the Wisdom of Crowds in action were provided by James Surowiecki in his book. Having been a city dweller for much of my life, I was particularly fascinated by his example of how walking down a busy sidewalk was a picture perfect example of how we navigate together to help each other get where we’re going as quickly as possible. Other examples I find fascinating – I think you need look only to the capital markets to find the best example of where collective knowledge quickly and constantly comes together to create markets for stocks and bonds. From a Wetpaint perspective, my favorite crowdsourcing project is the CSI wiki where fans of the show have built the definitive guide to the series. The amount of creativity and detail that has been quickly assembled could not have happened were it not for the collective efforts of the show’s fan base.

I haven’t read Surowiecki’s book yet but I would like to. At another point in the interview, Ben Elowitz talks about why people contribute for free:

Q: Is there money to be made with crowdsourcing? If so, why will some people work for free so that others can profit?

A:No question – there is money to be made with crowdsourcing. Crowdsourcing is a basic human attribute – by nature we like to work together to produce things of value. This type of behavior has happened offline for millenniums and now it’s taking shape online. The business opportunity comes when entrepreneurs make the human process of collaboration quicker, easier and more rewarding than before.

In terms of why people work for free so others can profit, we have to remember that the value equation on crowdsourcing is not only measured in dollars and cents. As long as the participant feels like they are getting compensation commensurate with their input, the incentive to continue participating is rather high.

I think he missed the chance to elaborate on this. I agree that the value equation isn’t only measured in money. The research project I am currently running with my colleagues at Baruch College looks at exactly that question. If there are no external motivating factors (such as money) then there must be internal or intrinsic motivating factors. These can be reputation, efficacy (the ability to make an impact), attachment to the online community, and so on.

We are in the process of analyzing data we collected from users of participatory websites that should give us an idea on exactly what those motivating factors are. Stay tuned.


The power of the online mob wins at Digg.

May 2, 2007

A user revolt has just happened at Digg (you can read about it at Techcrunch, BoingBoingthe NY Times, and TextYT). Digg, for those who don’t know, is a social content site where users can post links to articles and websites and other users can comment on them and vote them up (digg them) or down (bury them). Stories with most diggs float up to the top of the list and the front page of Digg.

After someone posted the decryption key for HD DVDs, Digg got a request threat from the company that owns the rights to HD DVD to take the article down. Digg caved in and did that.

Well, the Digg community was furious. Soon, the entire front page of Digg was full of stories and article posted by users with the same decryption key. Digg kept taking them down and suspending users. But eventually, the mob won. Digg co-founder Kevin Rose posted the following:

Today was an insane day. And as the founder of Digg, I just wanted to post my thoughts…

In building and shaping the site I’ve always tried to stay as hands on as possible. We’ve always given site moderation (digging/burying) power to the community. Occasionally we step in to remove stories that violate our terms of use (eg. linking to pornography, illegal downloads, racial hate sites, etc.). So today was a difficult day for us. We had to decide whether to remove stories containing a single code based on a cease and desist declaration. We had to make a call, and in our desire to avoid a scenario where Digg would be interrupted or shut down, we decided to comply and remove the stories with the code.

But now, after seeing hundreds of stories and reading thousands of comments, you’ve made it clear. You’d rather see Digg go down fighting than bow down to a bigger company. We hear you, and effective immediately we won’t delete stories or comments containing the code and will deal with whatever the consequences might be.

If we lose, then what the hell, at least we died trying.

This is fascinating stuff. This isn’t just collective intelligence at work. This is collective activism at work and it seems to have worked. Of course, some might wonder what would happen when the mob is wrong or has the wrong intentions. Some might say that this was the case here, where intellectual property rights were being challenged.

I’m not going to try to judge this is good or bad. I am just going to point out how powerful this can be.

In a lot of science fiction writing, a classic nightmare scenario is that where networked artificial intelligence built by humans becomes so powerful that it takes over the world. Maybe the scifi writers got it wrong. Maybe what we should really be worried about is networked human intelligence (or lack thereof), or the power of the online mob. Unless of course, you are like me, and you believe that humanity as a whole can be good and smart and can have the right intentions, and that it’s only a minority of people who are evil and it’s when they get a lot of power that all hell breaks loose.


Radical transparency and the death of spin.

April 14, 2007

I often talk to my students about the power of today’s networked information environment in shaping reputation. Keeping secrets has become extremely difficult, for businesses, politicians, and even 15-year-olds. All you need is one person (often yourself) to put the information in digital format and post it somewhere where others can read it. The network effect can be frightening. In a matter of minutes, it can become world-wide news. And even if you take the information down, it will always exist, cached on servers everywhere just waiting to be found. The result can be tremendous pressure on companies and politicians to clean up their act and be accountable. And I’ve always told my students, some of whom are budding CEOs and CIOs themselves, that the way to control your reputation is not by controlling information that is made public, but by making more information public yourself.

Well, enter Wiredmagazine, which has done a fantastic job in reporting on this idea and giving it a cool name: radical transparency. It has several articles on the subject but this one in particular tells the story beautifully. In fact, the whole article was written in the author’s blog inviting reader feedback, some of which is featured in the printed article. As the article’s author, Clive Thompson, says:

The Internet has inverted the social physics of information. Companies used to assume that details about their internal workings were valuable precisely because they were secret. If you were cagey about your plans, you had the upper hand; if you kept your next big idea to yourself, people couldn’t steal it. Now, billion- dollar ideas come to CEOs who give them away; corporations that publicize their failings grow stronger. Power comes not from your Rolodex but from how many bloggers link to you – and everyone trembles before search engine rankings.

Some of you may be thinking: What about industrial secrets? The secret recipe of Coke? As Clive points out:

[Some of my blog readers] enjoyed ripping apart my new theories. Several pointed out that secrecy can be necessary – CEOs are often required by law to keep mum, and many creative endeavors benefit from being closed: Steve Jobs came up with a terrific iPhone precisely because he acts like an artist and doesn’t consult everyone. In fact, secrecy is sometimes part of the fun. Who wants to know how this season of 24 is going to end? It’s not secrets that are dying, as one reader named gjudd noted, but lies.

Secrets can be useful tools for competitive advantage. Even though there are many cases when even those secrets are better left in the open. Just look at the success of all the open-source software out there.

So it’s not secrets that are dying. It’s spin control that’s dying. This spells trouble for PR firms.

What’s making me even more excited is the impact this is already starting to have on politics, where transparency is a dirty word and spin control is god. Remember George Allen’s “macaca” comment that got posted on YouTube and cost him a Senate seat?

Politics is going to be fun again.


Learn about Web 2.0, HTML, XML, and more with this cool video

April 3, 2007

I just found this today. I will definitely be using it next time I am teaching me e-business class, right before (or after) I discuss HTML, XML, and the Web 2.0 environment.


Internet radio grows up.

March 29, 2007

The NY Times today have a nice short article that clearly explains some of the more popular Internet radio services out there. These are web-based music streaming services that allow you to create a personalized radio station that chooses songs for you based on certain criteria. Those criteria vary depending on the service. Two of those services show the difference between using experts and the collective intelligence of the crowd to provide better recommendations:

Pandora uses “musicologists” who classify hundreds of thousands of songs and artists based on their musical “genetic” makeup. For example, the “genome” of Madonna’s “Let it be” is comprised of “electronica influences, danceable grooves, a subtle use of vocal harmony, use of string ensemble and a vocal-centric aesthetic.” As songs get played for you on Pandora, you can vote them up or down and thus teach your station to play better music for you.

Last.fm uses know experts but instead employs mathematical algorithms to analyze the playlists and listening habits of its members to create recommendations. In addition, Last.fm promotes itself as a social networking site, à la MySpace. Pandora does this a little bit by allowing people to share their personal stations, but not to the extent that Last.fm does.

 Which one works better? I don’t know. I’m a loyal Pandora listener, especially for an instrumental jazz station that I have created and honed over the last few months. I have never used Last.fm. Whenever I go to its site, I get overwhelmed by the social networking bells and whistles. All I want is a simple, customizable radio station and that’s what Pandora gives me.

Maybe in the end that’s what matters the most. The quality of the recommendations by the two services is probably comparable. What’s different is their functionality and look and feel. One is a simple, no-frills, customizable radio station. The other is a full blown social network that’s centered around music and musical tastes.


Wikipedia vs. Citizendium: Let the battle begin!

March 27, 2007

Citizendium, an alternative open online encyclopedia, has gone live. It was created by Larry Sanger who happens to also be a Wikipedia cofounder. The two encyclopedias share a lot of similarities. They are both free to read and open for anyone to edit. They are also both ad free. However, Citizendium requires contributors to use their real names while Wikipedia doesn’t. And Citizendium will go to experts for verification of its articles, while Wikipedia will not.

 I won’t debate the latter difference. I am always a little suspect of “experts.” Maybe it’s because I work in academia and I see how many times “expertise” is just misguided stubborness.

But the real-name-use rule is interesting. There is a plethora of research on the effects of anonymity in computer mediated groups. The results are not always consistent but some trends have emerged. For example, anonymity in such groups promotes flaming, the act of posting hostile and insulting messages that are often long and rambling. Flaming can be done by group members that would not otherwise engage in it if they weren’t anonymous. Anonymity can also increase group polarization. Both findings would favor Citizendium.

However, anonymity also increases the number of original ideas (though not necessarily the quality of the best idea), partly because it allows everyone to contribute, even if they are disagreeing with the majority of the group. This may favor Wikipedia, especially since it sometimes allows for multiple articles on a topic, if it’s one that is controversial or is subject to interpretation.

By requiring real names, Citizendium promotes accountability and responsibility which is a good thing. But it also may silence many contributors who may have important knowledge but are reluctant to provide it under their real name. Citizendium does allow one to apply for a pseudonym, but only in special cases.

Who will win the encyclopedia battle? I don’t know. Only time will tell. My bet is on Wikipedia. It has the momentum, the experience, and the brand name to attract enough contributions to sustain it. And despite the room for vandalism and malicious content, it allows everyone to add information without fear.

But Citizendium has definitely made this battle more interesting.


Investment strategies using collective intelligence

March 26, 2007

Fin.uti.ae is an experimental site that is trying to create funds that use the collective knowledge of investors to form investment strategies instead of using a few expert fund managers. Right now, it’s trying to prove the concept works, so its interface is really simple. They are giving away $100 to each of the five individuals who contribute the most to the site’s performance so go check it out.

It’s an interesting experiment. If investment knowledge truly is distributed throughout the market, theoretically you could capture that knowledge by aggregating all the individual bits of knowledge of each individual investor. What you need is a large enough community and a liquid enough knowledge repository. A large pool of individual contributors is also necessary to wash out the ones who provide wrong information (deliberately or not). According to one of the founders of the site, they are currently performing a modest 3% over the DOW with a continually improving alpha. But they’ve only been running for a short time and their pool of individual knowledge contributors is still relatively small.

The big question for me is whether in the context of financial markets, all the collective intelligence of the individual “lay” investors cannot even come close to the insider knowledge that the very few priviliged investors have (and who persumably will not share it). Fin.uti.ae may be able to give a clearer answer to that question.


Assignment Zero: Journalism through crowdsourcing.

March 19, 2007

Assignment Zero is a fascinating new experiment in journalism. It uses crowdsourcing (allowing large groups of people to collectively create or build a product though small individual contributions) to report a news story. This is the long tail of information at play.

Up till recently, the information we consumed came from few centralized sources that we considered the experts. They included the news organizations, the newspapers and the magazines. The billions of viewers and readers were simple passive consumers of the information created by the few. Today, we are moving away from the centralized information model and closer to a distributed “long tail” model of information creation. We consume information that may be created by multitudes of non-experts, simply because they happen to be closer to where the information resides. We can all be both consumers and active producers of information.

Assignment Zero is using online collaboration tools such as wikis to allow anyone out there to contribute to a massively produced news story. Besides, who better to provide information on a story than the people who are directly involved in it? This could include the victims of a big storm, the participants in a demonstration, the employees of a company under investigation, or anyone else who is part of the story itself.

There have already been attempts to do that. Yahoo is letting users contribute photos and videos of events online to report what happened. They call it “You Witness News.” A NY Times article on Assignment Zero describes other efforts to produce crowdsourced journalism:

At newspapers like The Asbury Park Press in New Jersey, Florida Today in Brevard County and The News-Press in Fort Meyers, Fla., citizens can dial into databases and public records, or contribute their own experiences to provide grist for reported efforts.

A project at The News-Press on the high cost of sewer and water lines (available in the newspaper’s paid archives at www.news-press.com/apps/pbcs.dll/frontpage) included volunteer engineers going over blueprints in their spare time and an insider who disclosed critical documents.

Is there still a need for journalists and editors? Of course. Journalists can still provide the expert analysis or the focused writing that crowds cannot. And editors are necessary for all types of news stories, whether produced by a journalist or thousands of contributors, in order to keep things smooth, correct, and verified.

But where we get our information is becoming less and less centralized. It’s also (hopefully) becoming less biased and more authentic. 


Web 2.1: Business 2.0’s Next Net 25

March 15, 2007

Business 2.0 has identified their first Next Net 25: 25 Web 2.0 companies that they think will make it big in the near future (months, not years). I call them Web 2.1. They are cool, smart, useful, and in many cases, profitable (or at least able to become so). Check them out here.

Some of my favorites: Joost, Turn, Vitrue, Fon, SuccessFactors


Wikis: They’re not just for encyclopedias anymore.

March 14, 2007

Business Week has a great special section on Wikis this week. For the uninitiated, Wikis are basically online collaboration tools that allow multiple users to collaboratively create and edit documents. It’s how Wikipedia works. But Wikis have made huge inroads within companies. The possibilities are endless. For example, a software development company can allow its developers to develop and test code through a wiki. It can also allow its team of remotely based technical writers to write the software documentation. One article in BW’s special section talks about many other examples of large corporations implementing Wikis for internal purposes.

But why stop with employees? Why not let the customers do the work? This is nothing new. Companies like Microsoft have successfully used forums and bulletin boards to allow their customers to help each other with all technical issues. Personally, I have always had better luck getting help from such online forums than any company’s technical support team. So, a Wiki can allow the customers to basically build and edit a knowledge base, one that isn’t organized in the form of questions and answers in long complicated threads, but in the form of self-contained articles. eBay has been trying this with eBay Wiki. The beauty of Wikis is that all the information created is never static. It’s constantly alive and changing to (hopefully) reflect the changes in the world it describes. To borrow a metaphor from biology, the information becomes the organism and the users who create and edit it become its adaptive cells.

Of course there are those who say it’s not going to work. That there’s too much room for vandalism and unscrupuplous misuse. But the same can be said for email or even the telephone. As a technology becomes well integrated with the social norms, regulations, and culture of an organization, it becomes less and less misused. Maybe it’s the fact that employees realize that everyone is collectively better off if the technology is used appropriately. Maybe it’s because organizations develop clearer guidelines and sanctions for misuse. Whatever the reason(s), the result is the same. So, let Wikis grow and use them as much as possible. Soon, there will come a time we won’t remember how we worked without them.