Recent events may provide cause for setting the differences, or lack thereof, between Wikipedia and Brittanica as well as Wikipedia and Open Source. A peer review of 42 articles in Wikipedia and Brittannica shows equal serious errors and a minor difference in minor errors:
Yet Nature's investigation suggests that Britannica's advantage may not be great, at least when it comes to science entries. In the study, entries were chosen from the websites of Wikipedia and Encyclopaedia Britannica on a broad range of scientific disciplines and sent to a relevant expert for peer review. Each reviewer examined the entry on a single subject from the two encyclopaedias; they were not told which article came from which encyclopaedia. A total of 42 usable reviews were returned out of 50 sent out, and were then examined by Nature's news team.
Only eight serious errors, such as misinterpretations of important concepts, were detected in the pairs of articles reviewed, four from each encyclopaedia. But reviewers also found many factual errors, omissions or misleading statements: 162 and 123 in Wikipedia and Britannica, respectively.
A key difference is the speed of update for Wikipedia not only helps for correction, but means it is some thinig different -- something that doesn't presently claim authoritativeness:
But Michael Twidale, an information scientist at the University of Illinois at Urbana-Champaign, says that Wikipedia's strongest suit is the speed at which it can updated, a factor not considered by Nature's reviewers.
"People will find it shocking to see how many errors there are in Britannica," Twidale adds. "Print encyclopaedias are often set up as the gold standards of information quality against which the failings of faster or cheaper resources can be compared. These findings remind us that we have an 18-carat standard, not a 24-carat one."
The dynamic nature of Wikipedia does present challenges for participants and results in a different product. There are plans to create a stable branch (basically tagging a revision as reviewed, although review processes are still experimental, while letting users continue to edit new revisions), which would result in a truely comparable product and support print versions.
My hope is the Nature article provides a proof point to at least simplify the neverending debate of quality in Wikipedia. But before taking a break (the interesting stuff isn't metaphysical debates, but understanding the social practices that make Wikipedia work) from the disconnected-meta excersize of blogging about Wikipedia (albeit a hair above the disconnect of criticizing an Article without submitting changes), let me take on one last issue.
The model of a stable branch is an approach borrowed from open source software development. Recently, Cnet posted a News Analysis in search of a problem that came across as an opinion piece by the author. The premise was that Wikipedia is often labeled as open source and the label was factually innacurate. Open source practices have project managers as gatekeepers for quality, while Wikipedia offloads this responsibility to an open group. A bunch of open source experts were rounded up to explain that, some speculated in agreement with the author that this may be the cause of quality issues in Wikipedia. While the piece is an interesting read, as the CEO of a commercial open source wiki company (more critically, I am a student of the topic), I can point out some significant flaws:
- Wikis do not require centralized project management because the work product is different. Coding software is an act of vertical information assembly, marked by the dependencies of contributions. By contrast, Wikipedia is horizontal information assembly without dependencies. It should be celebrated that the lack of central project management is a cause for the growth and success of the project.
- Having a central project manager for Wikipedia would also be complicated because of the lack of objective measures. From the article: "With software, you get the benefit of there being some objective measure of a change," said Brian Behlendorf, co-founder of the Apache Software Project. "It's not a question of aesthetics. Does this fix the bug people have noticed? Does it speed up performance?"
- Wikis and Open Source share the critical attribute of The Right to Fork
- Wikis and Open Source enable anyone to contribute, but with implicit reputation giving greater weight to different contributions. Even in Wikipedia, there is authority of participants.
- Wikis and Open Source share a common model of Commons-based Peer Production.
The definition of what is Open Source is often debated and is more likely to be settled on a wiki page than here or in the News Analysis. Here is an unpublished chapter in wiki format on Open Source Dynamics of Wiki Practices if you would like to learn, and possibly contribute, more.
