Wiki wiki wild wild west
The fifth in a series on the ideas we received
“Wiki” got the largest label on our idea submission form, which has now simply changed to be a “tag cloud” of the idea categories submitted. For those out of the loop a tag cloud typically gives the largest text to the most popular terms in the “cloud”.
A “wiki” is a website that allows anyone (with permission) to edit any page and to add new pages. Wikipedia
Submissions included wikis for breaking news, education and encyclopaedias of economics and democracy. They suggest turning some of The Economist books such as the Style Guide into a wiki. Steven previously mentioned this in his post “Stylin’”
The site would be part encyclopedia and part “handbook of economics” Will Ambrosin, idea submitter and all round good guy.
He goes on to mention
The site would be a wiki, i.e. open authoring, but some of the more recent ideas about quality control in that environment would be incorporated.
Kevin Chuang wants us to build.
The first and ONLY citable wiki resource for breaking news and current business developments.
In general the consensus, from our idea submitters, is that the conventional wiki needs to be improved with some form of moderation. Some of the people behind Wikipedia have left the Wikipedia foundation to build Citizendium, which coincidentally opened on March 25th. Citizendium describes itself as a “project, started by a founder of Wikipedia, [that] aims to improve on that model by adding “gentle expert oversight” and requiring contributors to use their real names.”
In addition, it’s my opinion that Wikipedia is hard to use. The “anyone can edit” phrase that appears on the front page is a bit naff. Why? Because you have to learn a non-trivial markup language to be able to correctly format your entries. They didn’t even use HTML, which has to be the most widespread mark up language available that people have a basic understanding of. Rich text editing is clearly something wikis should embrace. I think the idea of a editable encyclopaedia is a great one and it should be easy to add to.
Moderation could go a myriad of ways. I come from a software development background and could suggest a fairly simple model for moderation. In software projects, and this is simplified, you have a development environment, a test environment and then a live environment. To apply that to wikis you could firstly make changes to the development environment, then have someone moderate them slightly later on the test environment and finally promote them to the live environment when moderators are happy. This doesn’t address fact-checking but it could address the issue of children seeing objectionable content. A simple point-in-time snapshot approach could work with moderators only checking pages that have changed since the snapshot. Books can be written like this and, in fact, are. The O’Reilly “Version Control with Subversion” book is written online in this fashion. This is not exactly a wiki but it shows you can release moderated versions of things on the web. Please tell me how ridiculously flawed this idea is.
An example of an early wiki
Some idea contributors want revenue to be shared with the creators of content. Personally I find that intriguing. Challenges could lie here in deciding what is a payable contribution. Should someone get paid if the facts are wrong, how about if you fix spelling or grammar errors? If an article gets rewritten based on the existing content how do you split the cash? Is it fair to equally compensate all contributors? If so, would you get people using the long tail and being serial grammar and spelling mistake changers to earn payments from multiple pages? If we launch a wiki and it does share revenue, if it makes any, how should it pay moderators?
Even if a moderator has not made changes to a page, they have contributed time by checking the page. Editors get paid at traditional publications, and if wikis start paying where do you draw the line? This is a critical question in truly open and editable wikis. Who and how to pay is a tough cookie. I can’t see it working without a trusted moderator figuring out the percentages. How would people who earn money from their content feel about the content changing and the money disappearing?
“Nobody has found the de facto business model for wikis, it’s kind of the Wild West.” Ramit Sethi, co-founder of PBwiki
The contributors at wikiHow don’t seem to mind that they don’t get paid. wikiHow serves ads alongside their content and I don’t see contributors asking for payment. Those seeking payment have probably already left, in fact just googling for comments I can find disgruntled people. Then again, take any product on the planet and I think you’ll find someone moaning about it on the web. Perhaps earning revenue will keep new people away from contributing to wikiHow; they’ll take their content and efforts to wikipedia and then probably donate to wikipedia to keep their content alive.
At a first glance this looks like biting your nose off to spite your face as you may part with cash to help wikipedia run but wouldn’t contribute to a commercial wiki. However I can see why people might think like it.
Recent real world wiki
I personally think that education, especially for children, is a good area to look at. You could really test the waters of moderation there as you just can’t risk children seeing objectionable content. Perhaps we could get teachers, and other clever people, to write up articles, follow curriculums and help educate children in new ways.
A wiki is essentially a collaborative space. These things have been around for a long time. Be it as cave drawings or graffiti. So online wikis are, relatively speaking, nothing new. Graffiti artists, or contributors, have even figured out a moderation system of their own. They are often aware of other artist’s credibility and would think twice before overwriting a respected artist’s content. They have a strong incentive to follow the moderation system because of real world repercussions.
Please tell me how you would handle payment and moderation, and whether I’ve missed any points on the problems for future wikis.
(In addition to those mentioned in this post, we would like to thank all those who have contributed suggestions related to wikis.)
March 30th, 2007 at 2:34 pm
Moderation is really hard for wikis. Let’s say Michelle comes and edits a page, which is then sent to Mrs. Kundel for approval. If John comes along and wants to edit the page–but Mrs. Kundel hasn’t approved the change yet–which page is he editing? Version A or version B?
It becomes a huge mess.
We simply decided to not have moderation and allow people to edit the pages openly because we assume that most people are good. And we have the ability to go back in time to an older version of a page. It turns out that this is sufficient for almost everybody.
The challenge isn’t the technology or moderation, it’s creating a wiki easy enough where people will add meaningful content in the first place.
Thanks,
-Ramit
PBwiki Co-founder
March 30th, 2007 at 5:19 pm
To me, the “anyone can edit” phrase means anyone can edit up to their own level of expertise. It doesn’t mean that anyone can edit like an expert (which would require knowledge of that “non-trivial markup language”).
For example, if someone sees a factual error in a statement about a date, a place, or an event, then she/he can simply edit the page and fix the facts in English (or whatever language) by replacing the incorrect text or adding text. No special markup language knowledge is needed here. But if one wants to add links, sections, etc. they do need to learn a little bit more.
On the topic of deciding payment. It is a hard problem. Once payment is involved, how do you know the “problems” in the system are not created by some “cheaters”? What is a fair amount to pay someone? Some edits may be an easy fix and some may take hours to fact check. My current thinking on this is it may be easier to make the payments as donations to charities.
As an aside, if Wikipedia had a token payment system to “reward” their contributors directly, I am certain a different crowd of people will be attracted to it. And I will be quite surprised if it can be even half as good as it currently is. Just my 2 cents and wild guesses.
April 5th, 2007 at 2:01 pm
I am less interested in the moderation thing because I would like wiki-like features for adding and deleting from articles before sharing them with friends. Many newspaper articles are written in telegraph style, and begin with stuff my intended recipients know. I would delete that to let them get to the heart of the matter, and then perhaps add a question. The recipient might then further edit that either for me to see or another friend. We would not be editing the primary copy (what most people see), but something shared between a clique.
April 15th, 2007 at 7:50 pm
“Moderation” and “payment” are a source of problems in many aspects of life, but I do not see they are perceived as hard problems in every situation. During their professional life, many people use to contribute to writing articles or documents for which they will be paid only in part or indirectly, while others are paid for writing things they could hardly be considered as authors. Organizational and social contexts are extremely important in deciding what is appropriate and what is not. The same might be said about moderating or editing.
I agree that wikis are hard to moderate and pay for. The real challenge may lie in deciding the right granularity of contributions and in developing sensible organizational contexts. Maybe the “article form” proved to be good as long as single authors mainly used to produce them. An article may prove too large as a unit of moderation and, particularly, of payment. Instead of evaluating large, overlapped contributions, smaller contributions could be evaluated more easily, and the best one might be chosen in whole.
As long as a wiki is used to accumulate encyclopaedic knowledge in democracy, economy, language or anything else, this is even more similar to what happens in real life. Collective works are the norm in research communities, so payment could similarly be linked to some sort of citation index or Google-like ranking for the article as a whole.
It seems there is not an established model or a even best practice to take inspiration from, but there seem to be many different strategies to try.
April 16th, 2007 at 11:54 pm
Moderation the trickiest part of Wikis but we need to realize that all reputable sources of information exist because they have established a reputation over time for producing quality reliable information. This includes academic journals, newspapers, and magazines.
Reputation is very important in news and for a company like Economist and implementing a subscription based reputation system for all Wiki contributors gives them an edge over the public Wikipedia. Anyone can read the wiki/article, but if you want to edit, you need to pay to get an account or register your (real-life) journalist/reporter status with Economist. Abusive accounts are easily terminated. Truly public and unverified information do not belong in any encyclopedia.
Moreover, this creates a system for payment rewards, a reporter with an account can be paid for his contributions once his news has been confirmed by other media. As any journalist, they put their neck on the line to report news and if they report news poorly, their story may be ignored.
However, I don’t see the obsession with payment. People need to realize a wiki is a database of evolving information. Once you become the main portal for certain information, everyone will use you and contributers contribute because they need a place to summarize ideas. Payment may be an incentive for journalists (who use their real name/reputation) to fill out unfinished wikis, but you need to pay people to contribute to wikipedia. An Economist wiki is no different.
The main difference is the news changes quickly (unlike encyclopedias) and the organization of wiki information may get out of hand quickly if moderation is not standardized to some extent (eg. confirmed facts, rumors, opinions, first hand experiences, etc). In any case, the results of an attempt to organize the news of the world will generate a fascinating source of information. In fact, I can imagine an Amazon-like system to promote cross-linking of related articles; “People who read this Wiki also read Wiki XXX.” Let the readers do the work of linking related Wiki’s for you.
April 22nd, 2007 at 1:59 am
This is a hard one. I’m for the Economist to open a wiki. But still I cannot see how the website can be easily managed. An alternative is another type of wiki. Instead of knowledge, this one will provide fun to its users. This can be done by opening your archive to readers and invited experts. Once in they will be asked to help you review and adjust all the news and opinions. Of course, once this happens lots of knowledge and opinion clashes will ensue. And who will not want to watch these clashes, as the invited experts are world-class.
Since this is a fun wiki, the website should be for profit. I still cannot think of the name, but it should be like a club. One that is also very exclusive and pricey. In this way instead of paying, you will be receiving. And who can argue this will not produce the best knowledge and opinions.
April 22nd, 2007 at 5:47 pm
Of all organizations, I’d expect The Economist to comprehend exactly why unattributed editorially-backed content is superior to a bunch of attributed bylines. The wrongheaded Citizendium approach, or worse, the Panopticon that some legislatures (notably in South Korea and the US) wish to impose on the web, assumes that every idea or opinion must be attached to a living person. But that’s nonsense. What about dead people? Are we to open people to exposure and possibly harassment for quoting a philosopher or politician of the past? Most of the reason Wikipedia works, to the degree it does, is because it puts all its eggs in one basket - the editorial vetting and arbitration process - and watches it closely. Likewise, by avoiding bylines, The Economist forces all its authors to adhere to a common vetting and content arbitration process. Probably a less democratic and open one, but an effective one, as evidenced by The Economist’s reputation for accurate and balanced views. It’s time to simply kill off the belief that ideas must be attached to specific persons. IBM’s research at TJ Watson Labs on anonymous contribution in work groups demonstrated clearly that statements made egolessly were freer and more creative than those made with mandatory attribution. The “suggestion box” approach used at many manufacturing companies, the secret ballot, the anonymous police tip line, are precedents that show that anonymity is sometimes absolutely essential to getting clear and accurate feedback.
Total quality management begins by focusing on the process, not the product. No high quality product can emerge from a series of reactive steps trying to “fix the product” itself. Instead, it emerges from a process that undergoes continuous improvement and takes all product flaws as evidence in a diagnostic discipline, providing empirical data. From this perspective, products are simply experiments that seek to disprove the thesis that the process is the best it can be. Every flawed output is seized upon by the curious who track down the process flaws that cause it. Not the people/individuals that made an error, but the pressures, signals, inputs, rules, and so on that are deemed to be the cause.
For an example of how well this works, look at the air transport industry. Every crash is in fact treated as a forensic opportunity. Airplane manufacturers and airlines gain some limits on liabilities in return for cooperating with the process undertaken by, in the US, the FAA. Inclusion of ‘black box’ devices to track every action and motion of an airplane is a mandatory feature for an airplane to be sold at all. If it works in the air, producing the least dangerous mode of transport there is, there’s no reason it shouldn’t work with those products that are far easier to review, far more transparent, with far fewer consequences for error and many more opportunities for obvious improvement. Such as economic data.
“Please tell me how you would handle payment and moderation, and whether I’ve missed any points on the problems for future wikis.” OK. First, you’ve failed to deal with the plain reality that contributors vary drastically in their writing and editorial and dispute resolving ability, and that no prior restraint or credentialling will reliably sort out the best contributors in advance. Many academics are terrible writers, or don’t take criticism well, or are committed to pushing a point of view. Same for many authors or journalists. For many of these professional writers, it’s an art form, and they resist quality discipline for the same reason that craftsmen resisted mass production, quantitative productivity metrics and quality control. They feel, rightly, that it interferes with their ability to both create and to claim credit for innovations. You are going to face social barriers if you rely on these professionals who have, by definition, started their careers in other media that do not subject their work to mass peer review. Wikis aren’t new, they’re just revision control systems from software engineering applied to documents. I’ve been using them for 20 years by various names (usually ‘document repository’) though usually within corporations. The assembly of USENET FAQs was clearly a product of the exact same process that we call ‘wiki’ today: universal readership, mass peer review, anyone can fix or correct the content simply by creating their own version (though central repositories did exist for a few trusted authors to certify certain versions as correct). I’ll happily explain to anyone who has questions the long history of ‘wiki’, which certainly did not begin in the late 1990s. Free and open source software opened up the repositories somewhat with a more radical form of transparency, and made it possible to identify key contributors who no one could have identified in advance. Mostly new practitioners.
I’ll deal with payment and moderation in another post as there seems to be a length limit.
April 25th, 2007 at 10:02 pm
Ted Nelson in his Project Xanadu proposals forty years ago laid out a simple model of how to pay for collaboratively authored content, basically, per bit. It seems a lot of people are still under the impression that this might work, again failing to deal with the different levels of integrity and talent and verifiability and trust invested in different editors on different subjects. Someone could be great on Shakespeare and have utterly crank biology views, for instance, so trying to tie up the scheme with overly general credentials is a big mistake (why “PhD” just doesn’t work as a general credential).
Payment and moderation are areas in which The Economist might be able to innovate. And in which it would be widely embarassed if it failed, since it’s supposed to understand both economics and editorial policy better than anyone in the English speaking world. Right?
Accordingly, I’d make several recommendations which I can help you follow up with:
1. Allow wide-ranging credentials similar to those you’d use in journalistic best practices in attributing comments; Let people describe themselves as “a source close to the President of the United States” if you can in fact verify that is true, and defend the anonymity of that person as much as you’d defend it in the magazine. The difference is, the source can write what they want, at least in the developmental version of The Economist’s wiki edition.
2. Accept up front that factions exist and that many people write to argue positions on issues. Focus on getting the issues neutrally stated, possibly by relying on say economists of different stripes (classical, Marxist or welfare economics, neoclassical, green/ecological) who must sit in a room drinking dreadful tea until they agree (I will moderate if expenses are paid including someone to call when I am contemplating suicide from being stuck with all these madmen). Then permit multiple points of view to emerge, as position statements that can be as biased as the advocate prefers. Then back these with arguments that must again be more neutral and cite evidence from sources and authorities that The Economist’s readers would recognize and respect (For more details on why this approach works, see
http://openingpolitics.org ). Let even minority factions state their view as its own position but open it to the same scrutiny as anything else. The position statements could be vetted to some degree but it’s hardly a problem if the debate extends into obnoxious ideologies or dogmas. Work on the assumption that the more these are exposed, the better articles get. (See http://openpolitics.ca/Hamas as a good example of a well defined IPA statement).
3. Moderation should operate between factions, not between persons. Beyond some good general rules of engagement, once it’s clear that the debate is political or otherwise biased hopelessly by prior assumptions about the universe, the factions themselves must take the responsibility for sorting out collective responsibility. In other words, if someone says that “Marxists accept that…” then it must be near-automatic to have the Marxist faction review that comment to see if it is, in fact, something that the vast majority of Marxists do accept. Who else could speak for them? Without organized factions, you degrade quickly into chaos and definition gaming, which is why no parliament in the world avoids political parties. They save everyone time by forcing ideologically rigid people into debates with those most like themselves - but with the ultimate objective of convincing neutral voters. This is much better than street fighting, and it’s much better than trusting absolute judges.
4. Payment likewise should operate factionally. The overly reductionist and individualist neoclassical economists might be very happy with a model where all edits by their faction are counted up by the simple sentence equivalent and those that remain unmodified the longest get paid the most, with the money going directly to the individual author. After all, they seek aggregate numbers at all costs including rubber-stamping the noxious GDP, and believe in single-goal optimization as if it were holy writ. So why not let them work from their own rules? Meanwhile, Marxists could happily pool all revenues received and devote a portion of those to paying agitators and vanguard “trolls” to irritate neo-classicals by pointing out all the problems with the payment system. The classicals can set up a nice hierarchy of specializations and allocate just enough funds for defense, infrastructure, justice, education and a stable currency that they trade in amongst themselves - only. While the greens can charge editors who focus on aggregates an “abstraction tax” and tilt the playing field towards paying those that point out implications of economic choice on the natural world, and elaborating the economic models of say pollination, erosion and oceans.
5. Don’t make everyone join a faction, but make it so advantageous to do so that it makes little sense [not] to either do so or form one. Make allowance for students who are still choosing their approach to economics, and who are much more likely to pick up on the wiki method. Make a definite practice of recruiting talented editors from the user base and giving them minor editorial assignments like fact checking or compiling backgrounds on political and economic figures. Feed back to the open content community by releasing the dossiers to Wikipedia and SourceWatch under GFDL and CC-by-sa open content licenses, once you have versions that all factions and The Economist’s own editors agree on - which will be a temporary situation so act fast! This will provide links back to The Economist from these heavily read wikis, and enable recruiting more people from their talent base.
6. Pick a few high-profile debates that are going to be the subject of major Economist issues, books, events and roundtables. Devote some (or all) of the marketing budget to massive outreach to the entire planet to find literally everyone who’s written on that one issue or subject. Make clear that you intend to frame the debate for the next 20 years and it’s “now or never” to influence The Economist’s view of the subject. Coach the old folks into using the wiki and make clear that pompous letters to the editor will not be heard on this issue, they have to click edit, type something, and click save, even if they choose not to identify themselves or claim any credentials (it’s not hard for The Economist however to know who’s who, IP numbers don’t change that much). Devote some cash to pay a few people in India to put everything that was said in letters to the editor into the wiki to provide starting commentary and simple issue/position/argument trees that can be elaborated. At the actual events or roundtables, insist on capturing everything and keeping the dialogue in that form. Then present the IPA summary in The Economist itself. A dozen low-overhead minor efforts to do this compilation per year, including one major one (this year it could be climate change), would show everyone how to do it for their own pet issues. Expect The Economist’s own paid staff to be doing maybe 5-10% of the work, paid outsiders that you know and trust maybe another 10-15%, and paid newcomers 20%, so you are matching the massive input from outside with a fit degree of editorial guidance, but still clearly letting the lunatics rule the asylum.
7. Payment models can be dealt with democratically: why not let the users vote on them?
8. Prediction markets provide an easy way to hedge against dire predictions of failure or non-optimal contribution (which of course every economist will cry at first opportunity in order to propose their own pet model), so make use of them. Let those who bet that the content in certain areas will drop drastically without this or that payment provision, subsidy or tilted playing field, put their money where their mouth is. UK betting markets are very active on literally everything, and letting the bookies take on the bookish is always a fun show. Consider (as I’ve advocated in Canada) taking bets on political promises and their fulfilment, which is a great way to embarass politicians and force parties to oust their liars.
9. Think, ultimately, in terms of an entirely new currency with rules optimized not for the infrastructural/manufactured/physical capital we built our industries with, but instead for the instructional capital that we now use to build our nature-compatible biomimicry society.
And be ready to encounter whole new species of trolls emerging from the muck. Familiar perhaps, as they carry the DNA of all our ancestors who fought for freedom of speech or the press, for democracy, for transparency, for justice. Familiar also, because they are us.
Craig Hubley
known troll