1 MScBA Master’s Thesis
Value creation with online communities:
Selection systems in Wikipedia
By Arjen Riezebosch BSc, BA Supervisor: Dr. H. Snijders Examiner: Dr. T.L.J. Broekhuizen) University of Groningen
Specialization Program: Strategy and Innovation, Faculty of Economics and Business
Postbus 800, 9700 AV Groningen
February 2009
2
Abstract
3
Preface
“The best way to predict the future is inventing it.” (Alan Kaye)
When photography was invented people thought painting would disappear. When film was invented it was received as ‘moving image’. When radio was invented it was used to confirm that a telegraph message was clearly received. When SMS was invented telecom companies had no idea what it was good for.
In this thesis I tend to go along with the suggestion that internet is good for interaction, empowerment and collaboration. But I realise that this might seem an utterly foolish assumption fifteen years from now. We might long back by then to the times that we could passively watch television without choosing or contributing.
I see Wikipedia as an attempt to understand the possible use of internet. I think it is an early stage of development and will be totally different ten years from now. And yet it is interesting to look already at its impact and output. Most companies that have tried to capitalize on the new participatory culture of what is called Web 2.01, disappeared as fast as they came. Some are very successful. But still they changed how we can think of organising and collaborating between groups of people. I found that a very interesting development to look into.
I thank Hendrik Snijders for the open minded and reflective discussions that helped to shape this thesis towards its final result.
Groningen, February 2009,
Arjen Riezebosch
1
4
Index
1. Introduction ...5 1.1 Relevance ...7 1.2 Research method ...7 1.2.1 Case research ...7 1.2.2 The cases ...82. The wikification of knowledge ...9
3. Theory ... 13
3.1 Online Communities ... 13
3.2 Selection Systems ... 15
3.3 Actor Network Theory ... 17
3.4 Peer production ... 19 4. Case: Wikipedia ... 21 4.1 Introduction ... 21 4.2 Organisation ... 22 4.2.1 Interface ... 22 4.2.2 Roles ... 22 4.2.3 Tools ... 25 4.2.4 Rules ... 27 4.3 Selection mechanisms ... 28
4.3.1 Selection in article quality evaluation ... 28
4.3.2 Selection in editor quality evaluation ... 31
5. Case: KNOL ... 33 5.1 Organisation ... 33 5.2 Selection mechanisms ... 34 6. Case: uTest ... 36 6.1 Organisation ... 36 6.2 Selection mechanisms ... 37 7. Discussion... 38
8. Conclusions and recommendations ... 41
9. Bibliography ... 45
10. APPENDIX... 48
10.1 Glossary of terms... 48
10.2 Organisation Chart of the Wikimedia foundation ... 50
10.3 Example of a Wikipedia user-page ... 51
10.4 Example of a KNOL article ... 52
5
1. Introduction
Since its foundation in 2003, the Wikipedia online encyclopaedia has grown into one of the largest reference sites in the world. It is used by millions of visitors a day and now has more than 10 million articles in over 250 different languages. Most significant is however the fact that Wikipedia is edited by volunteers and everybody can contribute without having to meet special requirements. Wikipedia therefore is one of the best examples of a user-central social networking concept to link, create, distribute and rate a wide variety of content. It transforms the consumer into a simultaneous producer and consumer of the content.
The idea of the collaborative production of an encyclopaedia is not new. Diderot in 18th century France already tried to create an encyclopaedia in which “all things must be examined, debated, investigated without exception and without regard for anyone's feelings." (Dynes, 1999). What is new is the possibility to do so at low cost, low barriers to participation, and with collaboration over great geographical distances. As a result many people work voluntary to share their knowledge and create a knowledge base with more articles than ever before. By improving on each other and discussing improvements on the side of article development, consensus about the content of articles is reached. From empty skeletons that can change rapidly, articles grow into stable versions that almost reach a static form. Different encyclopaedias choose to make the article available to the reader at different stages of its development. The philosophy behind this choice is often based upon a discussion of minimum required quality. The quality discussion has influenced to a large extend the process of article creation and the role of the author in different encyclopaedias. This thesis refers to this development process in terms of internal selection. The reader on the other hand has a need and means to determine if the presented content fits his needs. This will be referred to as external selection. In this thesis I investigate how these selection systems function in the content creation of Wikipedia. Besides Wikipedia two other cases are analyzed. The first one is KNOL; an online authorative knowledge sharing tool from Google and the second one is uTest, an online testing community for software testing. KNOL was labelled a possible ‘Wikipedia killer’ at its release (Dignan, 2007) and uTest is interesting for its commercial aim specialized scope.
6 While the thesis aims specifically on Wikipedia and not on every possible online community with peer-producing mechanisms in it, I will first analyze what is so specific for the way in which Wiki-pages organize knowledge. The thesis therefore starts with a general introduction into the Wikification of knowledge. Online collaboration in content creation. Users can participate in a variety of roles in the development of (new) articles.
Next, a theoretical framework is defined for the analyses of current practices. The most important theories that are used are that of online communities, selection systems, Actor-Network Theory (ANT) and commons-based peer production. The description of the theories will also provide room to a critical analysis of it.
The first paragraph on online communities is an attempt to summarize some existing research on motivation, functioning and organisation of online communities. Much of the research is still very recent and as we will see in the discussion, not all of it can be applied directly to other communities.
Next the theory of selection systems is introduced. This theory is particularly useful since it looks to value creation as a process where different actors, individuals or organisations interact in an attempt to influence each other (Wijnberg, 2004). Selection is further split into internal- and external selection. General selection systems are market-, peer- and expert selection.
The Actor Network Theory (ANT) is closely related to the selection systems in that it investigates the interactions between what are called actants – that is human, non-human or technological actors. It states that the relation between actors and positions in a network is determining the distribution of value within the network (Boorsma, 2005). The different roles that users in Wikipedia can assume are an interesting example to see how roles and position interact and how that affects the importance of different editors in the system. By looking at the interaction I hope to see how the users shape the encyclopaedia and vice-versa.
Finally the theory of commons based peer production by Yokai Benkler. Why would people that do not know each other work voluntary together to give away their knowledge on a subject? Benkler searches for the specific aspects of a networked economy that allow for communities that produce content or information products.
7
1.1
Relevance
Web 2.0 or Wikipedia might not be the ultimate form the use of internet takes. They revolutionary changed however the way media content is produced and the way this production is organized. Since this is still in an early stage of development it offers opportunities to learn from Wikipedia the things that later can apply to other organisations or organisation forms.
Popularity is not the only or most perfect term to determine quality. Theory on selection systems differentiates between internal- and external selection. Applying this to Wikipedia may make an interesting case to see how the needs of external selectors are aligned with the result of internal selection.
With Wikipedia we face a strong shift from expert judgment from authority to popularity and peer-selection as a peer-selection form. This makes organisational forms less hierarchical but the question is also how it affects the outcome of selection processes. In other words, will it lead to a loss of quality? It will be interesting to see how Wikipedia manages their contributors to get a certain output.
The form of Wikipedia makes it very applicable in the area of knowledge-management. The tools and structure of Wikipedia can easily be transferred to a company’s own environment to collaboratively work on an internal knowledge base for formal and informal information and experiences. In this thesis recommendations will be made on how to organize the development process to suit the user needs. Two other cases are investigated to look for alternatives and as input for a discussion on Wikipedia.
Even if Wikipedia is not able to reach the highest possible quality, it can reach a very high quality at very low cost. For businesses it’s important what kind of work can be organized in this low cost, online collaborative manner and what are the disadvantages of doing so.
1.2
Research method
1.2.1 Case research
8 Wikipedia itself is a very rich source of information. All the discussions from articles are readily available and archived, together with vote instances, user statements and dispute resolutions. Next the metawiki gives insight in the development of the form, structure, rules and tools of Wikipedia. To extend the experience of the cases I signed up to the websites of all three cases. This allowed for interaction with other users and with the tools and thus for finding out if the real processes go as they are described.
Next to evidence acquired directly from the research object I studied existing research in a broader scope from crowdsourcing to user generated content. Since there are so numerous examples of Web 2.0 applications I decided to narrow down to one case with two other supporting cases.
The data is collected between August 2008 and January 2009. This is important since structure and content of all three cases can change rapidly.
1.2.2 The cases
Wikipedia is an online encyclopaedia written by its readers. It is one of the projects of the Wikimedia foundation that aims to grow, develop and distribute, multilingual content free of charge (Wikimedia, 2007). Wikipedia encourages readers to edit the articles, and thus created 10.7 million articles in over 250 languages. The question for this thesis is what processes can mange such an amount of voluntary work and what is the resulting output?
KNOL is a Google project that was introduced as an authorative tool to organize knowledge (Google Blog, 2007). It allows everybody to write articles about their knowledge. Collaboration with other editors is optional, verification of identity is encouraged and contribution to successful articles is rewarded. KNOL case will not be described tot the same extend as the Wikipedia case. It serves in this thesis to shed light on some specific aspects of Wikipedia where KNOL has made different choices. These choices are compared further in an overall discussion of the cases that are described.
9
2. The wikification of knowledge
In this section I will explain how Wikipedia is one example in the development of the internet that is still trying to invent itself. We need time to understand the possibilities of new media and what distinguishes them from old media. Maybe the Wikipedia model will not exist anymore in ten years. But looking at its organisation model, we can try to understand shifting principles of organisation that emerge out of the discovery of the unique properties of the medium internet. In other words: Wikipedia would have been thought impossible 10 years ago, and yet it’s there. That means that somehow we lack an economic model, or an organisational understanding of what makes this huge voluntary organisations function the way they do. And it makes us want to predict how these understandings might help us to create new organisations. The wikification of knowledge is an introduction into one of the principles underlying the collaborative creation of knowledge. It is an introduction into the topic of this thesis that will be developed upon in theory and practice in the next chapters.
Impossible or unpredictable? Discovering the ‘new’ in a new medium.
The internet as we know it is only less than 15 years old. All the things we have seen to come about would be merely impossible to imagine 10 years ago. We now have satellite images of the world, all patents of the world, the books of the world, phone numbers, everything that is for sale, all the weather and, movies, regulations, and home videos, at arm’s length. And all these things will be integrated with social networks, and tailored to our personal needs. All these developments will be made accessible with every device from a PDA Phone to the refrigerator. Not only are websites linked to other websites, or pages to other pages; within five years everything will be linked to everything else in the most literary sense of the word. Objects in the real world will have a virtual representation, that is to say they will exist as data, that points to other data and exchanges information about themselves (Kelly, 2007). That will make the internet basically a major machine that contains all the information in the world. That is very amazing, but we are not amazed. And the most amazing thing is that there is no economic model behind it (at least not one that we understand yet), everything is for free. Now this kind of change is almost impossible to imagine, nor predict. But what is not new, is that the impact of a new medium is only discovered long after its initial invention.
10 as wireless telegraphing. While radio was invented around 1895, the first application of radio as ‘broadcasting’ only emerged around 1920. It took 25 years to figure out that radio was not only one-to-one communication, but could be used for mass-communication. We first learned to understand photography as instant painting. Film was logically interpreted as moving image, radio as wireless telegraph, television as visual radio, email as digital post and with SMS the telecom companies had no idea what it was good for.
And what is even more striking, is that most often the inventors of these new applications where not even trying to invent something new. They just stumbled upon a new idea. In popular science this way of invention has been called ‘serendipity’. Or, in the more pregnant formulation of the Dutch scientist Pek van Andel, innovation is like looking for a needle in a haystack and returning from it with the farmer’s daughter.
Wikification
When you want to search a book, you can use an index or a glossary. If a text is digital, a search engine can find the word you are looking for. But machines have much more difficulty determining the meaning of a word, and the associations that go along. This is one of the things that can be reached trough wikification. Essentially it is the formatting of text to predetermined rules and making references within and outside this text.
At the beginning of Wikipedia, a large share of articles where collages from articles that could be found elsewhere on the web. Without much reference texts from everywhere where copied into articles and published. The major activity for Wikipedians was to search, select and merge existing knowledge. A paragraph from a scholarly article from a university could be followed by a schoolboys review from a forum. Wikipedia got much critique for this and the exercise of reworking text to a uniform encyclopaedia format was thought doomed to fail. Hereafter I will explain in what way wikification adds value to content. It leads to the aggregation, organisation, formalization, explication and summarization of knowledge. Furthermore it enhances the relation between- and the reflection on knowledge.
Formatting for re-presentation
11 content makes it possible to automatically use content in various ways. The content can now be re-presented, that is presented in another form for another interface. Let’s make this example practical. A website on ‘San Marks Basilica’ in Venice can be read on the internet. As a Wikipedia article however it can be taken apart in different components. Now the different components can be used for other purposes. A website with maps can show a little popup with pictures of the Saint Marc when you zoom in on Venice, a mobile device can show an architectural description, and a travel website can use the article summary. Now Wikipedia is not unique in this modular format of a page, but since so many pages are formatted in the same way, it becomes good for automatic reuse. Wikification thus adds value through the standardization of content that separates form and content.
Workflow (dis)organisation
Wikipedia’s organisational form is different from traditional firm or market organisation. It solves coordination problems not by forming a hierarchical scheme but by removing the difference between reader and editor. In other words it doesn’t put people around a problem but rather brings the problem to the people. While internet is expanding, the cost of communication and information exchange decrease. This allows for coordination of widely distributed (both in space and time) sources of creative effort (Benkler, 2002). Benkler suggests that “The dramatic decline in communications costs radically reduces the cost of peering relative to its cost in the material world. This allows substantially cheaper movement of information inputs to human beings, human talent to resources, and modular contributions to projects […]. It also allows communication among participants in peer production enterprises about who is doing what and what needs to be done.” However we should note that this is not only true for Wikification of knowledge but also in a much brother realm, for example in open source software development. Furthermore there is more to the Wikipedia work coordination than only managerial aspects, like for instance the sociological aspects of need recognition when being visible in a project online.
Collaborative Reflection
12
Semantic Linking
When you search for the term ‘Paris Hilton’, a machine has difficulties to determine if you are looking for a well known person or for a hotel room in Paris. It would be an important step forward if the machine understands when you look for the first and when for the second. The reason is not only to leave the results of the first out, but also to give suggestions that relate to the second. So if you are looking for a hotel room, you probable need a flight to and why not a restaurant to have dinner? These are advantages that require a service to semantically understand information.
In Wikipedia every possible topic that is mentioned in an article gets a link to that article. If that article does not exist yet, you can create the article. This method of internal reference goes much further than the traditional index or glossary in a printed book. It is now possible to extract from articles how individual terms within information relate to each other. By analyzing this links, both meaning and interpretation can be extracted from the article by machines. This allows for semantically build search and indexation methods that increase the manageability of information in an incredible way. This semantic web is believed to be an important element in the next generation internet (Spivack, 2006).
Conclusion
13
3. Theory
3.1
Online Communities
Online communities are at its core communication systems that allow many-to-many communication (Butler, 2001). Members have shared goals, needs, interests or activities that form the most important reason to join a community (Whittaker, Issacs, & O'Day, 1997). Relations between people are not necessarily personal, although research shows that interacting in online communities contributes to social needs of people (Bishop, 2006). Internet is particularly effective to support weak ties that are less personal and intimate (Wellman & Gulia, 1999). Because distance does not play a role in the ability to join, online communities can be more divers and even niche topics are suitable for forming communities around them.
The level of involvement of users with the community tends to increase over time, and several authors have made attempts to develop a ‘membership lifecycle’ for online communities. Kim (2000) describes an increasing participation from low involved visitors to fully committed insiders in a five steps.
Figure 1 Membership Life Cycle for Virtual Communities (Kim, 2000)
While some authors (Preece & Moloney-Krichmar, 2003) see ‘shared social conventions, language and protocols’ as a basic characteristic, the membership lifecycle shows that compliance to those values might vary over time. As a result, efforts have been made to inscribe social conventions and protocols in the technical usage of an online community. Users that govern the rules usually often get more editing rights, for example as moderator.
From a business perspective, the management of online communities is a difficult question. Most communities are self-organized and do not adhere to outside hierarchies. Goals are not always a-priori defined and from a management view it is difficult to align user behaviour with community goals.
Phase Behaviour
Peripheral Observation and viewing of the community content Inbound Making some contributions in content or comments
Insider Adds to the community on regular basis. Interacts with other users.
14 Moreover as Frederikson (2008) notes, “a large number of involved parties with misaligned goals, different capabilities and diverse degrees of involvement, raise the issue of governance of online communities.
Different community goals demand a different level of involvement. Rating a movie on imdb.com costs neither effort nor time and there is not much interaction with other community members. Writing a hotel review on tripadvisor.com takes more effort, but interaction is still low. Writing an article on Wikipedia however takes both effort and interaction, since articles can be adapted by everybody. The more pretention of objective reliability over subjectivity, the more interaction between users is demanded. At the basic level, however, the model of reliability is not based on an outside criterion, but reliability equals consensus.
15
3.2
Selection Systems
Online communities are aimed at creating value, for both users and owners. The framework of the selection system can be used to determine value in different competitive contexts. Wijnberg (2004) states that the framework is suited to organisational theory because “it views the process of value creation as a process in which different actors, individuals and organisations interact and attempt to influence each other.” (Wijnberg, 2004) The framework focuses on actors that are selecting, the actors that are selected, and the relation between the two. A typology is then made between three different selection systems: market selection, peer selection and expert selection.
In market selection the consumer is the selector and the product is the selected. In peer selection the group of selectors and selected are essentially the same. An example of this type of selection is peer-review of academic papers. Expert selection attributes a special authority or evaluating capacity to a small group of people that are not in the group of the selected. An example of this is an art council that distributes money according the rating they give to art initiatives.
Aldrich (1999) distinguishes between internal- and external selection. The internal selection system determines the outcome of the internal competitive program. With regard to innovation an example would be an organisation selecting one idea for product development over another. Once the result of the internally selected is distributed outside the organisation, the external selection system determines the value of it. In an economical environment however, the internal selection criteria will be connected to criteria of the external selectors. In other words, the expectation of the market puts weight to the selection inside the organisation (Knudsen, 2002).
Jacobs (2004) suggests that both internal- and peer-selection are special forms of hierarchical selection. The first for historical reasons, since organisations have hierarchical decision making processes and the latter because in the role of selector, the selector stands above the selected and therefore a hierarchical relation exists. Furthermore Jacobs mentions the possibility of hybrid forms of selection.
17
3.3
Actor Network Theory
Network theory poses that the relation between actors and the position in a network are determining the distribution of value within the network (Latour, 1987). It is used in different perspectives, from the quantification of structure characteristics to the qualification of relations. Networks can be studied from the perspective of an organisation, an individual, a network as a whole or a single object (Boorsma, 2005).
The Actor Network Theory (ANT) describes the development of artefacts by a network of actors. An artefact is created through the interaction of technological, human and non-human actors, which are called ‘actants’ (Latour, 1987). The roles that are played by different actants can be described in terms of their identities, their relations to other actants, and the nature of those relationships. ANT thus suggests that artefacts get their meaning through interaction between artefact and subject, between technological- and non-technological factors.
One of the early theorists of ANT, Algiradas Greimas, introduced the notion of narrative program: one subject affects another subject and thereby creates a state of change. The subject undergoing change is called the actant, which is broader than character since it includes also animals, objects or concepts (Czarniawska & Hernes, 2005). Throughout a narrative the change of roles can provide the actant with a character, allowing it to become an actor or to remain object of an actor’s action (Czarniawska & Hernes, 2005). From an innovation perspective, ANT does not assume that technologies are immutable. Rather it suggests that technological artefacts are moved and changed by social actors who are engaged with it (Holmström & Robey, 2005). The influence between artefact and actants changes both players: Wikipedia is shaped by its editors, but the editors’ roles and position change also by contributing to Wikipedia.
18 about ANT here is that it does not take authority as a source for action, but as a result from the performances of actors.
19
3.4
Peer production
Why would people that do not know each other work voluntary together to give away their knowledge on a subject? From an economic perspective there are many objectives to such an idea. First of all, nobody would participate in a project if they cannot appropriate the outcomes of it. That is to say, nobody owns the result and therefore a lack of motivation is to be expected (Ostrom, 1990). The second objective is that nobody has the power to organize collaboration and therefore organisation will fail. Yet on the other hand we have communities of peer production like Wikipedia and they did organize themselves, and we know they are successful. This leads us to the interesting conclusion that we find something working in practice, but we don’t know if it works in theory. For that we will focus on the theory of peer production from Yokai Benkler, a Yale law professor who researches commons-based organisation of managing resources in networks.
In his leading article ‘Coase Penguin’, Benkler contrast the transaction cost theory of Ronald Coase with the commons-based creation of Linux that did not involve any monetary incentives to the community that build it. Linux was not build upon managerial hierarchies nor used markets for the organisation of their production. Many scholars have attempted to explain this phenomenon by searching for a ‘hacker ethic’ that was supposed to live in the software programmers mind. What is interesting from Benkler however is that he departs from the product perspective and tries to generalize into a broader scope of information products. He looks for specific characteristics of the product that hint for a broader application of commons-based peer production. Benkler argues that information products are by nature more suitable for another form of production organisation. The question is what are the dynamics of this socially productive behaviour, what makes it successful and is there an economic value?
20 cannot start the other day in the office of Microsoft without having to break one contract and create another contract that defines a new relation to a new position in a new cluster of a new organisation.
Not every type of product, Benkler concludes, is suitable for peer production. The majority of commons based peer-production seems to go hand in hand with the growth of the networked information economy. Therefore Benkler looks for characteristics of the resources used in information production in relation to other products and the cost of communication within the productive enterprise (Benkler, 2002).
Factors that encourage commons based peer-production
Factor Implication
The result is nonrival Consumption does not decrease availability Information is the main resource There are no resources like raw material used
Production costs are low The availability of computers and networks decreases production costs Input of humans is variable One person can and will contribute more than another person
Communication is cheap This aspect makes the coordination of the effort easier Figure 2: Factors that allow for peer-production
Benkler finds four attributes of the networked economy particularly important for peer production of information. The first is that the object of production – information – is nonrival in the sense that consumption by one doesn’t decrease availability for another, and that next to human input the most important input is information. Secondly that information production costs have dramatically declined by the improvement of the availability of computers- and networks. Thirdly, that the input of humans – creative talent – is highly variable, that is to say they are themselves aware of their knowledge, skills, motivation and focus. Fourth, communication is cheaper and more efficient, the result of which is that coordination of the creative effort is now much easier (Benkler, 2002).
21
4. Case: Wikipedia
4.1
Introduction
Wikipedia was found in 2003 and is one of the free knowledge projects under the Wikimedia foundation2. The charitable foundation aims at expanding free knowledge around the world. It employs 20 paid staff and is the official representative of its projects3. Furthermore the foundation takes care of the legal, administrative and financial support and of technical maintenance of servers and software. All other tasks, that is to say the creation, editing and editorial oversight of the top-5 most visited website in the world, are run by volunteers.
In 2007 the revenues for Wikipedia totalled 7 million dollar, coming from individual donations, foundation grants and corporate in-kind donations. The major expenses for the foundation are in technology and salaries. Every year the foundation runs a funding program to finance the maintenance and development of the different projects (Wikipedia, 2008).
With 10,7 million articles in over 250 languages Wikipedia reaches over 250 million unique visitors each month. First found as Nupedia, Wikipedia wants to be a free encyclopaedia. While Nupedia was free for editing by everyone, but different from Wikipedia it had an extensive peer review mechanism and required a certain expertise to become editor. The publication process however was slow, and in 2000 Jimmy Wales and Larry Sanger decided to make a more open form of collaboration in a wiki format. Later on they had disagreement about the openness of Wikipedia and the role of the expert. Sanger left Wikipedia to start a new initiative, called Citizendium, which is based on expertise and peer-review. Wikipedia continues under the vision to give “all people on the planet free access to the sum of all knowledge” (Wales, 2004)
In 2006 the founder of Wikipedia, Jimmy Wales, found a new project outside the Wikimedia foundation called Wikia. This is a for-profit website aimed at creating ‘mini-Wikipedias’ for specific topics like cooking, gaming or genealogy. These websites use the same technique and methods as Wikipedia.
2
A screenshot of a Wikipedia user page can be found in the appendix under §10.3 3
22
4.2
Organisation
In the introduction we have seen how the website of Wikipedia is embedded into the broader organisation of the Wikimedia foundation. While the project is still growing, the roles, tools and procedures change over time. The process of shaping content and structure and the selection mechanisms of that process are described in the next paragraph. In this paragraph the technical and social actants and actors in the encyclopaedia are described.
The basic elements of Wikipedia are its interface, the roles, the tools and the rules.
4.2.1 Interface
The layout of the website determines the interaction between the user and the content. The usability of that interface is important since it determines the accessibility of the website and its tools for the user. Basically the main elements in Wikipedia are that of the article page and that of the user pages. Both elements have the same user interface and underneath they store a discussion page and a history page. A simple text-based editor allows for editing of the article, user and discussion pages, the history keeps track of changes and allows to restore older versions. Schematically the interface looks like this:
Navigation works via category-trees or referral via links within the article to other topics or via the search box. Coherence within categories is created by including tables with contextual information. Articles end with internal references (‘see also’), used citations and external references.
4.2.2 Roles
In ANT terms the user in Wikipedia is an actant that changes its roles as he interacts with other actants, or with actors which in this case can be articles, tools or rules. In other words, the articles, rules and tools are influenced by the user, and his role is on return changed through the interaction. The user of Wikipedia can assume two basic roles: that of the passive reader or that of the active editor. It requires no prove of experience, expertise or knowledge to assume an editing role, nor are contributions reviewed before they are published. There is however a number of roles that creates some sort of editorial oversight.
Userpage
Discussion Page Up d ate d h is to ryArticle
Discussion Page Up d ate d h is to ry23
Reader
Editor
Administrator
Bureaucrat &
Steward
Arbitration ComitteeIt is tempting to simply provide here an overview of hierarchical levels, based on the technical abilities that are linked to the different roles. But the basic idea of Wikipedia is that different technical levels do not per se reflect a hierarchical differentiation. The restrictions on editing for the basic editing level are thus very limited. The assumption is that on the level of editors different users will correct each other thus reaching consensus in the long run. As a result of this, the level of influence one has is more based on activeness than on hierarchy. This will further encourage editors to contribute to the topics they have more or less adopted.
While roles and abilities have changed and will changes with the growth of Wikipedia, there are a number of basic roles to describe. Next we describe the roles of reader, editors, administrator, bureaucrat and steward. Like the model shows there is a potential overlap between roles and relations are not necessarily hierarchical.
Figure 4: User roles in Wikipedia
Reader
The main difference with a normal encyclopaedia and Wikipedia is the interface and the absence of a distinction between reader and editor. While the interface points to the editing possibility it invites the user to interact with the content. The lack of official moderation means for the reader that he has no clear clues on the quality of what he is reading. So while reading is a passive role, the user is still invited to take a more active role.
Editors
24 specialize in cleanup, discussion, contribution or maintenance. Individual users cannot delete, lock or promote articles. There is no formal reputation mechanism, although the community seems to have developed its own criteria for assessment of someone’s reliability. Registered users and users with many edits are usually more credible than users that are new to the community. Research on the motivation of editors shows that the sense of community can be an important element (Kuznetsov, 2006).
Administrators
Administrators in Wikipedia are mainly very active editors with extended editing rights. They often are better aware of style and quality standards and can thus promote and enforce them on other users. Besides formatting issues administrators are responsible for moderation and closing of discussions. According to the Wikistatistics there are around 1600 administrators (Wikipedia, 2009).
Example of administrative involvement:
Amin.: “Removing the image was enforcing policy and thus an administrative act. [..] You say "never, or close to it, save the instances of blocking a blatant vandal who is vandalizing an article on which that administrator is working", but this just not our policy buddy, that is just what you think it should be. I have quoted the relevant policy, I have explained it, it is like you are not wanting to hear this. You keep saying editorial action, but it was enforcement of policy. I think I will just wait and let someone else explain it to you because you are not listening to me. Chillum 15:32, 1 August 2008 (UTC)
Editor: “I think you're wrong in your view of what constitutes an administrative act.”
(Discussion between an administrator and another editor on the discussion page of an article on the role of the administrator) (User:Chillum, 2008)
When compared with a normal organisational structure, the span of control of the administrator is enormous. The limited roles and levels implicate that although there are differences in functions, the most corrections must be made at the lowest level. Those guidelines are enforced upon each other in the normal process of editing and not top-down via administrators. The role of the administrator is mostly aimed at avoiding and correcting vandalism instead of managing the editing process.
Bureaucrats and stewards
25 for batch-editing. Administrators can become bureaucrat by requesting that role. The request initiates a voting process where other editors can interview and judge the application.
The steward function is established to separate the software development from the assignment of user rights. At this moment the assignment of user rights is still in the hands of the community. Stewards can be elected in a yearly election round. There are in total 36 stewards (Wikipedia, 2009).
Arbitration Committee and other roles
The arbitration committee dissolves disputes that cannot be solved within the community or by administrators or via mediation. Next to the roles described here, there are a number of other, mainly technical roles. These are among other things directed at maintaining the servers that run Wikipedia, and developing and maintaining its code.
4.2.3 Tools
Wikipedia has a number of tools and processes available to its users to support the work they are doing. These will be discussed in the next section. The most basic elements of editing and article development are shown below.
Article editing
The edit page opens the source code of the article. The source is in Wikipedia mark-up language, a style formatting editor that uses text coding to build the articles. This code forces the user to study to some extend the style guidelines because without these it will be difficult to learn the code. Modifications are immediately published. A disadvantage of that is the risk of exposing vandalized versions. There are however bots that can automatically revert edits with a high probability for vandalism. Controversial or popular articles can be semi-protected for. That means that users are demanded to log-in, so that bad usage can be linked back to individual users more easily.
Discussion Page
A discussion page is mostly used for communication among editors on content and work coordination of articles. That is on inclusiveness, scope and size of the content. The discussion page opens the possibility to build consensus for ones contribution and thus reduce the probability that the next editor reverts the change. A suggested change on a much edited article will sustain longer if it has been discussed with other users. Therefore the discussion page embeds to a large extend the vision of consensus that Wikipedia builds on.
26 guidelines. These percentages make the discussion pages a fairly efficient tool for work coordination (Visual Communication Lab, IBM Research, 2007).
Content of discussion pages
Information request 10,2% Off-topic remarks 8,5% Coordination request 58,8% Reference to guidelines 7,9% Internal resources 5,4% Reference to vandalism 3,5%
Peer review request 0,3%
Other 5,4%
Figure 5: What is the discussion page used for? (Visual Communication Lab, IBM Research, 2007)
User pages
The user page4 is the homepage that every user gets when he signs up to. In that way a community is created that shows ones activities and achievements within the community to other users. The larger community is divided in smaller groups that work around articles of their interest. Like the article page, the user page has its own discussion space. Therefore discussions not only form around articles, but also around contributions of specific editors. This is where actants become actors, that is to say their interests are inscribed into the content that is produced and their status changes. Discussion pages by their nature become a space of social interaction and can serve a communal feeling. Viégas (2006) shows how discussion pages have grown faster than the article- and user pages itself. This implicates that ties between different users have grown when editing on Wikipedia.
Figure 6: Coordination in Wikipedia; Growth rate of page types between 2003 and 2005 (Visual Communication Lab, IBM Research, 2007)
** Pages that explain policies and guidelines and talk about Wikipedia sister projects
4
See appendix for an example of an user-page
Page type Growth rate
27 The growth of user talk pages suggests also that peer recognition plays an important role in the motivations to contribute to Wikipedia. The user page makes visible what an individual user has contributed and achieved within the community. It also allows users to expose other aspects of their identity. This is often is seen when users put ‘real life’ information on their page. User discussions further allow users to be asked for their specific knowledge, opinions or expertise. The dynamics between the article discussion page and the user discussion page ties contributors to the article they are working on and keep them involved. That is to say that from the article discussion, readers and writers can be drawn to the user discussion page, thus granting the user authority or expertise on the particular topic.
Article history
In history the article version of the article shows and allows for reversion. The user can see the age, the number of edits, frequency of edits and information about the different editors. For important articles the number of edits easily reaches over 1000 in 1,5 year. Different versions can be compared and changes are linked to discussion pages, or to the user pages of the different editors. Stability of the number of edits is one criterion of good articles, since it points to a state of reached consensus among editors. The article history further makes the development process and development stage of an article transparent.
4.2.4 Rules
The rules are the glue between the roles, the tools and the results. Developed by the community there are guidelines for behavioural and formal norms. The support architecture furthermore includes best practices guides, templates to common article elements, style manuals and other coordination tools. Policy changes come from three sources (Wikipedia, 2008):
1. Documenting actual good practices and seeking consensus that the documentation truly reflects them
2. Proposing a change in practice and seeking consensus for implementation of that change 3. Declarations from Jimmy Wales, the Board, or the Developers, particularly for copyright,
legal issues or server load
28
Figure 7: The selection processes result in quality indications that are useful for the user in its role as reader, and for the user in its role as editor
Article
Discussion
History
Quality IndicationsReader
(External Selection) - Evaluation of articleEditor
(Internal Selection) - Suggestions for improvement - Feedback on performance - Motivation for effort - Structuring of discussion4.3
Selection mechanisms
Selection is used here to describe the moments in development stages where articles and users are promoted or demoted. By focusing on the different selection processes, we can identify how different articles move along the workflow towards a fully developed article. Next the selection mechanisms for users describe the movement through different roles. The mechanisms are aimed at improving the value of the output: good articles and good editors.
Selection in Wikipedia can be seen from two different perspectives. There is the perspective from the user that as a reader decides if an article is useful or not. From the reader there is no feedback to the editor, unless the reader decides to get involved in the editing process. Next, there is the selection from the perspective of the user as editor. That is the judgment on every contribution that is made and the decision to keep the contribution or reject it. Together with article promotion, the user can be promoted as well. The selection process for the user decides when an editor is given more editing rights.
In the qualitative development of articles there are three important processes (Bishop, 2005), Article quality evaluation
Editor quality evaluation
Development of work coordination tools
The last one is a very technical process and will not be further described here. It is important to recall however that work coordination tools are developed upon the roles and rules of the community. That means that tools reflect sociological processes, they are developed through interaction between technological- and non-technological factors as also described in the ANT.
4.3.1 Selection in article quality evaluation
29
Article
Figure 11: Selection of article quality IV: status warning templates indicates quality to the reader Figure 8: Selection of article quality II: assigning article status on discussion page by comparing with article criteria
Discussion
<< St at u s > > Article CriteriaThe most important selection processes for editors are the status assignment on the discussion page and the version history of changes. The version history is technically a selection process, since every new contribution is published directly. Other users can choose to keep the change, to revert it or to change more. Therefore every contribution is judged by the next contribution in a process of continuous peer-review.
Every editor can request for peer-review on an article in order to get the status raised to the next level. This assigned status is placed on the article discussion page and loosely defines the quality-level and basic suggestions for improvement. The assignment of a status means that the article is compared with the good article criteria that are defined by the community. Deletion of an article will happen when it does not meet the lowest criteria. Regular editors can use software tools to monitor changes in articles and improve the editing process. Only administrators have the technical ability to delete an article.
Both version history and article status are not directly visible to the reader on the article page. Therefore it is not immediately clear for him how to assess the quality of an article. The only exception to this is the featured article status and the quality templates on the article page. With featured articles a small sign is shown on the article page. Next there are status warnings above articles, like ‘this is a stub’, ‘neutral point of view disputed’ or ‘citations missing’. In the editing pages most articles show a status, together with editing suggestions.
Article
Figure 10: Selection of article quality III: featured article sign indicates quality to the reader
This is a featured article.
Click here for more information. Template with quality warning
History
Versionhistory
οx date | user | change οο date | user | change οο date | user | change οο date | user | change
30
Figure 12: Continuous selection of article quality:
There are Wikipedia article development guidelines that give a status to an article from ‘stub’ to ‘featured article’ on a status scale of 8 stages. The number of featured articles in the English Wikipedia is around 0,09% of all articles and it can take for over five years for an article to develop towards that status.
All the aforementioned selection processes also provide editors with feedback on the necessary tasks to improve an article. Therefore editing gets a game-like structure where improvements are rewarded with the article moving to the next level of quality.
Editors that want an article to be promoted can nominate an article. Once nominated, a peer review process checks if the Wikipedia quality norms are met. The norms are made explicit in policies, guidelines, assessment criteria and the overall Wikipedia philosophy. They contain norms like comprehensiveness, accuracy, verifiable, stable, uncontroversial, appropriate style and focus, etcetera. The peer review team should apply the community criteria which gives also the possibility to judge the quality of the peer review process.
The combined selection processes form a continuous evaluation loop. In the loop every new contribution is compared with the existing article and the result can be compared with the good article criteria developed by the community. The different selection mechanisms function as quality evaluation and also as an indication of the tasks for improvement.
Article criteria:
Assessment Criteria Policies
31
4.3.2 Selection in editor quality evaluation
While every reader can behave as an editor, the editing role is mainly the choice of the user. There is no hierarchical mechanism that appoints editors, but merely a process of self-selection. Selection mechanisms furthermore appear in the management of editors and the appointment of other roles such as administrator or bureaucrat. These selection-mechanisms are further described here.
Wikipedia has created a community by giving every editor a user-page. The layout of these pages mirrors the article page. The selection systems for users work in a similar way for articles. To give feedback on edits or for discussion the reader can easily go to the discussion-page of one editor or check what other contributions an editor made. Editors have the choice to start a discussion on the discussion page of an article, or on the discussion page of a user-page of one of the editors of the article. In that way the user discussion page grows, which gives the same idea of evolvement that the article development implicates.
In ANT terms the linking to and from articles can be a strategy of translation since one article is strengthening its authority by being referred to by other articles. The same goes for the editor since he is the actant that shapes himself around the actor (the article in this case). He develops his interaction with the actor and can therefore become himself an actor of importance. Hence by actively involving in the editing process of specific articles one editor can enhance his authority for that topic. This process is very similar from that of informal power in every organisation. Formal role and practical position become separate due to informal influence one gets in the organisation (Cobb, 1980). Interestingly Wikipedia shows this informal influence since all discussions are explicitly visible on the discussion pages.
The growth of the userpage therefore serves also as a prove of ‘seniority’ among editors, even though the role stays technically the same. This seniority is a demand when an editor wants to be promoted to administrator or bureaucrat. His contributions serve as material for review to be elected for another role.
User
(editor) Discussion Page Up d ate d h is to ryArticle
Discussion Page Up d ate d h is to ry32
Figure 14: Selection of editor quality II: providing user status by giving feedback on changes according to user policies and article criteria
Discussion
<< St at u s > > Article CriteriaThese two figures show the way in which activeness can become a source of authority for the editor. The first one is the active involvement in specific topics that grant the editor authority. The second one is the active knowledge of the Wikipedia tools and rules. By giving users feedback on their entries based on knowledge of those criteria, an editor can strengthen its authority as ‘Wikipedia specialist’ and thus become an actor in that position.
Figure 15:Selection of editor quality I: try to get regarded as authority by writing many entries on a specific topic
The selection process of administrators is similar to market selection. If an editor wants to become administrator he can request for adminship. Other editors can vote afterwards to grant an editor these rights. This process is also what Callon and Latour describe as a ‘power mechanism’; the editor tries to persuade the others that he should be granted the role of administrator and that he can speak as an authority over other editors because he is ‘enrolled’ in the rules of the system (Callon M. , 1986).
User
(editor)Sta
tu
33
5. Case: KNOL
KNOL is a Google project to generate and organize authored content5. Google announced KNOL in the end of 2007 as an “a way to help people share their knowledge” (Google Blog, 2007). The project is officially released in June 2008 and after half a year it had just over 55000 articles – for comparison, Wikipedia had 100.000 in one year.
Unlike Wikipedia, KNOL is a for profit project, displaying advertisements within the articles and allowing writers to profit from the advertising results. Other important differences are the individual ownership of articles and a moderated editing model. Articles are published under creative common license, allowing for sharing and remixing of the content.
While KNOL aims more at knowledge sharing, articles are less formatted than encyclopaedia entries. This results in a higher difference in scope, size and construction of the articles.
5.1
Organisation
Everybody in KNOL can create a new KNOL after signing in with a Google-account. Creators of a page are the owner of the article and from there chooses in what way they wish to collaborate with other users. The default collaboration model is moderated collaboration, meaning all signed in users can suggest edits that have to be approved by the author before being published. Other options are open collaboration, where all users may edit, and closed collaboration with only one author per article. KNOL allows for more than one article on the same topic.
The system of rules and regulations is far simpler than the Wikipedia model: there are hardly any rules. Editing articles is easier with an html-editor instead of a special formatting code like Wikipedia. The filtering system is based on a simple user-rating that allows readers to rank the article from one to five stars. Next another widget shows the recent and total number of views for one article. Besides ranking, readers have the opportunity to write a review for an article or to post comments. Authors keep track of editing suggestions or changes via a revision overview.
In KNOL authors are encouraged to verify their identity. This is the result of the aim to make an encyclopaedia that is based on authority and expertise. The identification mechanism is advanced and includes verification via mobile phone.
Editors can choose to publish their article under different licenses from all rights reserved to creative commons attribution 3.0 license. The owner of the article can allow Google advertisements in his
5
34 article via Google AdSense. The program searches the content off the page and links that to relevant advertisements. Part of the price the advertiser pays goes to the author of the article. Google also offers the option to turn advertisements in articles off.
5.2
Selection mechanisms
Basically KNOL could be described as a monetized Wikipedia. But KNOL is different in more than one perspective. KNOL aims to put authority over consensus, it lets the reader decide on quality, collaboration is only optional, and there is a monetary incentive to contribute.
The choice for authority over consensus is the logic answer to the major criticism on Wikipedia. In that encyclopaedia a schoolboy can revert the edit of Harvard law school professor. KNOL invites users to verify their identity, thus increasing insight to readers in the expertise of an author. In practice however identity-verification is not required and every user can sign up and write a KNOL. So while there is more authority in the sense of power, there is less use of the ‘wisdom of the crowd’ to correct and extend the knowledge of one person. Position is given either beforehand by starting new articles, or afterwards by being voted for and having many views.
Readers can rank articles with stars for their quality. This makes up a market selection system, where Wikipedia leans more towards peer-selection. That means that KNOL tries harder to align the needs of the reader with the content of articles. The reader selects what is good or not or otherwise suggests how to improve the article. That means that in ANT terms the editor has less control over his own position, since it is more determined by other peoples voting than by its own activeness.
For KNOL there is only one important quality criterion: relevance. This quality criterion is at the basis of all Google advertising products. For advertisements to result into clicks the quality criteria of Wikipedia are not important for KNOL if they are not for the reader. The more valued the content, the more readers, the more exposure to advertisements, the more relevant clicks are generated resulting in money for Google. Neutrality, consensus, well-written or not and comprehensiveness is reduced to one simple click, the five star rating:
35 The last important difference in article development is the share of advertising results that goes to the author. As a result of that the motivation for KNOL editors is likely to be different from that of Wikipedia editors. In Wikipedia the motivation appears to come more from their position in a Wikipedia community and their role within the (success)story of the project. KNOL has less community aspects to it. When relevance and popularity explain for the quality and success of an article, than unpopular articles have little chance to appear on KNOL. If peer-influence and appreciation determine the willingness to contribute to a project for free, than the absence of community aspects in KNOL will possibly result in less contributors.
36
6. Case: uTest
The third case is not an encyclopaedia but a software testing community. That choice places the mechanisms of peer produced content into a broader perspective of application. The mechanisms are however comparable in how tasks are performed by a community to get content that is collaboratively produced. UTest offers testing services to companies by a community of professional software testers6. Furthermore uTest provides online tools to define, manage and communicate around the tests of new software. The website has the largest testing community with over 13.000 subscribed testers in 148 countries. UTest was labelled as one of the first companies to come up with a profitable business model for crowdsourcing (Wired, 2008).
From the start in 2007 Utest has raised $7,8 million dollar from private investors in two consecutive funding rounds. The most important investors are Mesco Ltd and MTDC (Crunchbase, 2008). The website showcases 51 customers in January 2009 and a total of 169 testing cycles.
The revenue model of uTest is based on a monthly fee of subscribing companies. That companies furthermore pay the testers for each bug that is found and approved. Large projects like a new version of Windows can usually rely on a large voluntary group of beta-testers. Therefore uTest seems more suitable for smaller companies, that lack an own testing team and that need overnight results in fast test cycles.
6.1
Organisation
The Utest websites serves as a marketplace facilitator between companies and the testing community. A company can provide the software for testing and define the requirements to the project and the testers. The advantage of a large testing community is that software can be tested quickly over different platforms, software environments or geographic locations. For usability companies can choose to test on different experience levels.
Utest features list of active testing projects, and allows keeping track of testers, seeing how many bugs are found and check the solutions. Next it shows the application maturity level, a track of the budget, indicating the readiness for market-release. When a bug is approved, the tester is compensated and graded for the quality of the bug report. Each type of bug has its own reward fee from a $2 dollar interface bug to a $14 dollar technical bug. Under the rewards table, a list of Top testers is shown and on the top of the interface a sign shows how much you have earned so far.
6
37
6.2
Selection mechanisms
In uTest the community can only contribute in one step of the total process of software development. Different from users in KNOL or Wikipedia they are not in control of the input or the final output. The selection system of uTest is mostly hierarchical. The company that issues the software can approve or disapprove the suggested bugs. There are some community aspects but the main way to motivate testers is to open the homepage of every user with the total earnings he has made.
Except via the list of ‘top testers’ in a project there is no good way to communicate the quality of individual testers. Users do not have a homepage in uTest, nor does their rate of activeness show on any other page. While the community aspect seems to be unimportant at first, it will become very important when a competitor starts an alternative to uTest with lower costs for companies and higher incentives for testers. In this case the lack of collaboration between testers, or the lack of community feeling can become a threat to the success of uTest.
While uTest suggests that all testers are experts, in practice everybody can subscribe. There is no formal selection for editors other than the subscription form. If that is the case for companies too, there might be a risk of malicious software or spyware being posted in the community that harms computers of testers. That is only one of the legal risks attached to the community testing site, although there hasn’t been any criticism raised to the legal aspects of it. Other risks include pirating of early versions and low secrecy of new products towards competitors.
