Wikipedia in the eyes of its beholders: A systematic review of scholarly research on Wikipedia readers and readership

Hundreds of scholarly studies have investigated various aspects of Wikipedia. Although a number of literature reviews have provided overviews of this vast body of research, none has specifically focused on the readers of Wikipedia and issues concerning its readership. In this systematic literature review, we review 99 studies to synthesize current knowledge regarding the readership of Wikipedia and provide an analysis of research methods employed. The scholarly research has found that Wikipedia is popular not only for lighter topics such as entertainment but also for more serious topics such as health and legal information. Scholars, librarians, and students are common users, and Wikipedia provides a unique opportunity for educating students in digital literacy. We conclude with a summary of key findings, implications for researchers, and implications for the Wikipedia community.


Introduction
For many years, Wikipedia has been the world's most popular reference source on the web. Since at least 2006, it has been consistently ranked as one of the top 10 websites globally (Perez, 2007), with about 500 million unique visitors per month as of February 2014. 1 Not only because of its popularity, but more so because of the influence of such a widely disseminated information source, an enormous body of scholarly research has developed where scholars have investigated diverse questions to better understand the nature of Wikipedia, an icon of the contemporary Internet age. A number of literature reviews have followed to provide overviews of this large body of research. These reviews have typically focused on Wikipedia contributors (Jullien, 2012;Martin, 2010;Yasseri & Kertész, 2013), on the content of Wikipedia articles (Jullien, 2012;Martin, 2010), or on scholars using Wikipedia as a corpus for text-based research (Medelyan, Milne, Legg, & Witten, 2009), but none has specifically focused on the readers of Wikipedia and various aspects of its readership. Given that the readers are by far the most numerous part of the Wikipedia community-and we might argue its very raison d'être-it is necessary to specifically review scholarly findings on this important aspect of Wikipedia.
In contrast, this present review takes a decidedly favorable perspective on noncontributory Wikipedia readership. In general, noncontributing readers constitute 90% or more of any given discussion forum, online community, or social website (Arthur, 2006;Li & Bernoff, 2011); a 2011 Readership Survey showed that the number for Wikipedia was 94%. 2 Although often called a "participation inequality" (Nielsen, 2006), such statistics indicate that reading constitutes the norm, whereas contribution is the anomaly (albeit a necessary one). Although we recognize the crucial importance of the Wikipedia contribution (Okoli, Mehdi, Mesgari, Nielsen, & Lanamäki, 2012), the breadth of studies we review in this present article offer a rich variety of perspectives of Wikipedia readers and their reading habits, demonstrating that noncontributory readership is not an illness in need of a cure; such readership is valuable in itself.
Specifically, this review offers two major contributions. First, we synthesize current knowledge regarding the readership of Wikipedia, spanning a broad range of research questions. Beyond merely recognizing that Wikipedia is popular, what specific categories of information are more popular than others? Is Wikipedia used mainly for information on leisure topics, or is it also used for serious purposes? It is widely assumed that scholars and academics are generally uncomfortable about Wikipedia; how true is this perception? Is Wikipedia helpful for students or does it harm their education? Although Wikipedia is run by a not-forprofit corporation, does it have any commercial value or usefulness? Many of these important questions have been answered by scholarly research, although some only preliminarily. In this review we examine how current knowledge has come up with answers on such questions examining different aspects of Wikipedia readership.
The second major contribution of this review is, besides synthesizing current knowledge, to provide an overview of research methods for Wikipedia readership for the benefit of both new and experienced researchers of Wikipedia. Thus, we analyze the studies in detail and compile various characteristics of the studies of particular interest to researchers. For instance, we note the various research designs, data collection approaches, and units of analysis that Wikipedia readership research has employed. This analysis would help researchers quickly determine the various research approaches that have been adopted, which would be helpful in guiding their research decisions. This review of Wikipedia readership is part of a much larger review of scholarly research on all topics related to Wikipedia. In an overview paper of the entire project, we described our systematic review methodology in detail , in which we identified more than 500 scholarly studies on Wikipedia published in English. In brief, we mainly focused on identifying peer-reviewed journal publications and doctoral theses and tried to be as exhaustive as possible in identifying these. However, we also included more than 100 influential conference papers. We restricted our search time frame to June 2011, after which time the Wikimedia Foundation (WMF) launched their Wikimedia Research Newsletter, 3 which summarizes current scholarly research on Wikipedia and other WMF projects. We analyzed and parsed research details from 476 studies identified from the systematic search. These research details are published on an interactive website that permits visitors to conduct various analyses on them (WikiLit: http:// wikilit.referata.com). In addition to the systematic search, we included many other scholarly studies that did not match the strict systematic criteria, but which we found relevant and worthy of inclusion, giving a total of more than 500 studies.
This review covers the subset of the larger review focused on Wikipedia readership. Although all Wikipedia contributors are of course also readers, we restrict our focus here to topics that concern reading Wikipedia without substantial treatment of contribution. We cover issues related to Wikipedia contributors and participation in depth in a separate review .
This review is organized as follows: After this introduction, we organize all the identified Wikipedia readership studies by topic, and summarize and comment on them. Next, we describe various pertinent research trends as analyzed from data on the WikiLit website. We then conclude this review with a summary of key findings, implications for researchers, and implications for the Wikipedia community.

Findings From Scholarly Research on Wikipedia Readership
Table 1 displays the topic categories of studies in our sample, with the number of studies in each category. Our categorizations have been determined by literary warrant; that is, we carefully noted the topic of each study identified and then grouped them into topic categories and subcategories such that each category had between 3 and 20 studies. Note that the numbers of articles in the leaf (terminal) categories do not add up to any meaningful total because most articles cover more than one topic category and are thus counted multiple times. The WikiLit website has 91 studies systematically identified and analyzed in detail. On the website, we report methodological details for each of these studies that we are not able to include in this article. In addition, as we explain in our detailed methodology , we identified another eight studies that we also summarize and discuss in this review, for a total of 99 studies of Wikipedia readership. As we describe in our detailed methodology paper, all the coauthors were involved in several rounds of assigning and verifying the topic categories for each study.
The major categories here cover studies about Wikipedia's ranking and popularity compared to other knowledge sources; the use of Wikipedia as a general source of knowledge on the Internet; various topics related to students as readers of Wikipedia; the extent to which Wikipedia readers consider it credible; software tools targeted to helping Wikipedia readers; and commercial aspects of Wikipedia content. In each topic section, we summarize each study that treated the topic. At the end of each section, we make some general comments about all the studies on the topic. Later, in the discussion section of this review, we highlight key findings from all the studies.

Ranking and Popularity
Since 2007, Wikipedia has been consistently ranked one of the world's top 10 websites; it is the world's top reference website. However, beyond examining mere popularity statistics, various scholarly studies have investigated more nuanced aspects of Wikipedia's web rankings. This category of 12 studies (13%) includes studies that compared the use of Wikipedia with other knowledge sources for getting information as well as studies that investigated the popularity of topics within Wikipedia. These studies have consistently confirmed that Wikipedia is a premier source of knowledge on the Internet.
Some studies compared Wikipedia's ranking with that of other important websites. Höchstötter and Lewandowski (2009) compared search results of four major search engines: Google, Yahoo, Live.com, and Ask. They found that Wikipedia is the most frequently represented website in all search engines. However, there are some differences in how different search engines rank Wikipedia pages: "Yahoo and MSN place the most Wikipedia results on their results pages. Google boost Wikipedia result mostly on first position but shows less Wikipedia links in total [sic]" (2009, p. 1810). DiStaso and Messner (2010) examined the rankings of Wikipedia articles of ten companies over four years on three search engines. In 2006, the Wikipedia links for these companies were in the top 20 for all three search engines; in 2010, these links had all risen to the top 10, which led to the conclusion that Wikipedia articles about corporations are quite important. Lewandowski and Spree (2011) noted that Wikipedia results shown on search engines are quite dependent on the quality of articles.
In a more focused study, Bar-Ilan (2006) studied a case of "Google-bombing," where the top results to the search keyword "Jew" yield the Wikipedia article and an anti-Semitic website. She observed that the Google ranking is primarily due not to pages that actually discuss these two websites, but rather to links from discussions and blogs purposely inserted to influence search engine rankings.
Four studies examined what is popular on Wikipedia. Ratkiewicz, Fortunato, Flammini, Menczer, and Vespignani (2010) provided a quantitative analysis of the dynamics of online popularity of Wikipedia content. They found that the dynamics of popularity are characterized by "bursts, displaying characteristic features of critical systems such as fat-tailed distributions of magnitude and inter-event time" (2010, p. 1). Spoerri (2007b) examined which were the most popular articles and topics on Wikipedia. He found that more than half of the most-visited pages are related to entertainment and sexuality and that the popularity of Wikipedia pages is related to search behavior on the web. He further found that search engines-especially Google-fuel Wikipedia's growth and thus shape what is popular on Wikipedia. He also examined the 100 most visited Wikipedia articles for five consecutive months, finding that 40% of these-mostly related to sexuality and entertainmentwere highly visited in all five months, and 25% were highly visited only in a single month (2007a). Waller (2011) investigated the search queries that directed Australians to Wikipedia pages and found that they search more for lighter topics such as entertainment rather than for more serious information.
In a study not strictly related to comparative ranking, Koolen, Kazai, and Craswell (2009) found that highfrequency web search queries often directly relate to Wikipedia pages. Within a large sample of web queries, 38% exactly matched the title of a Wikipedia page. The content and context of the matched Wikipedia page could then be used to expand the query. Wikipedia pages can also form an intermediary between a user query and a collection of books being searched.
Some articles compare Wikipedia's ranking with other sources of health information; we discuss these in Health Information Source (Johnson, Chen, Eng, Makary, & Fishman, 2008;Laurent & Vickers, 2009;Mühlhauser & Oser, 2008). In addition, another article that dealt with issues  (34) -Student information literacy (13) -Domain-specific student readership (12) -Cross-domain student readership (17) -Reader perceptions of credibility (21) ▶ Software for readership (12) -Computational estimation of trustworthiness (9) -Reading support (3) -Commercial aspects (10) *The numbers in parentheses are the numbers of distinct studies in each subtopic. Since a study might cover multiple topics, these numbers are not additive. related to Wikipedia's ranking and popularity is described in the Commercial Aspects section of this review (Langlois & Elmer, 2009).
Because of Wikipedia's role as the world's leading reference source, scholarly studies on finer aspects of Wikipedia's popularity are important. Existing studies have found that Wikipedia's popularity is driven by search engineswhich are in turn driven by user clicks of page links across the web. The most commonly consulted pages were related to entertainment, which corresponds to the nature of other highly ranked websites (e.g., Facebook and Twitter for social networking and YouTube for videos). Thus, we suggest that it would be valuable to investigate the popularity of webpages not so much by website as by genre, as the leading websites necessarily cut across various genres. Nonetheless, Wikipedia's function as a general encyclopedia assures that it has an ample representation of all subjects of human interest and thus will likely remain a leading source of web information for the foreseeable future.

Knowledge Source
In this section, we discuss the use of Wikipedia as a source of various kinds of knowledge, including current news, health information, as a resource for scholars and librarians, and for other information purposes. This covers 31 peer-reviewed studies (34%) as one of the main categories representing scholarship on Wikipedia readership.
News source. Wikipedia is a popular source of background information on current events; in fact, major news websites sometimes link to it. Moreover, because it is freely editable, it is often used as a public source of reporting breaking news. However, because of its "No Original Research" policy, 4 Wikipedia only permits posting of news that has already been published in other news outlets that can then be cited in Wikipedia articles. In fact, related to these reasons, in late 2004 the Wikimedia Foundation launched a sister project, Wikinews, dedicated to collaborative reporting of current events including original news reports, unlike Wikipedia (Wikipedia Contributors, 2013). Nonetheless, Wikipedia itself, described as "the largest form of participatory journalism to date" (Lih, 2004, p. 1), remains a popular source of news (Messner & South, 2011).
In one of the earliest academic studies of Wikipedia, Lih (2004) gave an introductory sketch of the then 3-year-old endeavor. He analyzed Wikipedia articles cited in the news in a 13-month period to compare their quality before and after citation in the press. He used computational quality measures based on the numbers of edits and of contributors and his analysis showed that increases in his quality measure were correlated with citation in the press.
Wikipedia was used as one of many sources of documentation on details of the delayed response in 2005 to Hurricane Katrina in the United States (Chua, Kaynak, & Foo, 2007). Thelwall and Stuart (2007) found that "Web 2.0 resources such as Wikinews, the Wikipedia, and the Flickr picture sharing site" (2007, p. 523) were important secondary sources of information provision and sharing in the event of natural disasters and similar crises. Nonetheless, traditional mass media remained the predominant sources of information.
Some studies have examined how newspapers frame Wikipedia and use it as a source. Shaw (2008, p. 41) reported that the Philadelphia Inquirer instructed journalists never to use Wikipedia "to verify facts or to augment information in a story," and that one reporter complained: "there is no way for me to verify the information without factchecking, in which case it isn't really saving me any time" (p. 42). Other news organizations, such as the Los Angeles Times, did occasionally permit citation of Wikipedia as a source. Messner and South (2011) reported that although newspapers had not referenced Wikipedia much in the past, their reliance on this source had recently been increasing, and they tended to present it as a generally accurate source.
Although these studies indicate that Wikipedia is indeed included in the collection of resources that journalists draw from, their attitude towards it is generally more cautious (Shaw, 2008) than that of scholars and librarians, as we describe elsewhere in this review. We suggest that a likely reason for this caution is that Wikipedia articles about current events witness an unusually high pace of edits, including correction of errors. In fact, articles about current events are often flagged with a special template to warn about the unstable (and perhaps inaccurate) nature of the page contents. 5 Thus, scholars' general esteem of Wikipedia may apply to more "ripened" articles, whereas journalists may be more cautious about those currently in flux.
Health information source. Many articles discussed the use of Wikipedia as a source of health information for the general public as well as for health professionals. This is distinct from its use by medical students, which we cover in Domain-Specific Student Readership. The majority of these articles critically examined the factual accuracy of Wikipedia's health information, although some other articles examined other aspects of health information.
A popular area of investigation has been how Wikipedia compares with other online sources of healthcare information. Since this involves not only popularity, but also the responsible provision of information that could affect people's health, accuracy was of primary concern in these comparisons. However, we do not attempt here a comprehensive review of rigorous studies of Wikipedia's accuracy for health information-we discuss this aspect in the Reliability section of a separate review (Mesgari et al., in press). Our focus here is on studies that have specifically considered Wikipedia as a source for public health information.
Probably the best general overview thus far published on this topic was coauthored by 19 members of the WikiProject Medicine, 6 mainly consisting of medical doctors (Heilman et al., 2011). They reviewed the literature in this area, finding that Wikipedia is an extremely popular health information source; Wikipedia's up-to-dateness is a major strength; its variable article quality is a major weakness for health information; health articles contain few errors but are often incomplete; and among the multitude of medical wikis, Wikipedia has the best potential for providing a unified platform of public dissemination of health information. They used the article as an opportunity to call on medical professionals to contribute in increasing the quality of Wikipedia's health information. In the rest of this section, we discuss various specific studies on this general topic.
In a study on the efficiency of web resources for identifying medical information for clinical questions, Wikipedia failed to give the desired answer in about one third of the cases, whereas web search engines, especially Google, were much more effective. However, Wikipedia was more efficient than medical sites such as UpToDate and eMedicine in terms of failed searches and number of links visited, and it proved to be the most frequent "end site" that provided the ultimate answer from a Google search (Johnson et al., 2008). Mühlhauser and Oser (2008) found the German Wikipedia comparable in quality to the websites of two major German statutory health insurance providers for content and presentation of patient information. However, in their assessment based on the standards of evidence-based medicine, none of the three sources proved satisfactory. Yermilov, Chow, Devgan, Makary, and Ko (2008) compared the quality of Internet sources of surgery information. Wikipedia's average information quality was less than that of professional societies, government, and hospital sites, but it was higher than the average quality of universities and manufacturer or pharmaceutical sites.
Using search engine optimization techniques, Laurent and Vickers (2009) investigated the Google ranking of the English Wikipedia for health topics. Queries based on 1,726 keywords from an index of the American MedlinePlus, 966 keywords from an NHS Direct Online index, and 1,173 keywords from the U.S. National Organization of Rare Diseases, they compared Wikipedia to .gov domains, Medlin-ePlus, Medscape, NHS Direct Online, and a number of other domains. They found the English Wikipedia as the website with the most top rankings. Using data from stats.grok.se for June and January 2008, they also examined health-related topics with probable seasonal effects, such as frostbite, hypothermia, hyperthermia, and sunburn. They found a clear effect in the page views. They also analyzed the page view statistics of three articles describing melamine, salmonella, and ricin. These examples were associated with official health alerts in 2008, and page view statistics showed a marked increase correlating with the timing of announcements.
Beyond investigating the quality of Wikipedia's health information, two studies examined the practical use of Wikipedia in awareness of existing concerns. Hughes, Joshi, Lemonde, and Wareham (2009) examined how Web 2.0 tools such as Wikipedia were being used in clinical contexts. They found that although medical practitioners were aware of credibility deficiencies of Wikipedia, they employed different strategies to cope with the risk while meeting their background information needs. Younger (2010) studied the potential of wikis and Wikipedia as an information source for nurses. She argued that although it is not likely for Wikipedia to replace the traditional printed valid information sources, it would be a promising starting point for nurses in searching for evidence-based patient-related information.
Two studies focused on how Wikipedia's wiki technology facilitates dialog on public health matters. Hickerson and Thompson (2009) examined Wikipedia and WikiHealth as case studies for promoting health information online, particularly these wikis' ability to engage the target public in productive dialog. Cimini (2010) investigated the impact of online dialogs on the meaning of Down syndrome and the extent to which these dialogs can change the way that "disability" is theorized.
In summary, most studies on this topic have found Wikipedia useful for general health information, especially because of its dialogic wiki characteristics and its broad popularity. Nonetheless, unsurprisingly, most medical scholars do not consider it a reliable source for healthcare decisions. Wikipedia, for its part, explicitly disclaims giving medical advice, 7 and beyond attempting to provide generally useful information, has no goal of being a primary source for medical decisions. In contrast, Medpedia (http:// www.medpedia.com) was launched in 2009 to meet the need for a wiki-driven medical knowledge base. It only permits editing by certified physicians or biomedical researchers, although it accepts suggestions from the general public. As of 2013, it was labeled as "beta"; only time will tell if it will eventually achieve its goal of becoming the web's primary source of authoritative medical information, replacing commercial sources.
Knowledge source by scholars and librarians. Perhaps surprisingly, quite a body of studies has been built up that investigate Wikipedia's use for their own research purposes by scholars and librarians, as distinct from their role as student educators. Although it is true that Wikipedia's most vociferous critics hail from these ranks, a large number of academicians in fact have quite positive, if nuanced, perceptions of Wikipedia's value.
Source for scholarly research. A Wikimedia Foundation survey has found researchers to be generally positive toward Wikipedia: More than 90% of 1,743 self-selected respondents were "very favorable" or "somewhat favorable" (Moeller, 2009). Among Public Library of Science (PLoS) authors, the result was 96%. To the question, "Would you be in favor of efforts to invite scientists to add or improve Wikipedia articles?" 68% answered, "Yes, on a large scale." Such results are positive for Wikipedia but may be biased due to the self-selection of respondents and because the publisher website with initial reference to the survey was open access-the responding researchers were likely favorable to open information resources in general.
Three studies have surveyed scholars to understand their usage and attitudes toward Wikipedia. Dooley (2010) surveyed 105 university faculty members and found that 54.4% considered Wikipedia to be moderately or very credible, 26.6% considered it having some credibility, and 20% considered that it had "no credibility." Of 105 respondents, 45 said they used Wikipedia moderately to frequently in their teaching or research, 40 only occasionally, and 20 said they never used Wikipedia for teaching or research. Despite controversies with student citation of Wikipedia, many professors and other researchers do in fact cite Wikipedia. Concerning citations of Wikipedia, Dooley examined 250 research reports published in 2009 and early 2010 from the Academic OneFile electronic database that contained "Wikipedia" in their text. She found that 27 of the papers featured Wikipedia as the main topic and 62 had brief mentions of Wikipedia. In all, 249 of these papers cited Wikipedia as a source. Chen (2010) found that although academics extensively use online information resources and databases for teaching and research purposes, they are often concerned about the credibility of Wikipedia content. Those who use Wikipedia are more likely to also be Wikipedia contributors. Eijkman (2010) observed how academics cautiously use Wikipedia along with other sources of knowledge. He found that although they are aware that it disrupts their traditional power as knowledge providers, academics are not generally as antagonistic toward Wikipedia as is commonly assumed. Page (2010, p. 343) compared the current state of Wikipedia's documentation of biological species with Wilson's vision of an "encyclopedia of life." He contended that in its dominance of search result rankings, contributor size, and potential linkage to other data, Wikipedia is currently the closest achievement of this vision. Surprisingly, he made no mention whatsoever of Wikispecies, the Wikimedia Foundation's project whose goal is more closely aligned to that vision.
In his article on the Israel-Lebanon conflict on Wikipedia and Wikinews, Hardy (2007) argued that although "Wikipedia is not a threat to the peer reviewed publications of academia, it is definitely a competitor, not only by dint of the number of people who consult it, but the quality of some of the articles in their own right" (2007, p. 22). Other scholars were more cautious in their appraisal of Wikipedia and pointed out some challenges that its use presents scholars. In her paper about conducting art history research, Chena (2009) recommended caution in using Wikipedia: "the content of the Wikipedia article can be used as tips for possible approaches to this object, but not as a source for the actual paper" (2009, p. 123). Knapp (2008) discussed the challenges that arise from citing amorphous web sources such as Wikipedia, where the source materials are constantly changing. She explored possible resolutions, such as a "virtual bookshelf" which includes electronic attachments of source materials along with a published work.
In a separate article , we discussed in more depth several studies that go beyond scholars reading and using Wikipedia to examine direct contribution to Wikipedia by scholars. Here we more briefly comment on such studies, emphasizing their implications for scholarly use of scholar-contributed Wikipedia content. Davis, Nikolic, and Dijkema (2010) noted that Wikipedia could function as a central hub of scholarly information for industrial ecology. They also illustrated how DBpedia, an online service with information from Wikipedia restructured in database format (Bizer et al., 2009), can be used to query Wikipedia with industrial ecology research questions. Huss et al. (2012) reported on Portal:Gene Wiki, an organized grouping of gene-related information on the English Wikipedia.
In addition to the studies on scholarly use of Wikipedia that we have discussed here, we later discuss Kubiszewski, Noordewier, and Costanza (2011) in the Reader Perceptions of Credibility section because they empirically examined scholarly use from that perspective.

Source for librarians.
Several studies discussed issues particularly pertinent to librarians. In our WikiLit website, we identify all library-related research in the "Library science" domain 8 -we identified more than 30 such studies. However, we normally assigned such articles in other topic categories if they are more specific about the focus of the article, but here we discuss those that are essentially focused on librarianship. These studies unanimously called on librarians to consider Wikipedia a positive phenomenon and to take advantage of it.
Some studies considered the use of Wikipedia inevitable, and so called on librarians to thus embrace it and even use it as an opportunity to teach information literacy. Choolhun (2009) documented that Wikipedia is increasingly being used as the first source for legal information inquiries by lawyers and law students. She thus called for increased engagement with Web 2.0 use by legal librarians. Gunnels and Sisson (2009) cautioned against avoiding Wikipedia and other Web 2.0 tools for research but, rather, urged teaching students to be critical about the information found on these sources and how to validate them through reliable sources. East (2010) examined the future of subject encyclopedias in the age of Wikipedia. He concluded that if librarians want to keep the use of subject encyclopedias alive, they must make them available online, easily searchable, and cross-linked with other resources.
Three studies went further to regard Wikipedia as a unique opportunity to promote libraries and librarianship and to spearhead their relevance to the forefront of the information age. Belden (2008) presented a case of how a university library was able to gain "dramatic increases in web usage and reference requests by harnessing the power of social networks such as Wikipedia and MySpace" (2008, p. 99). She explained that sites such as Wikipedia "provide the tools to allow dynamic, interactive means of sharing information and helping connect the dots," and that the skills needed in these activities are "the very abilities that librarians and scholars hope to inculcate in our educational endeavors" (2008, p. 110). Luyt, Ally, Low, and Ismail (2010) found that many librarians view Wikipedia as an opportunity for the profession rather than a threat. They considered this finding as an opportunity to connect with non-Western users and content and also as an opportunity for librarians to forge a leading role in the emerging information society. Jacobs (2009) commented on Hahn's (2009) study of student use of Wikipedia on iPods (described in the Cross-Domain Student Readership section). She examined how new information technologies such as Wikipedia influence librarians and academic libraries, suggesting that librarians engage in and promote the new technologies to the library world.
Source for legal studies. In addition to general scholarly studies, three legal studies examined the judiciary use of Wikipedia and discussed the controversy of using Wikipedia as an authority. Stoddard (2009) traced an increasing trend of usage over time. Breinholt (2008) classified the different uses of Wikipedia into four categories: Peoples (2009, p. 27) examined the quality of the Wikipedia articles cited by American judicial opinions. He found that the "majority of citations to Wikipedia entries in cases were not significant to the case but were merely collateral references." Peoples proposed a number of best practices for citing Wikipedia. He presented some cases and scenarios where Wikipedia should not be cited and others where citing Wikipedia could be deemed appropriate.
Overall, the studies about the use of Wikipedia as a source for scholars and for librarians show that, contrary to widespread assumptions, these professional creators and disseminators of knowledge more often consider Wikipedia a positive phenomenon than otherwise. Scholars and librarians not only widely use Wikipedia themselves, but often even promote it as a valuable knowledge resource. Although such a positive attitude is by no means universal, it indicates that even the traditional gatekeepers of knowledge have come to terms with the significance and value of a publicly generated knowledge source in the dawn of the 21st century.

Student Readership
A total of 34 studies (37%) investigated how students use Wikipedia, both as a general source of information and in student projects where they were assigned work that explicitly involved reading Wikipedia articles. These studies treated students in secondary school, undergraduate institutions, and in postgraduate education. Articles we classify here mainly involve student information literacy in critically reading and using information from Wikipedia articles. In contrast, we classify articles concerning projects where students are assigned to contribute to and develop Wikipedia articles as Student Contribution, which we discuss in a separate review . Because all Student Contribution activities necessarily require students to read Wikipedia, we normally discuss here only those articles that have no substantial component involving the students contributing content to Wikipedia. We categorize the following kinds of student readership articles: those that dealt with general matters of student literacy, those that treated students and Wikipedia articles across various domains of knowledge or field of study, and those that are restricted to students in a specific domain.

Student information literacy.
A popular research stream has involved studies of how Wikipedia relates to student information literacy. These studies unanimously called on teachers and professors to embrace rather than ban Wikipedia, urging them to seize the opportunity to educate students in information literacy skills needed for the 21st century.
Some studies discussed information literacy in general. Rand (2010) argued that students can learn critical thinking skills using Wikipedia. Jennings (2008) argued the necessity of information literacy skills for 21st-century students to become lifelong learners in using all information resources. He highlighted the importance of Wikipedia, as it facilitates teaching and learning such skills. Gunnels (2007) proposed using Wikipedia as a starting point toward a new way of teaching information literacy skills which enhances the quality of both the users and creators of information resources. Judd and Kennedy (2011) looked into how medical students use Wikipedia and other online resources to acquire their needed information. They concluded that higher emphasis on information literacy skills training is required to make sure students are able to locate and use the best available information.
Three studies described teaching experiences engaging Wikipedia for information literacy. Harouni (2009) reported using Wikipedia in a literacy class to teach students critical reading skills. After the lessons, students could clearly articulate their reference choices, and they were able to discern and use more comprehensive and unbiased Wikipedia articles. Patch (2010) argued that as students are already using Wikipedia, writing teachers should follow suit and incorporate Wikipedia into their teaching. She described some of her experiences with students and Wikipedia and argued that "many students are 'underprepared' to consume and use online texts responsibly" (2010, p. 282). She concluded that by employing Wikipedia, students can "have an easier time making the leap to higher-level inquiry and responsible scholarship" (2010, p. 282). In sharing their respective experiences in teaching about Wikipedia in computer science and anthropology courses, Aycock and Aycock (2008) proposed using Wikipedia to teach students not only about the use and interpretation of information resources, but also about management of rapidly changing collaborative information resources.
Three studies featured in-depth investigation of how students handle Wikipedia information. Calkins and Kelley (2009) examined history students' perception of Wikipedia credibility and collaborative work. Although the students were aware of factual errors in Wikipedia, they nonetheless believed that it was getting better as more people contribute and correct errors; they mostly favored accuracy and collaboration on Wikipedia. Sundin and Francke (2009) investigated how secondary school students negotiate the credibility of information in their learning process. Although these students used Wikipedia information, they were uncertain about its credibility because they employed traditional methods for credibility assessment based on authorship and origin, neither of which is clear in Wikipedia. Sundin and Francke suggested that the students need to update their credibility assessment approaches. We describe a subsequent related study by Francke, Sundin, and Limberg (2011) in the Reader Perceptions of Credibility section.
Whereas most of the articles we discuss here involve helping students to read Wikipedia critically, Chandler-Olcott (2009) argued that constructive writing is a crucial aspect of digital literacy, and so teachers should encourage students to write in digital collaborative environments such as Wikipedia.
Two other articles related to student information literacy more directly discuss how librarians should leverage Wikipedia as a resource for students, and so we describe them in the section of this review of Wikipedia as a Source for Librarians (Chandler-Olcott, 2009;Choolhun, 2009;Gunnels & Sisson, 2009).
As we noted, the studies on this topic all call for engaging students with Wikipedia as an essential part of their digital literacy education. The Internet has unleashed a flood of accessible information as never before known in human history; Wikipedia is only one manifestation of this. Because of its explicit attempt to provide neutral information, Wikipedia is an excellent context for educating young people in the critical skills required to make sense of the information deluge.

Domain-specific and cross-domain student readership.
A large body of studies focused on student use of Wikipedia articles, either focusing on students within a specific knowledge domain, or on students from various disciplines.
Humanities and social sciences. One of the perennially controversial questions about Wikipedia is whether it should be allowed as a citation, especially for student work. In early 2007, the History Department at Middlebury College decided to hold students responsible for using Wikipedia as a source after a batch of students had used erroneous information on Wikipedia about certain topics in the history of Japan (Waters, 2007). Media reports implied that the department of Waters, the teacher of the class involved, "was at war with Wikipedia itself." However, Waters himself actually told students "that Wikipedia is a fine place to search for a paper topic or begin the research process." The department adopted the following policy: Whereas Wikipedia is extraordinarily convenient and, for some general purposes, extremely useful, it nonetheless suffers inevitably from inaccuracies deriving in large measure from its unique manner of compilation. . . . Students are responsible for the accuracy of information they provide, and they cannot point to Wikipedia or any similar source that may appear in the future to escape the consequences of errors. (Read, 2007).
This policy is actually in line with the opinion of the Wikimedia Foundation. However, Wales, founder of Wikipedia, later said that he saw no problem in younger students using Wikipedia as a reference, and that it should be used as a stepping stone to other sources (Coleman, 2007).
Some researchers examined how and why journalism and mass communication university students use Wikipedia. Lim (2009) affirmed many other research findings that although students commonly used Wikipedia for finding background information with acceptable qualities, they were quite aware of its quality concerns and did not use it blindly. However, they did not verify the information on Wikipedia, but rather used its sources and links to get further information. Lim and Kwon (2010) compared student usage of Wikipedia by gender. They found that although male students used Wikipedia more frequently and had a positive attitude toward it, "female students displayed more cautious or conservative attitudes, emotions, and behaviors." Healthcare. Many studies considered usage by medical students, who are being trained to make life-and-death decisions based on their evaluation of information. Kennedy (2010, 2011) studied Australian biomedical students' on-campus use of Internet websites. Wikipedia's use increased "from only 2% of sessions in 2005 to 16% in 2008 and 2009" (2010, p. 1568). They concluded, among other issues, that students "are increasingly reliant on generalist information retrieval tools, particularly Google and Wikipedia, to support their learning activities" (2010, p. 1570). Fiore (2011) found that 47% of 186 medical students who recently completed psychiatric clinical clerkship used Wikipedia as one of the primary sources for preparing for psychiatry exams. Question books (88%) and the peer-reviewed website Up-to-Date (59%) were more frequently used, but textbooks (10%) less used. Among the students using Wikipedia, 84% also used question books.
Lavsa, Corman, Culley, and Pummer (2011) evaluated Wikipedia's appropriateness for pharmacy students. They found that all the examined Wikipedia articles provided incomplete or inaccurate drug information, and thus recommended against its use. In contrast, Haigh (2010) concluded that Wikipedia is an appropriate resource for use of nursing students, as a large sample of health articles cited reputable sources for their information.
Other sciences and mathematics. Korosec, Limacher, Lüthi, and Brändle (2010) found that students of chemical thermodynamics preferred the German Wikipedia over the chemistry encyclopedia Rompp Online because they considered Wikipedia more comprehensive and readable. They concluded that while both resources were good for initiating research, students should learn how to use both peerreviewed and non-peer-reviewed material in their learning. Wedemeyer et al. (2008) asked students to evaluate Wikipedia biochemistry articles. One third responded that they never used Wikipedia. Among the remaining two thirds, 12% used Wikipedia as their primary source and 31% used their textbook and Wikipedia equally. The remaining 57% used Wikipedia only as a supplement. The majority of the students preferred Wikipedia to the textbook. Jancarik and Jancarikova (2010) examined the appropriateness of Wikipedia material for preparing teachers of mathematics and biology in Czech. They observed that, whereas the English Wikipedia properly covered the topics with highly detailed articles, the Czech Wikipedia, whose scientific topics mostly consisted of English translations, was less detailed and comprehensive, and so inadequate as an e-learning resource. Schweitzer (2008) examined the coverage of psychology-related topics on Wikipedia. These were not only well covered, but the articles were also displayed on top of the major search engines. Students were found to use Wikipedia for personal and school-related activities, but generally not for their academic citations.
Cross-domain student readership. Many articles treated student readership in general, regardless of their domain of knowledge or field of study. Some of these generally investigated how students use Wikipedia. Tann and Sanderson (2009) examined the web-based information-seeking practices of university students. They found that many queries that have been previously considered informational by past research have since taken on a more navigational nature. Moreover, IMDb and Wikipedia have both accumulated a sufficient level of information to address the users' information needs. Head (2007) examined how students used online and offline resources in their research inquiries. She found that they used hybrids of both types of sources, although they used Wikipedia less because of their concerns about its reliability. Head and Eisenberg (2010) later discovered that when university students use Wikipedia, they are generally aware of its limitations in credibility and depth. It is more often used in the initial stages of research to obtain background information, and then is complemented with scholarly resources. Students mainly appreciate Wikipedia for its coverage, currency, comprehensibility, and convenience. Luyt, Zainal, Mayo, and Yun (2008) interviewed young people concerning their perception and use of Wikipedia. They found that Wikipedia played only a minor role in the lives of their interviewees, but these young people were quite aware of its drawbacks when they did use it. The researchers thus concluded that common concerns about Wikipedia's negative effects on young people are exaggerated. However, their study is hard to generalize, as only 15 subjects were interviewed, apparently all in Singapore. Maehre (2009) explored various pedagogical principles to encourage instructors to allow the use of Wikipedia in their students' projects. He argued in favor of producing and engaging in information creation. Moreover, Maehre promoted the focus on the content of a resource rather than the credibility of authors. According to Maehre, the world outside of universities and colleges classrooms is an interactive world. Thus, educators should work towards having information creators rather than information readers or finders.
In two unique studies, Hahn (2009Hahn ( , 2010 observed that undergraduates running the iPod Wikipedia app mainly searched for recreational and for short factual information. However, the students were all satisfied with the experience and found it useful for preparing a research paper. In addition to these studies described here, many other articles related to domain-specific and cross-domain student readership are described elsewhere in this review in sections where we discuss studies on Wikipedia as a Source for Scholars (Eijkman, 2010), Reader Perceptions of Credibility (Kubiszewski et al., 2011), and Student Information Literacy (Aycock & Aycock, 2008;Gunnels, 2007;Harouni, 2009;Jennings, 2008;Patch, 2010;Rand, 2010;Sundin & Francke, 2009).
Amidst all these studies on student readership of Wikipedia, a few generalities stand out. First, as can be expected, students widely use Wikipedia. Second, perhaps not as expected, even students who use Wikipedia for academic assignments are generally aware of its limitations as a reference source. Notably, for disciplines where accurate information can be life-critical (healthcare) or a matter of professional reputation (journalism), students were more likely to use Wikipedia as a source for citations to more "reputable" sources. Accordingly, healthcare scholars have sometimes recommended against the use of Wikipedia for student learning, although some have recommended its use. In one case in reaction to students misusing Wikipedia, a history department has formally cautioned against its uncritical use. Nonetheless, despite some professors' reservations, Wikipedia is popular among students because of its comprehensiveness in information included and accessibility in its readability, at least compared to some alternate more traditionally authoritative sources.
Rather than recommending against Wikipedia's use even in cases of inappropriate use by students, we reiterate the calls of scholars and librarians who emphasize the need to actively educate students in the critical assessment and usage of not only Wikipedia, but of any web resource. Regardless of professors' attitudes toward it, Wikipedia is a major information resource for students, and they would be best served in being taught how to critically and profitably use it rather than being cautioned to avoid it.

Reader Perceptions of Credibility
A total of 21 studies (23%) examined the credibility of Wikipedia from its readers' perspectives. We categorize articles here that examined readers' perceptions of credibility without attempting some kind of objective evaluation of reliability, such as by subject experts-we discuss those in a separate review in the Reliability section of Mesgari et al. (in press). This is also distinct from Wikipedians' insiderview perceptions of credibility of the articles they collaboratively create; we discuss those in a separate article as Contributor Perceptions of Credibility in .
Some studies examined characteristics of articles' presentation that affect readers' perceptions of their credibility. Veltman (2005) argued that access to the entirety of knowledge is becoming feasible with the open source and open content movements with information widely available on the Internet. Tracing a very brief history of encyclopedic compilation, he criticized Wikipedia because "critical tools concerning variants, certainty, authority, and significance are lacking" (2005, p. 23). He highlighted the central importance of quality along with accessibility in terms of quantity and proposed techniques for presenting knowledge in general on the Internet that facilitate readers' rapid assessment of its credibility. Korosec et al. (2010) compared chemical thermodynamics topics on German Wikipedia and the chemistry encyclopedia Rompp Online based on 30 articles. After evaluating the two references in terms of varied aspects of content quality, they report that both encyclopedias obtained very good marks and performed nearly equally with regard to their accuracy. Kubiszewski et al. (2011, p. 659) performed an experiment on "whether certain webpage characteristics affect academics' and students' perception of the credibility of information presented in an online article." They concluded that "compared to Encyclopedia Britannica, article information appearing in both Encyclopedia of Earth and Wikipedia is perceived as significantly less credible" (2011, p. 664). They also found that the appearance of a biased sponsor lowered credibility.
Chenb (2009) examined how information technology professionals used Wikipedia information for work-related purposes. He found that they treated Wikipedia as a ready reference for general information, but did not consider it sufficiently developed for professional use. They considered that Wikipedia needs to improve its contribution and editorial process in order to raise its quality. Arazy and Kopak (2011) developed an instrument for measuring information quality, using Wikipedia articles as a data set. They deliberately excluded expert evaluation of the quality of the articles, but rather focused on the quality of the information as evaluated by nonexpert student readers. Although their test was not meant to be a comprehensive evaluation of Wikipedia's quality, they found that the readers generally judged the sample articles to be accurate, objective, and representative, but only moderately complete; overall, the readers considered the articles to be of fairly high quality. Francke et al. (2011) conducted an ethnographic study of a class of 29 upper-secondary students to examine how, after being specifically trained on information literacy, they navigated and assessed the credibility of online sources. In addition to the Swedish, English, and German Wikipedias, the students were required to use a national Swedish encyclopedia, Greenpeace's website, and a few other sources; they were free to use other sources as well. They found that the students assessed credibility mainly from four perspectives: some valued the apparent control or authoritativeness of the sources; some the balance of viewpoints presented; some the sources' commitment to a cause or opinion viewpoint; and some valued collaboratively compiled information-most notably, Wikipedia.
Flanagin and Mitzger (2011) conducted two surveys among young people and adults to examine their use of Wikipedia and to what extent they trust its content. A small experiment was also embedded in the survey to compare the participants' credibility assessments of three online encyclopedias, namely Wikipedia, Citizendium, and Britannica. The survey's results revealed the unpreparedness of people to "fully relinquish traditional models of information provision" (p. 371). It also showed that both young users and adults preferred content generated by experts. However, young users preferred user-generated content when they were unaware that it was user-generated. This study demonstrated a slow shift in the perception of user-and expertgenerated content.
Because of the diversity of studies on readers' perceptions of Wikipedia's credibility, it is difficult to draw any kind of general conclusions from this body of research. In fact, most such articles normally treated other subjects more substantially; thus, we describe them elsewhere in this review (Calkins & Kelley, 2009;Chen, 2010;Dooley, 2010;Head & Eisenberg, 2010;Kaplan & Haenlein, 2010;Lim, 2009;Lim & Kwon, 2010;Luyt et al., 2008McGuinness et al., 2006;Page, 2010;Sundin & Francke, 2009;Zeng, Alhossaini, Ding, Fikes, & McGuinness, 2006). Nonetheless, in general, we can observe that although Wikipedia is widely read, most surveyed readers note that they are conscious that not everything they read might be accurate. We note "surveyed readers" because the enormous amount of readers might suggest that most of them in practice consider most of what they read to be true, although when explicitly asked in a survey they would emphasize their cautiousness more than their de facto acceptance of Wikipedia as a worthwhile reference source. Despite readers' expressed guardedness, they evidently consider it sufficiently credible to be a major reference source.

Software for Readership
Whereas most of the scholarly studies on Wikipedia readership examined social human factors, there were a few streams of research that adopted a computer science approach to investigate software specifically developed to help Wikipedia readers. Among these 12 studies (13%), some of these programs attempted to alert Wikipedia readers to the trustworthiness of articles, and others were designed to enhance the usefulness of external content by automatically identifying relevant Wikipedia content.
Computational estimation of trustworthiness. A number of studies developed computational methods for estimating the trustworthiness of articles, mainly to help readers assess whether articles were more or less reliable. Although the data used in these studies are usually independent of the reader and perform the computational modeling based on content and contributors, we discuss these articles here since the tools are directed to Wikipedia readers. This topic is distinct from human evaluations of articles' accuracy-we discuss those in a separate article as studies on the Reliability of Wikipedia (Mesgari et al., in press). Over the years, many studies have noted and suggested various means for rapidly estimating the quality of an article, usually through the observation of reliable proxies. We discuss these techniques generally in chronological progression. Zeng et al. (2006) developed a method to predict trustworthiness of Wikipedia articles based on the revision history of the articles, validated using featured articles. They concluded that Wikipedia is generally trustworthy and that visualizations of article trustworthiness can enable users to access the more trustworthy versions of the articles and to avoid vandalism and malicious content. They also designed and implemented a trust management layer for collaborative information repositories in general, and Wikipedia in particular . Dondio and Barrett (2007) later developed a different method to predict trustworthiness using computational trust techniques by specifically analyzing the quality of the content and the collaborative editing contexts. They validated their method by differentiating featured articles from others using the method. Cross (2006) offered a text-colorizing software as "a visual cue that enables [users] to see what assertions in an article have . . . survived the scrutiny of a large number of people, and what assertions are relatively fresh, and may not be as reliable." Korfiatis, Poulos, Bokos, and Sicilia (2006) investigated the development of quality articles in Wikipedia by using social network analysis to determine the authoritativeness of articles. They developed an approach to calculate social network measures such as centrality. They used a web crawler (before these software agents were banned on Wikipedia because of the excess server load they cause). They argued that as Wikipedia keeps growing, it will be more challenging to keep the content reliable. Hu, Lim, Sun, Lauw, and Vuong (2007) proposed three models for assessing quality of Wikipedia articles based on the interaction data between articles and their contributors. They found that simple article length often improved model performance. Similarly, Blumenstock (2008) found that the word count of an article performs surprisingly well as a predictor for article quality, at least when distinguishing between featured and random articles, with an error rate of around 96% on a corpus of 1,554 featured and 9,513 randomly selected articles. He suggested setting a cutoff at 2,000 words between the two sets. However, the best model that Hu et al. (2007) developed, called "ProbReview," superiorly predicted quality irrespective of article length. We suggest that the number of editors having an article on their watch list could possibly also make a good indicator of the article quality. 9 The Wikiganda, formerly available from www.wikiwatcher.com, used automated text analysis to detect biased edits (Chandy, 2009). The sentiment analysis technique uses a lexicon of more than 20,000 words from General Inquirer and Wiebe wordlists so that each revision can get a Propaganda Score labeled as negative, positive, or "vague" propaganda. In conjunction with the WikiTrust system and evaluated against 200 manually labeled revisions, the system showed a precision/recall performance of 52%/63%.
A related body of work includes studies on Wikipedia reputation systems, which compute the trustworthiness of Wikipedia contributors. Although the reputation of an article's authors could be considered an indirect measure of the trustworthiness of the article itself, this topic is more directly related to Wikipedia contributors than it is to readers; hence, we discuss such studies in the Reputation Systems section of a separate review dedicated to studies of participation in and contribution to Wikipedia .
Overall, the studies we have reviewed here have employed a wide variety of measures to computationally estimate article reliability for the benefit of Wikipedia readers: article revision history, longitudinal readership of portions of text, social network analysis, computational trust techniques, word counts, article-contributor interaction data, and wordlists from external corpuses. These techniques have varying degrees of value, but Wikipedia has not incorporated any of these or any other computational reliability tools into their software. Various tags (such as the ubiquitous "citation needed") signal to readers that discernment is needed in assessing the reliability of what they are reading, but there has been no attempt to implement an automatic objective estimate of article reliability. Perhaps such tools are best employed in external reader software that accesses Wikipedia articles and then presents them to readers with enhanced features, including reliability estimates.
Reading support. Encountering knowledge gaps while reading is an issue that people face daily. This problem has motivated some researchers to develop reading support tools using Wikipedia to fill these gaps. In other words, Wikipedia articles were extracted to fill the missing information resulting from knowledge gaps. The studies we discuss here do not support reading of Wikipedia articles; rather, they employ Wikipedia articles to support the reading of other texts.
Jordan and Watters (2009) designed a prototype to bring up the single most relevant Wikipedia article when a user selects part of a text in a separate software text reader. The most successful model could accurately find the best article in 70% of the cases and help readers to fill the gap in their personal knowledge using Wikipedia articles. As an application for the previous study, Jordan (2009) proposed a system to help people reading academic abstracts to be able to highlight part of them, and then a pop-up would appear with a single Wikipedia article explaining the highlighted part. The system tries to suggest the most related article based on understanding the context of the abstract and article categories.
With the increase of blogs and social networks comes the need for a support system to fill the content holes. Nadamoto, Aramaki, Abekawa, and Murakami (2010) suggested a new method to search for these content holes, defined as "the user's unawareness of information." Wikipedia articles were used to extract and present the holes in community-type content. Their proposed method differs from otherwise related information retrieval tasks in that it searches for different instead of similar information.

Commercial Aspects
Wikipedia is an open content project maintained by a not-for-profit foundation. Nonetheless, 10 studies (11%) have investigated commercial aspects of Wikipedia. These range from those that investigate reasons why Wikipedia chose to adopt a not-for-profit path early on, those that consider commercial enterprises' responses to Wikipedia's entries on them, and those that investigate the use of Wikipedia content for commercial benefit.
Some studies investigated characteristics of Wikipedia that relate to its not-for-profit direction. In his dissertation that covered numerous other themes, Gehl (2010) discussed Wikipedia's early (2002) consideration of featuring profitgarnering advertisements. However, the fork of the Spanish Wikipedia and other widespread community protest killed that idea. Wikipedia was thus firmly steered in a not-forprofit direction, for the primary interest of its users (both readers and contributors) rather than commercial enterprises such as Jim Wales's Bomis, Inc. Although Wikipedia is not itself a commercial project, Cedergren (2003) argued that its production of useful articles mirrors a commercial value chain in creating valuable resources for readers. Although not strictly a commercial perspective, Rahman (2006Rahman ( , 2008 conducts economic analyses of Wikipedia as a public good. He concluded that Wikipedia's uniqueness "as a public good, combined with free-riding and free-editing help to maintain the [large size and] reliability of Wikipedia" (Rahman, 2008, p. 96) relative to other open source systems.
Although Wikipedia is itself a not-for-profit endeavor, its contents include information about commercial enterprises. Wikipedia articles about corporate entities have high prominence on the web, and are often perceived by readers, whether justifiably or not, as a less-biased source than the company's own website. As such, some researchers argue that corporations should be interested in Wikipedia's contents relating to them. Hickerson and Thompson (2009, p. 9) considered the potential of Wikipedia as a tool for public relations, investigating how "wiki sites uphold dialogic principles and encourage dialogue." The fact that the site is open, free, and does not serve financial interests of any single party at the expense of others contributes to participants' feeling of partial ownership. This, in turn, "may encourage repeat visits to the site and an increased investment in the organisation" (2009, p. 9). Kaplan and Haenlein (2010) introduced businesses to using social media. They noted that Wikipedia is very restrictive in permitting commercial participation in its community, yet urged businesses to pay attention to it because "although not everything written on Wikipedia may actually be true, it is believed to be true by more and more Internet users" (Kaplan & Haenlein, 2010, p. 62). However, they warned that trying to gloss corporate image by getting third parties to edit Wikipedia articles is probably futile at best and, at worst, could likely backfire. DiStaso and Messner (2010) analyzed 10 companies in 2006. They found that during this period the search engine prominence of their Wikipedia articles increased, the tone changed for some companies in a negative direction, and the percentage of topics on controversial issues (e.g., "legal concerns/scandals") increased. They concluded that "the monitoring of Wikipedia in public relations should be included in all social media plans" (p. 18).
Another commercial perspective on Wikipedia hinges on the fact that its Creative Commons Attribution-ShareAlike license explicitly permits commercial reuse of its content. Some studies investigated phenomena that try to leverage or take advantage of this permission.
Two studies investigated aspects of how companies might profit directly from Wikipedia. Langlois and Elmer (2009) investigated how Wikipedia content is being used anywhere across the Internet. They found that it is mostly used for generating commercial content or for increasing traffic through search engines links. Plaza (2011) investigated how Wikipedia entries can get traffic to a tourism website in comparison with other traffic sources such as Google. She found that Wikipedia entries are effective in getting people to visit and navigate through the sample website she studied.
In related work, Rubin and Rubin (2010) hypothesized that the degree of web activity about a company correlates with the extent to which investors are generally informed about their companies. To test this, they investigated the frequency of edits of Dow Jones Industrial firms' entries on Wikipedia in relation to analysts' forecasts and recommendations, and confirmed that Wikipedia edit frequencies are indeed correlated with the accuracy of corporate analysts' forecasts.
In general, these studies on the commercial aspects of Wikipedia found that although Wikipedia itself is not commercial, it has important implications for commercial enterprises. Wikipedia's broad readership makes it a primary public source of information about companies. Although readers do not necessarily believe everything they read on Wikipedia, they generally consider it a more reliable source on a company than the biased information that the company itself might provide to the public. Thus, enterprises are advised to pay attention to their Wikipedia image and to work within the community guidelines to ensure that their information on Wikipedia is not inaccurate and that positive factual information is prominently included. Other studies have begun to explore how commercial enterprises can exploit Wikipedia's rich information base to help earn more revenue for themselves. We expect such uses of Wikipedia to increase as enterprises gain expertise in the commercial use of social media and user-generated content in general.

Research Trends for Studies of Wikipedia Readership
Our presentation of the various readership topics in Wikipedia shows a wide variety of research topics. On our WikiLit website, we have examined each scholarly article to identify various key details of interest to researchers. In fact, the data analysis on the WikiLit website is highly interactive, and researchers can analyze these data themselves with tables and graphs. 10 In this major section of our review, we present an overview of the various research trends of studies of Wikipedia readership. For both new and experienced Wikipedia researchers, we believe it is valuable to understand how past research on Wikipedia readership has been conducted. These perspectives can help researchers determine how to conduct future research, and also identify underutilized techniques that may be appropriate for future work.
We have two important notes concerning the research trends we report here. First, not all the 99 studies that we have summarized in the preceding section on Findings were parsed for their research details in our data set. Although we did parse most of them, because of the work involved we only parsed details from 91 studies, as mentioned in the Introduction. Second, in our data set of parsed papers on the WikiLit website, papers were frequently assigned to multiple topics. For example, as we explained earlier, many papers treated Reader Perceptions of Credibility along with other topics. Thus, although our data set included only 91 studies, if the counts in most tables were added up they would exceed this number. However, we publish at the bottom of each table the number of distinct studies for each criterion without duplicates.
In the rest of this section, we present detailed research trends for studies of Wikipedia Readership. In the Discussion section that follows, we highlight a few noteworthy findings from these trends as well as from the detailed summaries in prior sections of this review.

Readership Topics by Year
As can be expected, studies of Wikipedia research have increased over the years, as shown in Table 2

Readership Topics by Domain of Knowledge
Tables 3 and 4 display the domains of knowledge, that is, the fields or disciplines, of the various studies of Wikipedia readership. The most common domain has been education, with 22 studies (24%), most of them concerning Student Readership. Library science is next, with 16 studies, covering Wikipedia as a Knowledge Source for Librarians and various aspects of Student Readership. Next, there have been 15 computer science studies, mainly on software for readership. Health studies, information science, and information systems are the other three fairly common domains covered, with 11 or 12 studies of Wikipedia readership each. treated Wikipedia as the main topic of the study. Thirty studies (33%) considered Wikipedia only as one case among other websites or resources that it studied. Six studies (7%) were not explicitly focused on Wikipedia, but employed data extracted from Wikipedia to test their hypotheses or develop their tools. Table 6 displays the three different types of publication in our data set: conference papers, journal articles, and theses. Because we explicitly focused journal articles and doctoral theses, we have 82 journal articles and three theses and only six conference papers. Interestingly, two of the three theses treated commercial aspects of Wikipedia. Four of the six conference papers focused on computational estimation of trustworthiness, which is not surprising since computer science topics such as this are frequently published in conference proceedings and not in journals. Table 7 displays the source of the data that were collected in the various empirical studies. (N/A refers to nonempirical conceptual studies.) As can be expected, the most common data source for Wikipedia readership research has been various kinds of Wikipedia pages (we examine the specific kinds of pages shortly). This was particularly the case in studies of software for readership, since these were directly related to enhancing and encouraging the reading of Wikipedia articles. After Wikipedia pages, the next most common source has been survey data, which has in fact been the single most popular data collection approach for studies of student readership and for perceptions of credibility.

Readership Topics by Data Source
Comparisons of Wikipedia among other websites follows as the next most common data collection approach, particularly in studies of Wikipedia as a knowledge source and of Wikipedia's ranking and popularity, followed by interviews. Experiments and interviews have also been popular. In contrast to other kinds of Wikipedia research, only very few readership studies have employed archival records, computer usage logs, direct observations, or other kinds of documents (not including Wikipedia pages).  Table 9 displays the units of analysis of the Wikipedia data analyzed in the studies. A third of the studies (30 studies, 33%) used individual Wikipedia articles as the unit of analysis. This was particularly the case for Ranking and Popularity studies (which often assessed the popularity of individual articles) and for Software for Readership studies (which estimated the reliability of individual articles or presented individual articles as suggestions). Second was website as a unit of analysis (22 studies, 24%), when Wikipedia was compared with other websites. Next was the individual Wikipedia user as a unit of analysis (17 studies, 19%), most frequently among studies of Reader Perceptions of Credibility, where the edit history or reputation of a contributor to an article was sometimes found to affect how readers judged an article, and among Student Readership studies, which analyzed individual students as reader-users.

Readership Topics by Time Dimension of Collected Data
Table 10 displays the time dimensions of the Wikipedia data that were collected for the studies. More than half of the studies (51 of 91, 56%) collected snapshot cross-sectional data from Wikipedia, that is, data collected at just one point in time. It is notable that all but one of the Software for Readership studies were cross-sectional, as they extracted data from the live Wikipedia at a given point in time to process for their analyses. Only 17 studies collected longitudinal data, that is, data from the same Wikipedia pages on at least two distinct dates. The remaining 23 studies were either nonempirical or they collected data from sources other than Wikipedia, yet nonetheless treated Wikipedia readership substantially.

Readership Topics by Means of Wikipedia Data Extraction
Table 11 displays the approaches employed by the studies to extracting data from Wikipedia. Nearly half of the studies (42 of 91, 46%) extracted data from the live, running web version of Wikipedia. Ten studies, however, cloned the Wikipedia database, or at least parts of it, to conduct more extensive analysis. This was particularly the case for studies that analyzed complex longitudinal data and for those that used Wikipedia as sample data for testing complex algorithms. The other studies either did not use empirical data, or the means of extracting data from Wikipedia was not very relevant to their analysis approaches.   treated a language version, one treated the Czech version, five treated the German, and two the Swedish version of Wikipedia. Eleven studies were conducted such that the Wikipedia language version was irrelevant to their analysis. Although the sample size is too small to draw significant findings, we do find it interesting that studies of the German Wikipedia were predominantly interested in rankings and comparisons: two studies were about the Ranking and Popularity of Wikipedia in general, and two others specifically on its value as a Health Information Source. Table 13 displays the various research designs adopted for studying Wikipedia readership. The most common approach has been the case study (28 studies, 31%), which qualitatively considers various aspects of the use of Wikipedia. This has been the predominant approach in studying Wikipedia as a Knowledge Source for Scholars and Librarians. The next most common approach (22 studies, 24%) has been various kinds of statistical analysis, mainly involving quantitative analysis of survey data. Experiments have also been popular (11 studies, 12%), mainly with the computer science studies of Software for Readership.

Key Findings About Wikipedia Readership
Even without peer-reviewed academic research, it is obvious that Wikipedia is the world's most popular reference website. However, scholarly research has given us a  (Spoerri, 2007a(Spoerri, , 2007bWaller, 2011). More "serious" topics are also widely consulted, but it is not surprising that Wikipedia's primary attraction is in topics with the widest popular appeal. • In contrast, Wikipedia is not used as a primary news source, even though news is a very popular topic on the web (Thelwall & Stuart, 2007). Although it is most certainly used, as it has articles that quickly appear and develop in line with most current news topics (Chua et al., 2007;Lih, 2004), Wikipedia's collaborative editing by volunteers cannot match the instant, professional journalistic updates of numerous news organizations that publish on the web. • Regarding "serious" topics, Wikipedia is a widely used health information source, not only by the general public but even by health professionals (Hughes et al., 2009;Younger, 2010). However, numerous studies have cautioned that while helpful and informative, Wikipedia cannot be regarded as an authoritative health resource (Clauson et al., 2008;Laurent & Vickers, 2009;Heilman et al., 2011). • Another "serious" topic concerns Wikipedia's use for legal information, which has in fact been quite widespread (Breinholt, 2008;Peoples, 2009;Stoddard, 2009), sometimes perhaps too much. Although it is valuable for providing background information surrounding legal issues, Wikipedia itself explicitly disclaims its use as a source of legal advice, and is not suitable for such purposes. We suggest, however, that Wikipedia articles can be appropriately used for definitions, when such definitions have been considered by many readers and multiple editors have edited the article over time-these could be considered consensus definitions. • The most common research topics have been Student Read-ership and Knowledge Source for Scholars and Librarians in the education and library science domains. These topics are naturally very interesting to the professors and research librarians who have conducted much Wikipedia research. Health information studies have also been popular among professors in healthcare. • As with all research fields where survey and experimental research is common, students are a common subject for Wikipedia research. On one hand, they are a demographic of young and intelligent people, and so are highly relevant for forwardlooking topics of inquiry, such as the use of Web 2.0 resources such as Wikipedia. On the other hand, they provide a convenience sample because professors can easily persuade their students to participate in studies, either purely voluntarily or for course credit, if some learning component is incorporated into the exercise. • Contrary to popular opinion, most scholars and librarians are generally quite favorable towards Wikipedia. They sometimes not only encourage students to use it, but sometimes use it themselves in their research and publications. Even one of the most notorious public detractors of student use of Wikipedia was less categorical in his censure than the media depicted (Waters, 2007). • There is a strong academic consensus that, like it or not, Wikipedia is a dominant form in today's information landscape and so students should be proactively trained how to critically assess and productively use Wikipedia and other contemporary Internet resources-this is a very important aspect of their education in information literacy. • Previous research has stigmatized noncontributory readership as "free-riding" or "lurking," or valuable only in its potential to convert readers to contributors. However, almost 100 studies covering a rich diversity of Wikipedia readership demonstrate the value of readership in itself. Noncontributory readership is not a problem; it is a valuable aspect of the Wikipedia ecosystem.

Implications for Wikipedia Researchers
Our review is mainly targeted to scholars, both new and experienced in Wikipedia research. Wikipedia is a culturally significant and revolutionary concept that has singlehandedly considerably uplifted the quality of web-based information. Not only is it in itself an important subject for research inquiry, but as the original highly successful nonsoftware open content endeavor, it serves as a model and example for other open content projects, ranging from open educational resources to open data. Moreover, because of its open data policies, Wikipedia is uniquely rich and readily-accessible source of both quantitative and qualitative data for a very broad range of research topics. In a separate review, we identify a number of articles and numerous resources that can help newer Wikipedia researchers get started in this eclectic research platform .
In addition to these general observations, there are a few specific implications and suggestions for future research that this present review has brought to life: • We sense a contradiction in that, although Wikipedia is such a widely used resource, surveys of Wikipedia readers have generally reported that the readers are cautious about the credibility of Wikipedia information. We would think that if they were as cautious as the surveys report, they would probably not use it as much. We fear that the surveys that ask them questions related to Wikipedia's credibility naturally incite them to report that they are more cautious than they actually might be. It may be more elucidating to investigate readers' perceptions of Wikipedia's credibility by behavior-based measures rather than by self-reporting. That is, a study might observe and compare how readers use Wikipedia information alongside other information sources, perhaps in an exercise such as preparing a report for work or school. Such observations of what information they actually use might be a more accurate reflection of which sources they lend greater credibility. Such a study would have to carefully examine the actual source of the information used, not just the explicit citations, as people might not cite Wikipedia as often as they use it, being aware of its controversial status. • Researchers should perhaps give more attention to other Wikimedia Foundation projects. Although Wikipedia is the foundation's flagship project, other projects might be more closely aligned with certain disciplinary fields. For example, we were surprised that even a study of Wikipedia's representation of biological taxa (Page, 2010) made no mention of Wikispecies, the project dedicated to this purpose-indeed, we have not found any scholarly studies that examine this project. Wikinews, on the other hand, has been covered to a fair degree by journalistic studies (e.g., Bruns, 2006;Thelwall & Hellsten, 2006;Thorsen, 2008). The broad range of Wikimedia Foundation projects provides similarly promising platforms for research in various domains. 11 • Although commercial use of "the free encyclopedia" sounds oxymoronic, such a thought is in fact explicitly embedded in the open content licenses that Wikipedia has historically employed. The studies on commercial aspects of Wikipedia show that there are various approaches to monetizing Wikipedia content. However, there are many other existing approaches that have not been researched, such as the printing of books composed entirely of Wikipedia content 12 and incorporating Wikipedia pages into other websites to enhance content. This research topic could be particularly applicable to other open content projects which, although they might not have anywhere as extensive a reach as Wikipedia, might be able to provide sufficient value to both commercially sustain themselves and provide commercial opportunities to the general public. • Relatively few studies (14 of 91, 15%) analyzed longitudinal data over time. We note that none of the studies discussed in the Readers' Perceptions of Credibility section examined changes in readers' perceptions over time. Any given Wikipedia article will generally improve over time, so it would be insightful for a study to examine whether individual readers' perceptions change over time. Understandably, such a study would be considerably more difficult to conduct than the cross-sectional studies that have been conducted to date.

Implications for the Wikipedia Community
We also offer various implications of our findings to contributing members of the Wikipedia community, including the general Wikipedia-reading public.
• The Wikimedia Foundation could consider incorporating one or more of the algorithms for computational estimation of the reliability of Wikipedia articles that have been developed to help address credibility concerns so that readers could be readily sensitized. This is consistent with the open spirit of Wikipedia's Article Feedback Tool, 13 which lets readers rate articles. Moreover, articles with low estimates could be targeted for priority editing by regular contributors. There have been such efforts in the past (most notably WikiTrust), 14 but their current status is unclear. • Educators and librarians should embrace Wikipedia as a unique opportunity for training students in digital literacy. In fact, although this review is restricted to studies of Wikipedia readership, numerous studies emphasize the writing aspect of digital literacy-we cover such studies in a separate review . Students, for their part, should proactively learn how Wikipedia works, and especially study its Neutral Point of View 15 and Verifiability 16 policies as prime examples of Internet-age digital literacy. • Studies on commercial aspects of Wikipedia repeatedly echoed that organizations need to pay attention to their Wikipedia profiles (DiStaso & Messner, 2010;Hickerson & Thompson, 2009;Kaplan & Haenlein, 2010). Whether accurate or not, balanced or not, Wikipedia is often seen as a more neutral source of information than what an organization would provide on its own website. Although favorably editing their own Wikipedia articles could backfire, by studying and understanding the policies and culture of Wikipedia, organizations can ensure that they are more fairly presented on the site. • Because Wikipedia consistently tops search engine hit lists, website managers who want to attract visitors should learn how to appropriate Wikipedia content. This involves both learning how to appropriately get their site linked from Wikipedia (Plaza, 2011) and legally incorporating Wikipedia articles to enrich site content, as we suggested in the implications for researchers.

Conclusion
This systematic review has examined the scholarly research on readership of Wikipedia. We first presented summaries of scholarly studies grouped according to topics covering the ranking and popularity of Wikipedia; Wikipedia as a knowledge source for news, health information, and for scholars and librarians; student readership; reader perceptions of credibility; and commercial aspects of Wikipedia. We next discussed trends in the research details of the study, such as years of publication, knowledge domains covered, research design, and so on. We ended with a discussion of some of the most striking findings, and implications for researchers and for the Wikipedia community and readers.
Given Wikipedia's robust framework and wide popularity, it will surely continue to be a dominant force in web reference for many years to come. The scholarly research has found that Wikipedia is popular not only for lighter topics such as entertainment, but also for more serious topics such as health information and legal background. Scholars, librarians, and students are common users of Wikipedia, and it provides a unique opportunity for educating students in digital literacy. Wikipedia is an important and accessible subject for research in a wide variety of domains. Although a wide variety of subjects have already been studied, we believe that more nuanced studies of how readers assess Wikipedia's credibility could be conducted. We also feel that, despite a fair number of studies in that direction, there is yet much to be explored concerning commercial aspects of Wikipedia. By better understanding the nature of Wikipedia readership and its potential applications, such studies could help extend the reach of this valuable resource even further.