Skip to main content
SpringerLink
Log in

A new look at evidence of scholarly citation in citation indexes and from web sources

  • Special-Session Papers Presented at the 9th International Conference on Science and Technology Indicators
  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

A sample of 1,483 publications, representative of the scholarly production of LIS faculty, was searched in Web of Science (WoS), Google, and Google Scholar. The median number of citations found through WoS was zero for all types of publications except book chapters; the median for Google Scholar ranged from 1 for print/subscription journal articles to 3 for books and book chapters. For Google the median number of citations ranged from 9 for conference papers to 41 for books. A sample of the web citations was examined and classified as representing intellectual or non-intellectual impact. Almost 92% of the citations identified through Google Scholar represented intellectual impact — primarily citations from journal articles. Bibliographic services (non-intellectual impact) were the largest single contributor of citations identified through Google. Open access journal articles attracted more web citations but the citations to print/subscription journal articles more often represented intellectual impact. In spite of problems with Google Scholar, it has the potential to provide useful data for research evaluation, especially in a field where rapid and fine-grained analysis is desirable.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Antelman, K. (2004), Do open-access articles have a greater research impact? College & Research Libraries, 65(5): 372–382.

    Google Scholar 

  • Bakkalbasei, N., Bauer, K., Glover, J., Wang, L. (2006), Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomedical Digital Libraries, 3. Retrieved August 28, 2006, from http://www.bio-diglib.com/content/pdf/1742-5581-3-7.pdf

  • Bakkalbasi, N., Goodman, D. (2004), Do science researchers use books? Paper presented at the XXIV Annual Charleston Conference: Issues in Book and Serial Acquisition, Charleston, South Carolina. PowerPoint slides retrieved June 23, 2006, from http://dlist.sir.arizona.edu/1106

  • Bauer K., Bakkalbasi, N. (2005), An examination of citation counts in a new scholarly communication environment. D-Lib Magazine, 11(9). Retrieved February 15, 2006, from http://www.dlib.org/dlib/september05/bauer/09bauer.html

  • Belew, R. K. (2005), Scientific Impact Quantity and Quality: Analysis of Two Sources of Bibliographic Data. Retrieved September 6, 2006, from arXiv:cs.IR/0504036v1

  • Broadus, R. M. (1971), The literature of the social sciences: A survey of citation studies. International Social Science Journal, 23(2): 236–243.

    Google Scholar 

  • Cronin, B. (2005), A hundred million acts of whimsy. Current Science, 89(9): 1505–1509. Retrieved June 12, 2006, from http://www.ias.ac.in/currsci/nov102005/1505.pdf

    Google Scholar 

  • Davis, P. M. (2006), Do Open-Access articles really have a greater research impact? College & Research Libraries, 67(2): 103–104. Retrieved August 25, 2006, from http://hdl.handle.net/1813/2881

    Google Scholar 

  • Devin, R. B., Kellogg, M. (1990), The serial/monograph ratio in research libraries: Budgeting in light of citation studies. College & Research Libraries, 51(1): 46–54.

    Google Scholar 

  • Harnad, S. (2006), Let 1000 RAE metric flowers bloom: Avoid Matthew effect as self-fulfilling prophecy [posting to the ASIST SIG/Metrics listserv]. Retrieved June 23, 2006, from http://listserv.utk/edu/cgi-bin/wa?A2=ind0606&L=sigmetrics&D=1&O=D&F=&S=&P=8071

  • Jacsó, P. (2005a), As we may search — comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases. Current Science, 89(9): 1537–1547. Retrieved January 4, 2006, from http://www.ias.ac.in/currsci/nov102005/1537.pdf

    Google Scholar 

  • Jacsó, P. (2005b), Comparison and analysis of the citedness scores in Web of Science and Google Scholar. Digital Libraries: Implementing Strategies and Sharing Experiences, Proceedings of the International Conference on Asian Digital Libraries (Lecture Notes in Computer Science, 3815), 360–369. Retrieved June 12, 2006, from http://projects.ics.hawaii.edu/:_jacso/PDFs/jacso-comparison-analysis-of-citedness.pdf

  • Jacsó, P. (2005c), Google Scholar and The Scientist. Retrieved June 12, 2006, from http://www2.hawaii.edu/:_jacso/extra/gs

  • Kousha, K., Thelwall, M. (2006), Motivations for URL citations to open access library and information science articles, Scientometrics, 68: 501–517.

    Article  Google Scholar 

  • Kousha, K., Thelwall, M. (2007), Google Scholar citations and Google Web/URL citations: A multidiscipline exploratory analysis. Journal of the American Society for Information Science and Technology, 58(7): 1055–1065.

    Article  Google Scholar 

  • Kurtz, M. J. (2004), Restrictive Access Policies Cut Readership of Electronic Research Journal Articles by a Factor of Two. Cambridge, MA: Harvard-Smithsonian Center for Astrophysics. Retrieved June 12, 2006, from http://opcit.eprints.org/feb19oa/kurtz.pdf

    Google Scholar 

  • Lawrence, S. (2001), Online or invisible? Nature, 411(6837): 521.

    Article  Google Scholar 

  • Meho, L. I. (2006), The End of Monopoly: Citation Analysis beyond Web of Science. Submitted for publication.

  • Meho, L. I., Spurgin, K. M. (2005), Ranking the research productivity of library and information science faculty and schools: An evaluation of data sources and research methods. Journal of the American Society for Information Science and Technology, 56(12): 1314–1331.

    Article  Google Scholar 

  • Meho, L. I., Yang, K. (2007), Impact of data sources on citation counts and rankings of LIS faculty: Web of Science, Scopus, and Google Scholar. Journal of the American Society for Information Science and Technology, 58(13): 2105–2125.

    Article  Google Scholar 

  • Merton, R. K. (1968), The Matthew Effect in science. Science, 159(3810): 56–63.

    Article  Google Scholar 

  • Noruzi, A. (2005), Google Scholar: The new generation of citation indexes. Libri, 55(4): 170–180.

    Article  Google Scholar 

  • Neuhaus, C., Neuhaus, E., Asher, A., Wrede, C. (2006), The depth and breadth of Google Scholar: An empirical study. Portal: Libraries and the Academy, 6(2): 127–141.

    Article  Google Scholar 

  • Pauly, D., Stergiou, K. I. (2005), Equivalence of results from two citation analyses: Thomson ISI’s citation index and Google’s scholar service. Ethics in Science and Environmental Politics, 2005: 33–35.

  • Roth, D. L. (2005), The emergence of competitors to the Science Citation Index and the Web of Science. Current Science, 89(9): 1531–1536.

    Google Scholar 

  • Schaffer, T. (2004), Psychology citations revisited: Behavioral research in the age of electronic resources. The Journal of Academic Librarianship, 30(5): 354–360.

    Article  Google Scholar 

  • Shadbolt, N., Brody, T., Carr, L., Harnad, S. (2006), The open research web: A preview of the optimal and the inevitable. In N. Jacobs (Ed.), Open Access: Key Strategic, Technical and Economic Aspects. Retrieved June 23, 2006, from http://eprints.ecs.soton.ac.uk/12453

  • Simboli, B. (2006), Professional Associations and Impact Factors [posting on the CHEMINF-L listserv]. Retrieved June 23, 2006, from https://listserv.indiana.edu/cgi-bin/wa-iub.exe?A2=ind0606&L=chminf-1&T=0&P=2608

  • Van Impe, S., Rousseau, R. (2006), Web-to-print citations and the humanities. Information: Wissenschaft und Praxis, 57(8): 422–426.

    Google Scholar 

  • Vaughan, L., Shaw, D. (2003), Bibliographic and web citations: What is the difference? Journal of the American Society for Information Science and Technology, 54(14): 1313–1322.

    Article  Google Scholar 

  • Vaughan, L., Shaw, D. (2005), Web citation data for impact assessment: A comparison of four science disciplines. Journal of the American Society for Information Science and Technology, 56(10): 1075–1087.

    Article  Google Scholar 

  • Zhao, D. Z. (2005), Challenges of scholarly publications on the web to the evaluation of science: A comparison of author visibility on the web and in print journals. Information Processing & Management, 41(6): 1403–1418.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Information and Media Studies, University of Western Ontario, London, Ontario, Canada, N6A 5B7

    Liwen Vaughan

  2. School of Library and Information Science, Indiana University, Bloomington, USA

    Debora Shaw

Authors
  1. Liwen Vaughan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Debora Shaw
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liwen Vaughan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Vaughan, L., Shaw, D. A new look at evidence of scholarly citation in citation indexes and from web sources. Scientometrics 74, 317–330 (2008). https://doi.org/10.1007/s11192-008-0220-2

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-008-0220-2

Keywords

Search

Navigation