Citation Metrics
There are numerous approaches to talking about scholarly impact. Now that we've explored The Metrics Toolkit, let's narrow our focus to indexes of scholarly research and how you can use them to analyze citation data and calculate quantitative measures of research impact.
In particular, let's look at the h-index.
Created in 2005 by physicist Jorge Hirsch, the h-index is a commonly used metric that is a measure of both the productivity and the impact of an individual author. A scholar has an index of h when they have published h papers, each of which has been cited at least h times. (Sugimoto pp. 100-101, 2018).
BONUS Challenge: Find your h-index!
Follow the instructions below to locate an h-index for the same author in Web of Science and Google Scholar. If your publications are indexed in both places, we recommend that you search for your own name. If not, search for a scholar whose work you have used in your research. Here's how to do it:
Web of Science
- Open Web of Science (linked above).
- In the search box, click "Author" from the dropdown menu.
- Perform an author search, then click "Citation Report" on the top right of the results page.
Google Scholar
- Open Google Scholar (linked above).
- If you are searching for yourself, you can simply navigate directly to the Google Scholar profile you created this week. Your h-index will appear in the box on the right-hand side of the screen.
- If you are searching for a different author, go to the main search box, enter the author's name or search by article title. In the search results page, click on the author's name to view their Google Scholar profile. Note: your chosen author will need to have a public Google Scholar profile in order for you to view their Google Scholar h-index.
Reflect on what you find:
- Does the h-index stay the same or vary across these databases? If the score changes, can you figure out why?
- What strengths do you see in the h-index as a measure of productivity and impact? What limitations do you see?
- If asked to provide an h-index as part of an evaluation process, how would you proceed?
Key Takeaways:
-
The h-index always depends upon the data source from which it was calculated. When reporting an h-index, you will always want to indicate the data source.
-
The Google Scholar h-index will often be higher than the h-index from other sources. This is because Google Scholar is more inclusive than Web of Science, indexing many more types of material than peer-reviewed research articles.
-
The h-index inherently favors scholars with longer careers, who have had the time both to publish more work and to accrue more citations.
-
The h-index will not adequately represent the work of scholars when some of their publications are not indexed in the data source being used.
What next?
- There are other citation metrics that you may have heard of or that may be of interest to you, your department, or your discipline, like the Journal Impact Factor.
Image from SlideShare
The Journal Impact Factor is the amount of times articles from a specific journal are cited, divided by the number of citable articles in that journal over a 2-year period. Find Impact Factors for a range of journal titles in Journal Citation Reports (library subscription).
Another example is SciMago's Journal Rank (freely available online), which aims to measure the prestige of a journal. See a detailed description of how SJR works. NOTE: The same takeaways for the h-index apply to these measures.
- Explore additional types of metrics beyond the citation alone in the Altmetrics page on this guide.
- For a quick, at-a-glance reference, this poster from Elsevier Library Connect provides a user-friendly overview of key research impact measures.
Learn more:
- Coleman, A. (2007). Assessing the value of a journal beyond the impact factor. Journal of the American Society for Information Science and Technology, 58(8), 1148-1161.
- Durrani, J. (July 2021). Reliability of researcher metric the h-index is in decline. Chemistry World.
- Greenwood, D. C. (2007). Reliability of journal impact factor rankings. BMC Medical Research Methodology, 7(48) [6 pages].
- Jacso, P. (2008). Testing the calculation of a realistic h-index in Google Scholar, Scopus, and Web of Science for F. W. Lancaster. Library Trends, 56(4), 784-815.
- San Francisco Declaration on Research Assessment. (2012). This document emphasizes the need to improve the ways in which researchers and the outputs of scholarly research are evaluated, and provides recommendations that are nuanced and which account for the quality and impact of diverse scholarly outputs.
- Sugimoto, C.R. & Lariviere, V. (2018). Measuring research: What everyone needs to know. Oxford: Oxford University Press (available in print in Fogler Library). This practical, accessible handbook provides valuable history and critical context for some of the major research impact measures and data sources (plus pros and cons of using various metrics to define scholarly impact). Highly recommended for those who want to dig deeper!