International Journal of Community Medicine and Public Health

Pathak VK et al. Int J Community Med Public Health. 2019 Dec;6(12):5368-5371

http://www.ijcmph.com

pISSN 2394-6032 | eISSN 2394-6040

Letter to the Editor

DOI: http://dx.doi.org/10.18203/2394-6040.ijcmph20195501

Understanding the metrics used to assess the quality of journals

Vineet Kumar Pathak*, Mohan Kumar M.

Department of Community and Family Medicine, All India Institute of Medical Sciences, Raipur, Chhattisgarh, India

*Correspondence to

Dr. Vineet Kumar Pathak,

E-mail: pathakvineet2089@gmail.com

Sir,

Various databases offer access to thousands of academic journals. A miscellaneous picture with respect to quality, scientific influence and prestige is presented by umpteen numbers of journals present. Authors wishing to publish their research aim to publish in journals with the highest ratings and are ultimately in no man’s land.1

Traditionally, measures for journal ranking or evaluations have been provided simply through institutional lists established by academic leaders or through committee votes. An approach of this kind is highly politicized and does not reflect actual prestige and quality. This also causes the problem of highly disparate evaluations across institutions.2,3

Though quality and impact of the journal is usually apparent through how widely it is read, how often it is cited, and its perception in the community, several metrics have been proposed to assess the quality of the increasing quantity of journals.4

Most indices are based on citation counts, though they sometimes combine with other indicators such as the number of citable documents or the average number of citations per article. These bibliometric should replace expert assessment as they are complete, a less expensive, and a more efficient.2

For an author it is the need of the hour to consider the various metrics, its advantages and disadvantages before deciding on the quality of journals.5

Impact factor

A journal’s impact factor is defined as the average number of times articles from the journal, which have been published in the past two years, have been cited in the present year by journals indexed by Thomson ISI.6

Example

A= The number of times articles published in 2008 and 2009 were cited by indexed journals during 2010; B= the total number of "citable items" published in 2008 and 2009. A/B= Impact factor for the year 2010.

If a journal recorded an impact factor of 1.0 in 2018 that means articles published in 2016 or 2017 have been cited, on average, one time in 2018.

Limitations

  • • Impact factor is derived from journals indexed in Web of science and so no other journals can have an impact factor.5,6
  • • Since the impact factor only looks at citations in the current year to articles in the previous two years, it only works well for disciplines in which rapid citation is the standard.7
  • • It doesn’t take into account disciplinary differences in expected numbers of citations.7
  • • Impact factor cannot be computed for those journals that are <2 years old.5

Five year impact factor

A modification of impact factor is which articles of past five years are considered in place of two years.8

H-index

The H-index is a measure of both quantity (number of publications (Np)) and quality (number of citations for each publication (Nc)). It looks at the number of highly impactful publications a journal has published.9,10 The higher the number of cited publications, the higher the H-index. H-index can be used for impact of authors and journals.

It is defined by how many h of a researcher’s publications (Np) have at least h citations each. This means that if you have one publication with at least one citation, your H-index is 1, if you have two publications with at least two citations each, your H-index would be 2, and so on.11

Example

A journal will have an H-index value of 20, if 20 of its articles had been cited at least 20 times each. An H-index of 10 would indicate 10 articles that had each been cited at least 10 times.

Advantages

  • • The H-index metric is not limited to a fixed time period. The citation window can be set at whatever time frame is most appropriate for a given discipline.12,13
  • • The H-index is not based upon the mean, it attenuates the effect of highly cited articles on computations of journal quality. The H-index emphasizes sustained and durable performance, rather than a few one hit wonders. As such, it more accurately reflects the unit of analysis, which is journals, rather than individual articles.12,14
  • • It is used by Google Scholar which provides much wider coverage of the social science literature, relative to Thomson ISI.15,16

G-index

Egghe in his paper “Theory and practice of the G-index” aims to improve on the H-index by giving more weightage to highly-cited articles. It can also be used for both authors and journals.10,11

G-index is calculated this way “[Given a set of articles] ranked in decreasing order of the number of citations that they received, the G-Index is the (unique) largest number such that the top g articles received (together) at least g2 citations”.11,17

Example

A researcher with 20 papers, 15 of which have no citations with the remaining five having respectively 350, 35, 10, 4 and 1 citations would have a g-index of 20. H-index will be 4.

Advantages of the G-index

  • • Accounts for the performance of author's top articles.11
  • • Helps to make more apparent the difference between authors' respective impacts. The inflated values of the G-index help to give credit to lowly-cited or non-cited papers while giving credit for highly-cited papers.11

i10/i20 index

These indices are the newest in the line of journal metrics and were introduced by Google Scholar in 2011. It is a simple and straightforward measure found by tallying a journal’s total number of published papers with at least 10 or 20 citations respectively.18

CiteScore

CiteScore calculates the average number of citations received in a calendar year by all items published in that journal in the preceding three years (3-year citation window). The calendar year to which a serial title's issues are assigned is determined by their cover dates, and not the dates that the serial issues were made available online. It is a part of Scopus basket of Journal Metrics (from Elsevier). It is calculated from the Scopus journal list, which is much larger than the Web of science list and includes more journals in social sciences and humanities.19,20

Source normalized impact per paper

Source normalized impact per paper (SNIP) is also a part of Scopus basket of Journal Metrics measures citation impact by weighting citations based on the total number of citations in a subject field. The impact of a single citation is given a higher value in subject areas where citations are less likely, and vice versa. This is extremely useful, as even high-quality journals in arts and humanities typically have much lower citation counts than journals in science and medicine.21-23

Scimago journal rank

It is based on the concept of a transfer of prestige between journals via their citation links. Drawing on a similar approach to the Google PageRank algorithm which assumes that important websites are linked to from other important websites. Scimago journal rank (SJR) weights each incoming citation to a journal by the SJR of the citing journal, with a citation from a high-SJR source counting for more than a citation from a low-SJR source. Like CiteScore, SJR accounts for journal size by averaging across recent publications and is calculated annually. SJR is also powered by Scopus data and is freely available alongside CiteScore.24-26

Eigenfactor

Although the calculation is complicated, a journal Eigenfactor is basically a measure of how many people read a journal and think its contents are important. Since this cannot be directly calculated, it is measured by counting the total number of citations a journal receives over a five-year period. Note that Eigenfactor measures the total number of citations and eliminates the problem of self-citation. Eigenfactor is meant to measure the importance of a journal within the scientific community and rewards large journals that publish a variety of topics.27-29

Example

Journal A, which publishes 1000 articles a year, will have twice the Eigenfactor of journal B, which puts out 500 articles annually, if each article is cited the same number of times. It’s no surprise that the journal Nature, a large journal which publishes on pretty much everything in science, has the highest Eigenfactor. But this is true only because its contents are considered valuable and are much read and cited.

To conclude, no single metric is perfect to measure the quality and impact of journals. It is again the author who takes the side of a metric he/she considers the best.

Though these metrics rank journals in terms of their impact, let science have the ultimate benefit.

REFERENCES

  1. 1. Cornillier F, Charles V. Measuring the attractiveness of academic journals: a direct influence aggregation model. Oper Res Lett. 2015;43(2):172-6.
  2. 2. Lowry PB, Moody GD, Gaskin J, Galletta D, Humphreys S, Barlow JB, et al. Evaluating Journal Quality and the Association for Information Systems Senior Scholars’ Journal Basket via Bibliometric Measures: Do Expert Journal Assessments Add Value?. Social Sci Res Network; 2013: 2186798.
  3. 3. Lowry PB, Romans D, Curtis AM. Global journal prestige and supporting disciplines: a scientometric study of information systems journals. Social Sci Res Network. 2005: 666145.
  4. 4. Bradshaw CJA, Brook BW. How to rank journals. Dorta-González P, editor. Plos One. 2016;11(3):e0149852.
  5. 5. Shrivastava S, Shrivastava P, Ramasamy J. Assessment of the quality of a journal: Scientific approach. Int J Adv Med Health Res. 2015;2(1):61.
  6. 6. The Clarivate Analytics Impact Factor. Web of Science Group. Available at: https://clarivate.com/webofsciencegroup/essays/impact-factor/. Accessed on 6 September 2019.
  7. 7. Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ. 1997;314(7079):497.
  8. 8. Journal Citation Reports: 5-Year Impact Factors. Available at: https://support.clarivate.com/ScientificandAcademicResearch/s/article/Journal-Citation-Reports-5-Year-Impact-Factors?language=en_US. Accessed on 6 September 2019.
  9. 9. Izet M. H-index and how to improve it? Donald Sch J Ultrasound Obstet Gynecol. 2016;10(1):83-9.
  10. 10. AL-Fatlawi AA. Development G-Index and H-Index : Dgh-Index. Available at: https://pdfs.Semanticscholar.org/5eff/abdfcba850aa4f1223f3eb5bc8de23054b01.pdf. Accessed on 6 September 2019.
  11. 11. An improvement of the H-index: the g-index - Semantic Scholar. Available at: https://www.semanticscholar.org/paper/An-improvement-of-the-h-index%3A-the-g-index-Egghe/bdca34b16ccefe61901c167402b9ff73e0c6629e. Accessed on 6 September 2019.
  12. 12. Hodge DR, Lacasse JR. Evaluating journal quality: is the h-index a better measure than impact factors?. Res Soc Work Pract. 2011;21(2):222-30.
  13. 13. Costas R, Bordons M. The h-index: advantages, limitations and its relation with other bibliometric indicators at the micro level. J Informetr. 2007;1(3):193-203.
  14. 14. Bornmann L, Daniel HD. The state of h index research. Is the h index the ideal way to measure research performance? EMBO Rep. 2009;10(1):2-6.
  15. 15. Gusenbauer M. Google Scholar to overshadow them all? Comparing the sizes of 12 academic search engines and bibliographic databases. Scientometrics. 2019;118(1):177-214.
  16. 16. Google Scholar Metrics Help. Available at: https://scholar.google.com/intl/en/scholar/metrics.html. Accessed on 6 September 2019.
  17. 17. Eldermire E. LibGuides: Measuring your research impact: G-Index. Available at: //guides.library. cornell.edu/c.php?g=32272&p=203392. Accessed on 6 September 2019.
  18. 18. Noruzi A. Impact Factor, h-index, i10-index and i20-index of Webology. Available at: http://www.webology.org/2016/v13n1/editorial21.pdf. [Last accessed on 2019 Sept 6]
  19. 19. Elsevier. CiteScore: A new metric to help you track journal performance and make decisions. Editors’ Update. Available at: https://www.elsevier.com/editors-update/story/journal-metrics/citescore-a-new-metric-to-help-you-choose-the-right-journal. Accessed on 6 September 2019.
  20. 20. Elsevier releases 2018 CiteScoreTM values. Available at: https://www.elsevier.com/about/press-releases/science-and-technology/elsevier-releases-2018-citescore-values. Accessed on 6 September 2019.
  21. 21. Measuring a journals impact. Available at: https://www.elsevier.com/authors/journal-authors/measuring-a-journals-impact. Accessed on 6 September 2019.
  22. 22. Elsevier Journal Metrics Visualization. Helping Authors. Visualizing Key Metrics. Delivering Journal Insights. Available at: https://journalinsights.elsevier.com/journals/0969-806X/snip. Accessed on 6 September 2019.
  23. 23. Journal Metrics in Scopus: Source Normalized Impact per Paper (SNIP). Elsevier Scopus Blog. Available at: https://blog.scopus.com/posts/journal-metrics-in-scopus-source-normalized-impact-per-paper-snip. Accessed on 6 September 2019.
  24. 24. Falagas ME, Kouranos VD, Arencibia-Jorge R, Karageorgopoulos DE. Comparison of SCImago journal rank indicator with journal impact factor. FASEB J. 2008;22(8):2623-8.
  25. 25. Journal Metrics in Scopus: SCImago Journal Rank (SJR), Elsevier Scopus Blog. Available at: https://blog.scopus.com/posts/journal-metrics-in-scopus-scimago-journal-rank-sjr. Accessed on 6 September 2019.
  26. 26. Guerrero-Bote VP, Moya-Anegón F. A further step forward in measuring journals’ scientific prestige: The SJR2 indicator. J Informetr. 2012;6(4):674-88.
  27. 27. Tchangalova N. Research guides: bibliometrics and altmetrics: measuring the impact of knowledge: Eigenfactor. Available at: //lib.guides.umd.edu/bibliometrics/eigenfactor. Accessed on 6 September 2019.
  28. 28. Eldermire E. LibGuides: Measuring your research impact: Eigenfactor and Article Influence. Available at: //guides.library.cornell.edu/c.php?g=32272&p=203396. Accessed on 6 September 2019.
  29. 29. A Closer Look at the EigenfactorTM Metrics. Web of Science Group. 2017. Available at: https://clarivate.com/webofsciencegroup/blog/closer-look-eigenfactor-metrics/. Accessed on 6 September 2019.