Publication:
Pros and cons of the impact factor in a rapidly changing digital world

Loading...
Thumbnail Image
Official URL
Full text at PDC
Publication Date
2018
Advisors (or tutors)
Editors
Journal Title
Journal ISSN
Volume Title
Publisher
Facultad de Ciencias Económicas y Empresariales. Instituto Complutense de Análisis Económico (ICAE)
Citations
Google Scholar
Research Projects
Organizational Units
Journal Issue
Abstract
The purpose of the paper is to present arguments for and against the use of the Impact Factor (IF) in a rapidly changing digital world. The paper discusses the calculation of IF, as well as the pros and cons of IF. Editorial policies that affect IF are examined, and the merits of open access online publishing are presented. Scientific quality and the IF dilemma are analysed, and alternative measures of impact and quality are evaluated. The San Francisco declaration on research assessment is also discussed.
Description
Keywords
Citation
Abraham, P. (2000), Duplicate and salami publications, Journal of Postgraduate Medicine, 46, 67-69. Adler, R., J. Ewing and P. Taylor (2008), Joint committee on quantitative assessment of research: Citation statistics. [A report from the International Mathematical Union (IMU) in cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the Institute of Mathematical Statistics (IMS).] Australian Mathematical Society Gazette, 35(3), 166-188. http://www.austms.org.au/Gazette/2008/Jul08/Gazette35(3)Web.pdf#page=24. Adler, R., J. Ewing and P. Taylor (2009), Citation statistics, Statistical Science, 24(1), 1-14. Agrawal, A. (2005), Corruption of journal impact factors, Trends in Ecology and Evolution, 20(4), 157. Antelman, K. (2004), Do open-access articles have a greater research impact?, College & Research Libraries News, 65(5), 372-382. Archambault E., and V. Lariviere (2009), History of the journal impact factor: Contingencies and consequences, Scientometrics, 79, 635-649. Anauati, M.V., S. Galiani, and R.H. Gálvez (2014), Quantifying the life cycle of scholarly articles across fields of economic research, p. 23 (12 November 2014), Social Science Research Network (SSRN), SSRN-id2542612. Available at SSRN http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2523078. Arnold, D.N., and K.K. Fowler (2011), Nefarious numbers, Notices of the American Mathematical Society, 58(3), 434-437. Bergstrom, C.T. (2007), Eigenfactor: Measuring the value and prestige of scholarly journals, College & Research Libraries, 68(5), 314-316. Berquist T.H. (2008), Duplicate publishing or journal publication ethics 101, American Journal of Roentgenology, 191, 311-312. Bollen, J., M.A. Rodriguez, and H. Van de Sompel (2006), Journal status, Scientometrics, 69, 669-687. Bornmann, L., and H.D. Daniel (2008), What do citation counts measure? A review of studies on citing behavior, Journal of Documentation, 64(1), 45-80. Bornmann L., I. Nast, and H.D. Daniel (2008), Do editors and referees look for signs of scientific misconduct when reviewing manuscripts? A quantitative content analysis of studies that examined review criteria and reasons for accepting and rejecting manuscripts for publication, Scientometrics, 77, 415-432. Bornmann, L. (2013), A better alternative to the h index, Journal of Informetrics, 7(1), 100. Bornmann, L., and W. Marx (2013), How good is research really? Measuring the citation impact of publications with percentiles increases correct assessments and fair comparisons, EMBO Reports, 14(3), 226-230. Brenner, S. (1995), Loose end, Current Biology, 5(5), 568. Brodman, E. (1944), Choosing physiology journals, Bulletin of the Medical Library Association, 32 (4), 479-483. Butakov, S., and V. Scherbinin (2009), The toolbox for local and global plagiarism detection, Computers & Education, 52, 781-788. Bruce, A. (2013), Impact factor distortions, Science, 340 (6134), 787. Butler, D. (2008), Free journal-ranking tool enters citation market, Nature, 451, 6. Callaham, M., and C. McCulloch (2011), Longitudinal trends in the performance of scientific peer reviewers, Annals of Emergency Medicine, 57, 141-148. Chang, C.-L., E. Maasoumi and M. McAleer (2016), Robust ranking of journal quality: An application to economics, Econometric Reviews, 35(1), 50-97. Chang, C.-L., and M. McAleer (2015), Bibliometric rankings of journals based on the Thomson Reuters citations database, Journal of Reviews on Global Economics, 4, 120-125. Chang, C.-L., M. McAleer and L. Oxley (2011), Great expectatrics: Great papers, great journals, great econometrics, Econometric Reviews, 30(6), 583-619. Chang, C.-L., M. McAleer and L. Oxley (2013), Coercive journal self citations, impact factor, journal influence and article influence, Mathematics and Computers in Simulation, 93, 190-197. Cohen, J.B. (1938), The misuse of statistics, Journal of the American Statistical Association, 33(204), 657-674. CWTS (2015), CWTS Journal Indicators, Centre for Science and Technology Studies, Leiden University, The Netherlands. http://www.journalindicators.com/methodology. DFG (2010), Deutsche Forschungsgemeinschaft (DFG), “Quality not quantity” - DFG Adopts Rules to Counter the Flood of Publications in Research Press Release No. 7/23, February 2010. http://dfg.de/en/service/press/press_releases/2010/pressemitteilung_nr_07/index.html. DORA (2013), San Francisco Declaration on Research Assesment (DORA). Available at http://www.embo.org/news/research-news/research-news-2013/san-franciscodeclaration-on-research-assessment [Accessed 17 June 2015]. Dost, F.N. (2008), Peer review at a crossroads - A case study, Environmental Science and Pollution Research, 15(6), 443-447. EASE (2007), EASE statement on inappropriate use of impact factors, European Science Editing, 33(4), 99-100. Editorial (2005), Not-so-deep impact. Research assessment rests too heavily on the inflated status of the impact factor, Nature, 435(7045), 1003–1004. Editorial (2009), Editorial, Journal of Biological Physics and Chemistry, 9(4), 139-140. Errami, M., and H. Garner (2008), A tale of two citations, Nature, 451, 397-399. Eysenbach, G. (2011), Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact, Journal of Medical Internet Research, 13(4): e123. Fassoulaki, A., K. Papilas, A. Paraskeva, and K. Patris (2002), Impact factor bias and proposed adjustments for its determination, Acta Anaesthesiologica Scandinavica, 46(7), 902-905. Ganley, E. (2013), A lot can happen in a decade, PLoS Biol, 11(10), e1001689. https://doi.org/10.1371/journal.pbio.1001689. Garfield, E. (1955), Citation indexes to science: A new dimension in documentation through association of ideas, Science, 122(3159), 108-111. Garfield, E. (1972), Citation analysis as a tool in journal evaluation, Science, 178(4060), 471-479. Garfield, E. (2006), The history and meaning of the journal impact factor, Journal of the American Medical Association, 295(1), 90-93. Garfield, E. (1998a), Long-term vs. short-term journal impact: Does it matter?, The Scientist, 12(3), 10-12. Garfield, E. (1998b), Long-term vs. short-term journal impact (part II), The Scientist 12(14), 12-13. Garfield, E., and I.H. Sher (1963), New factors in the evaluation of scientific literature through citation indexing, American Documentation, 14(3), 195-201. Gross, P.L.K., and E.M. Gross (1927), College libraries and chemical education, Science, New Series, 66(1713), 385-389. Gwilym, S.E., M.C. Swan, and H. Giele (2004), One in 13 ‘original’ articles in the Journal of Bone and Joint Surgery are duplicate or fragmented publications, Journal of Bone and Joint Surgery, 86-B(5), 743-745. Habibzadeh, F., and M.A. Winker (2009), Duplicate publication and plagiarism: Causes and cures, Notfall Rettungsmed, 12, 415-418. Hames, I. (2007), Peer Review and Manuscript Management in Scientific Journals: Guidelines for Good Practice, Wiley. Hansen, H.B., and J.H. Henriksen (1997), How well does journal “impact” work in the assessment of papers on clinical physiology and nuclear medicine?, Clinical Physiology, 17(4), 409-418. Hernan, M.A. (2009), Impact factor: A call to reason, Epidemiology, 20, 317-318. Hopewell, S., G. Collins, Ly-Mee Yu, J. Cook, I. Boutron, L. Shamseer, M. Shanyinde, R. Wharton, and D.G. Altman (2014), The reporting of randomized trials: Changes after peer review (CAPRI), BMJ, 349, g4145. ICSU (2008), Publication practices and indices and the role of peer review in research assessment (July 2008), International Council for Science. http://www.icsu.org/publications/cfrs-statements/publication-practices-peer-review/statementpublication-practices-and-indices-and-the-role-of-peer-review-in-research-assessment-july2008. ISI (2002), Journal self-citation in the journal citation reports - Science Edition (2002), The Institute for Scientific Information (ISI), Thomson Reuters. http://wokinfo.com/essays/journal-self-citation-jcr/. Jeacle, I., and C. Carter (2011), In TripAdvisor we trust: Rankings, calculative regimes and abstract systems, Accounting, Organizations and Society, 36(4-5), 293-309. Johnson, C. (2006), Repetitive, duplicate, and redundant publications: A review for authors and readers, Journal of Manipulative and Physiological Therapeutics, 29(7), 505-509. Kodrzycki, Y.K., and P.K. Yu (2006), New approaches to ranking economics journals, B.E. Journal of Economic Analysis & Policy, 5(1). doi:10.2202/1538-0645.1520. Knothe, G. (2006), Comparative citation analysis of duplicate or highly related publications, Journal of the American Society for Information Science and Technology, 57(13), 1830-1839. Kurtz, M.J., G. Eichhorn, A. Accomazzi, C. Grant, M. Demleither, E. Henneken, S.S. Murray, N. Martimbeau, and B. Elwell (2005), The effect of use and access on citations, Information Processing & Management, 41(6), 1395-1402. Laband, D.N. (1990), Is there value-added from the review process in economics? Preliminary evidence from authors, Quarterly Journal of Economics, 105, 341-352. Lander, E.S., L.M. Linton, B. Birren, C. Nusbaum, M.C. Zody, J. Baldwin, K. Devon, K. Dewar, M. Doyle, W. FitzHugh, R. Funke, D. Gage, K. Harris, A. Heaford, J. Howland, et al. (2001), Initial sequencing and analysis of the human genome, Nature, 409(6822), 860-921. Larivière, V., and Y. Gingras (2010), The impact factor’s Matthew effect: A natural experiment in bibliometrics, Journal of the American Society for Information Science and Technology, 61, 424-427. Lawrence, S. (2001), Free online availability substantially increases a paper’s impact, Nature, 2001, 411, 521. Liebowitz, I.S.J. and J.P. Palmer (1984), Assessing the relative impacts of economics journals, Journal of Economic Literature, 22(1), 77-88. Liu, S.V. (2007), Hwang’s retracted publication still contributes to Science’s impact factor, Sci. Ethics, 2, 44-45. Lozano, G.A., V. Larivière, and Y. Gingras (2012), The weakening relationship between the impact factor and papers’ citations in the digital age, Journal of the American Society for Information Science and Technology, 63(11), 2140-2145. Lowry, O.H., N.J. Rosebrough, A.L. Farr, and R.J. Randall (1951), Protein measurement with the folin phenol reagent, Journal of Biological Chemistry, 193(1), 265-275. MacRoberts, M.H., and B.R. MacRoberts (1989), Problems of citation analysis: A critical review, Journal of the Association for Information Science and Technology, 40(5), 342-349. Malhotra, V., and E. Marder (2015), Peer review: The pleasure of publishing, eLife, 4, e05770. DOI: http://dx.doi.org/10.7554/eLife.05770. http://elifesciences.org/content/4/e05770. Martin, B.R. (1996), The use of multiple indicators in the assessment of basic research, Scientometrics, 36, 343-362. Marx, W., and L. Bornmann (2013), Journal impact factor: “The poor man’s citation analysis” and alternative approaches, European Science Editing, 39(3), 62-63. Maslow, S., and S. Redner (2008), Promise and pitfalls of extending Google’s PageRank algorithm to citation networks, Journal of Neuroscience, 28(44), 11103-11105. Moed, H.F., Th.N. Van Leeuwen, and J. Reedijk (1996), A critical analysis of the journal impact factors of Angewandte Chemie and the Journal of the American Chemical Society: Inaccuracies in published impact factors based on overall citations only, Scientometrics, 37, 105-116. Moustafa, K. (2015), The disaster of the impact factor, Science and Engineering Ethics, 21 (1), 139-142. van Nierop, E. (2009), Why do statistics journals have low impact factors?, Statistica Neerlandica, 63 (1), 52-62. Opthof, T. (1999), Submission, acceptance rate, rapid review system and impact factor, Cardiovasc Research, 41, 1-4. Palacios-Huerta, I., and O. Volij (2004), The measurement of intellectual influence, Econometrica, 72(3), 963-977. Perneger, T.V. (2004), Relation between online “hit counts” and subsequent citations: Prospective study of research papers in the BMJ, BMJ, 329(7465), 546-547. Pinski, G., and F. Narin (1976), Citation influence for journal aggregates of scientific publications: Theory with application to literature of physics, Information Processing & Management, 12(5), 297-312. PLoS Medicine Editors (6 June 2006), The impact factor game, PLoS Medicine, 3(6): e291. Postma, E. (2007), Inflated impact factors? The true impact of evolutionary papers in nonevolutionary journals, PLoS ONE, 2(10): e999. Pudovkin, A.I., and E. Garfield (2004), Rank-normalized impact factor: A way to compare journal performance across subject categories, Presented at the American Society for Information Science and Technology Annual Meeting, Providence, RI, 17 November 2004. Published in Proceedings of the 67th Annual Meeting of the American Society for Information Science & Technology, 41, pp. 507-515. REF (2014), Research Excellence Framework (REF), REF2014 guidelines. http://www.ref.ac.uk/about/guidance/. Rieseberg, L., and H. Smith (2008), Editorial and retrospective, Molecular Ecology, 17, 501-513. Rieseberg, L., T. Vines, and N. Kane (2011), Editorial - 20 years of molecular ecology, Molecular Ecology, 20, 1-21. Roberts, J. (2009), An author’s guide to publication ethics: A review of emerging standards in biomedical journals, Headache, 49, 578-589. Rossner, M., H. Van Epps, and E. Hill (2007), Show me the data, Journal of Cell Biology, 179(6), 1091-1092. Saunders, R., and J. Savulescu (2008), Research ethics and lessons from Hwanggate: What can we learn from the Korean cloning fraud?, J Med Ethics, 34, 214-221. Schuttea, H.K., and J.G. Svec (2007), Reaction of Folia Phoniatrica et Logopaedica on the current trend of impact factor measures, Folia Phoniatrica et Logopaedica, 59(6), 281-285. Seglen, P.O. (1992), Journal impact: How representative is the journal impact factor?, Research Evaluation, 2, 143-149. Seglen, P.O. (1995), Siteringer og tidsskrift-impakt som kvalitetsmål for forskning, Klinisk Kemi i Norden, 7, 59-63. Seglen, P.O. (1997), Why the impact factor of journals should not be used for evaluating research, BMJ, 314(7079), 498-502. Southern, E.M. (1975), Detection of specific sequences among DNA fragments separated by gel-electrophoresis, J Mol Biol, 98, 503-517. Smith, R. (1997), Journal accused of manipulating impact factor, BMJ, 314(7079), 461. Schubert, A., and T. Braun (1986), Relative indicators and relational charts for comparative assessment of publication output and citation impact, Scientometrics, 9(5-6), 281-291. Thelwall, M., S. Haustein, V. Larivière, and C.R. Sugimoto (2013), Do Altmetrics work? Twitter and ten other social web services, PLoS ONE, 8(5), e64841. Thomson Reuters (2013), Thomson Reuters statement regarding the San Francisco Declaration on Research Assessment. Available at http://researchanalytics.thomsonreuters.com/statement_re_sfdra/. Thomson Reuters (2015), Journal Citation Reports. http://wokinfo.com/products_tools/analytical/jcr/. Tobin, M.J. (2002), AJRCCM’s policy on duplicate publication, American Journal of Respiratory and Critical Care Medicine, 166, 433-437. Vaughan, L., and D. Shaw (2003), Bibliographic and web citations: What is the difference?, Journal of the Association for Information Science and technology, 54, 1313-1322. Vanclay, J.K. (2012), Impact factor: Outdated artefact or stepping-stone to journal certification?, Scientometrics, 92(2), 211-238. Vinkler P. (2010), The Evaluation of Research by Scientometric Indicators, Oxford, UK: Chandos. van Wesel, M. (2016), Evaluation by citation: Trends in publication behavior, evaluation criteria, and the strive for high impact publications, Science and Engineering Ethics, 22(1), 199-225. Wager E., S. Fiack, C. Graf, A. Robinson, and I. Rowlands (2009), Science journal editors’ views on publication ethics: Results of an international survey, Journal of Medical Ethics, 35, 348-353. Waltman, L., and N.J. Van Eck (2013), Source normalized indicators of citation impact: An overview of different approaches and an empirical comparison, Scientometrics, 96(3), 699-716. Waltman, L., N.J. Van Eck, T.N. Van Leeuwen, and M.S Visser (2013), Some modifications to the SNIP journal impact indicator, Journal of Informetrics, 7(2), 272-285. Weller A.C. (2001), Editorial peer review: Its strengths and weaknesses, American Society for Information Science and Technology Monograph Series, Information Today, ISBN 9781573871006, pp. 342. Wilhite, A.W. and Fong, E.A. (2012), Coercive Citation in Academic Publishing, Science, 335 (6068), 542-543. Wouters, P. (2013a), Bibliometrics of individual researchers - The debate in Berlin, Citation Culture, page 3, 3 October 2013. https://citationculture.wordpress.com/2013/10/03/bibliometrics-of-individual-researchers-thedebate-in-berlin/. Wouters, P. (2013b), The evidence on the journal impact factor, Citation Culture, page 2, 3 June 2013. https://citationculture.wordpress.com/2013/06/03/the-evidence-on-the-journal-impact-factor/.