Skip to main content
Log in

Impact factor: outdated artefact or stepping-stone to journal certification?

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

A review of Garfield’s journal impact factor and its specific implementation as the Thomson Reuters impact factor reveals several weaknesses in this commonly-used indicator of journal standing. Key limitations include the mismatch between citing and cited documents, the deceptive display of three decimals that belies the real precision, and the absence of confidence intervals. These are minor issues that are easily amended and should be corrected, but more substantive improvements are needed. There are indications that the scientific community seeks and needs better certification of journal procedures to improve the quality of published science. Comprehensive certification of editorial and review procedures could help ensure adequate procedures to detect duplicate and fraudulent submissions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Abraham, P. (2000). Duplicate and salami publications. Journal of Postgraduate Medicine, 46, 67–69.

    Google Scholar 

  • Abramo, G., D’Angelo, C. A., & Di Costa, F. (2010). Citations versus journal impact factor as proxy of quality: Could the latter ever be preferable? Scientometrics, 84, 821–833.

    Google Scholar 

  • Adler, R., Ewing, J., & Taylor, P. (2008). Joint committee on quantitative assessment of research: Citation statistics. The Australian Mathematical Society Gazette, 35(3), 166–188.

    MATH  Google Scholar 

  • Althouse, B. M., West, J. D., Bergstrom, C. T., & Bergstrom, T. (2009). Differences in impact factor across fields and over time. Journal of the American Society for Information Science and Technology, 60, 27–34.

    Google Scholar 

  • Altmann, K. G., & Gorman, G. E. (1998). The usefulness of impact factors in serial selection: A rank and mean analysis using ecology journals. Library Acquisitions: Practice and Theory, 22(2), 147–159.

    Google Scholar 

  • Andrade, A., González-Jonte, R., & Campanario, J. M. (2009). Journals that increase their impact factor at least fourfold in a few years: The role of journal self-citations. Scientometrics, 80, 515–528.

    Google Scholar 

  • Anonymous. (2004). Complacency about misconduct. Nature, 427, 1.

    Google Scholar 

  • Anonymous (2011). The Thomson Reuters journal selection process. Thomson Reuters, http://thomsonreuters.com/products_services/science/free/essays/journal_selection_process/. Accessed 2011 September 5.

  • ARC (2010). The excellence in research for Australia (ERA) Initiative. Australian Researh Council, Canberra. http://www.arc.gov.au/era/default.htm. Accessed 2011 August 5.

  • Archambault, E., & Lariviere, V. (2009). History of the journal impact factor: Contingencies and consequences. Scientometrics, 79, 635–649.

    Google Scholar 

  • Awrey, J., Inaba, K., Barmparas, G., Recinos, G., Teixeira, P. G. R., Chan, L. S., et al. (2011). Reference accuracy in the general surgery literature. World Journal of Surgery, 35, 475–479.

    Google Scholar 

  • Bain, C. R., & Myles, P. S. (2005). Relationship between journal impact factor and levels of evidence in anaesthesia. Anaesthesia and Intensive Care, 33(5), 567–570.

    Google Scholar 

  • Barbour, V. (2010). How ghost-writing threatens the credibility of medical knowledge and medical journals. Haematologica, 95, 1–2.

    Google Scholar 

  • Bath, F. J., Owen, V. E., & Bath, P. M. W. (1998). Quality of full and final publications reporting acute stroke trials: A systematic review. Stroke, 29(10), 2203–2210.

    Google Scholar 

  • Beal, J. (2011). Wiley-Blackwell announces continued growth in impact factor journals. Press release 11 July 2011, http://www.eurekalert.org/pub_releases/2011-07/w-wac071111.php. Accessed 2011 August 25.

  • Bensman, S. J. (2007). Garfield and the impact factor. Annual Review of Information Science and Technology, 41, 93–155.

    Google Scholar 

  • Bensman, S. J. (2008). Distributional differences of the impact factor in the sciences versus the social sciences: An analysis of the probabilistic structure of the 2005 journal citation reports. Journal of the American Society for Information Science and Technology, 59, 1366–1382.

    Google Scholar 

  • Bensman, S. J., Smolinsky, L. J., & Pudovkin, A. I. (2010). Mean citation rate per article in mathematics journals: Differences from the scientific model. Journal of the American Society for Information Science and Technology, 61, 1440–1463.

    Google Scholar 

  • Bernstam, E. V., Herskovic, J. R., Aphinyanaphongs, Y., Aliferis, C. F., Sriram, M. G., & Hersh, W. R. (2006). Using citation data to improve retrieval from MEDLINE. Journal of the American Medical Informatics Association, 13, 96–105.

    Google Scholar 

  • Berquist, T. H. (2008). Duplicate publishing or journal publication ethics 101. American Journal of Roentgenology, 191, 311–312.

    Google Scholar 

  • Bollen, J., Van de Sompel, H., Hagberg, A., & Chute, R. (2009). A principal component analysis of 39 scientific impact measures. PLoS ONE, 4(6), e6022.

    Google Scholar 

  • Bonillo Perales, A. (2002). Spanish pediatric publications in pubmed between 1996 and 2001 [Publicaciones pediátricas españolas en PubMed en los años 1996 y 2001]. Anales Espanoles de Pediatria, 57(2), 152–156.

    Google Scholar 

  • Bornmann, L., Nast, I., & Daniel, H.-D. (2008). Do editors and referees look for signs of scientific misconduct when reviewing manuscripts? A quantitative content analysis of studies that examined review criteria and reasons for accepting and rejecting manuscripts for publication. Scientometrics, 77, 415–432.

    Google Scholar 

  • Braun, T. (Ed.) (2007). The Impact Factor of scientific and scholarly journals: Its use and misuse in research evaluation. Scientometrics Guidebooks Series (vol. 2, pp. 686). Budapest: Akademiai Kiado.

  • Braun, T., & Glanzel, W. (1995). On a source of error in computing journal impact factors. Chemical Intelligencer, 1, 31–32.

    Google Scholar 

  • Brown, H. (2007). How impact factors changed medical publishing: and science. BMJ, 334, 561–564.

    Google Scholar 

  • Brumback, R. A. (2009a). Impact factor wars: Episode V: The empire strikes back. Journal of Child Neurology, 24, 260–262.

    Google Scholar 

  • Brumback, R. A. (2009b). Impact factor: Let’s be unreasonable! Epidemiology, 20, 932–933.

    Google Scholar 

  • Butakov, S., & Scherbinin, V. (2009). The toolbox for local and global plagiarism detection. Computers & Education, 52, 781–788.

    Google Scholar 

  • Callaham, M., & McCulloch, C. (2011). Longitudinal trends in the performance of scientific peer reviewers. Annals of Emergency Medicine, 57, 141–148.

    Google Scholar 

  • Callaham, M., Wears, R. L., & Weber, E. (2002). Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. JAMA, 287(21), 2847–2850.

    Google Scholar 

  • Calver, M. C., & Bradley, J. S. (2010). Patterns of citations of open access and non-open access conservation biology journal papers and book chapters. Conservation Biology, 24(3), 872–880.

    Google Scholar 

  • Cameron, B. D. (2005). Trends in the usage of ISI bibliometric data: Uses, abuses, and implications. Portal, 5, 105–125.

    Google Scholar 

  • Campanario, J. M. (2011a). Empirical study of journal impact factors obtained using the classical two-year citation window versus a 5-year citation window. Scientometrics, 87, 189–204.

    Google Scholar 

  • Campanario, J. M. (2011b). Large increases and decreases in journal impact factors in only 1 year: The effect of journal self-citations. Journal of the American Society for Information Science and Technology, 62, 230–235.

    Google Scholar 

  • Campanario, J. M., Carretero, J., Marangon, V., Molina, A., & Ros, G. (2011). Effect on the journal impact factor of the number and document type of citing records: A wide-scale study. Scientometrics, 87, 75–84.

    Google Scholar 

  • Campbell, P. (2008). Escape from the impact factor. Ethics in Science and Environmental Politics, 8, 5–7.

    Google Scholar 

  • Cartwright, V. A., & McGhee, C. N. J. (2005). Ophthalmology and vision science research. Part 1: Understanding and using journal impact factors and citation indices. Journal of Cataract and Refractive Surgery, 31, 1999–2007.

    Google Scholar 

  • Cashore, B., van Kooten, C. G., Vertinsky, I., Auld, G., & Affolderbach, J. (2005). Private or self-regulation? A comparative study of forest certification choices in Canada, the United States and Germany. Forest Policy and Economics, 7, 53–69.

    Google Scholar 

  • Chalmers, I. (2006). Role of systematic reviews in detecting plagiarism: Case of Asim Kurjak. BMJ, 333, 594.

    Google Scholar 

  • Chapman, S., Ragg, M., & McGeechan, K. (2009). Citation bias in reported smoking prevalence in people with schizophrenia. Australian and New Zealand Journal of Psychiatry, 43(3), 277–282.

    Google Scholar 

  • Cheek, J., Garnham, B., & Quan, J. (2006). What’s in a number? Issues in providing evidence of impact and quality of research(ers). Qualitative Health Research, 16, 423–435.

    Google Scholar 

  • Chen, C. (2006). CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature. Journal of the American Society for Information Science and Technology, 57(3), 359–377.

    Google Scholar 

  • Chew, M., Villanueva, E. V., & Van Der Weyden, M. B. (2007). Life and times of the impact factor: Retrospective analysis of trends for seven medical journals (1994–2005) and their Editors’ views. Journal of the Royal Society of Medicine, 100, 142–150.

    Google Scholar 

  • Cobo, M. J., Lopez-Herrera, A. G., Herrera-Viedma, E., & Herrera, F. (2011). Science mapping software tools: Review, analysis, and cooperative study among tools. JASIST, 62(7), 1382–1402.

    Google Scholar 

  • Cohen, J. B. (1938). The misuse of statistics. Journal of the American Statistical Association, 33(204), 657–674.

    Google Scholar 

  • Coleman, R. (1999). Impact factors: Use and abuse in biomedical research. Anatomical Record, 257, 54–57.

    Google Scholar 

  • Coleman, A. (2007). Assessing the value of a journal beyond the impact factor. Journal of the American Society for Information Science and Technology, 58(8), 1148–1161.

    Google Scholar 

  • Davies, J. (1955). Phrenology: Fad and science: A 19th-century American crusade (p. 203). New Haven: Yale University Press.

    Google Scholar 

  • Dempsey, J. A. (2009). Impact factor and its role in academic promotion: a statement adopted by the international respiratory journal editors roundtable. Journal of Applied Physiology, 107, 1005.

    Google Scholar 

  • Dong, P., Loh, M., & Mondry, A. (2005). The “impact factor” revisited. Biomedical Digital Libraries, 2, 7.

    Google Scholar 

  • Dost, F. N. (2008). Peer review at a crossroads: A case study. Environmental Science and Pollution Research, 15, 443–447.

    Google Scholar 

  • Drew, D. E., & Karpf, R. (1981). Ranking academic departments: Empirical findings and a theoretical perspective. Research in Higher Education, 14(4), 305–320.

    Google Scholar 

  • Eaton, L. (2005). Medical editors issue guidance on ghost writing. BMJ, 330(7498), 988.

    Google Scholar 

  • Elsevier (2011). Article posting policies. Elsevier BV. http://www.elsevier.com/wps/find/authorsview.authors/postingpolicy. Accessed 2011 September 10.

  • Epstein, R. J. (2004). Journal impact factors do not equitably reflect academic staff performance in different medical subspecialties. Journal of Investigative Medicine, 52, 531–536.

    Google Scholar 

  • Errami, M., & Garner, H. (2008). A tale of two citations. Nature, 451, 397–399.

    Google Scholar 

  • Errami, M., Hicks, J. M., Fisher, W., Trusty, D., Wren, J. D., Long, T. C., et al. (2008). Deja vu: A study of duplicate citations in medline. Bioinformatics, 24, 243–249.

    Google Scholar 

  • Falagas, M. E., & Alexiou, V. G. (2008). The top-ten in journal impact factor manipulation. Archivum Immunologiae et therapiae Experimentalis, 56(4), 223–226.

    Google Scholar 

  • Fassoulaki, A., Paraskeva, A., Papilas, A., & Karabinis, G. (2000). Self-citations in six anaesthesia journals and their significance in determining the impact factor. British Journal of Anaesthesia, 84(2), 266–269.

    Google Scholar 

  • Favaloro, E. J. (2009). The journal impact factor: Don’t expect its demise any time soon. Clinical Chemistry and Laboratory Medicine, 47, 1319–1324.

    Google Scholar 

  • Foo, J. Y. A. (2009). The retrospective analysis of bibliographical trends for nine biomedical engineering journals from 1999 to 2007. Annals of Biomedical Engineering, 37, 1474–1481.

    Google Scholar 

  • Foo J. Y. A. (2010). A retrospective analysis of the trend of retracted publications in the field of biomedical and life sciences. Science and Engineering Ethics. doi:10.1007/s11948-010-9212-8.

  • Foo, J. Y. A. (2011). Impact of excessive journal self-citations: A case study on the folia phoniatrica et logopaedica journal. Science and Engineering Ethics, 17, 65–73.

    Google Scholar 

  • Fox, M. F. (1994). Scientific misconduct and editorial and peer review processes. J Higher Educ, 65, 298–309.

    Google Scholar 

  • Frank, M. (2003). Impact factors: arbiter of excellence? J Med Libr Assoc, 91, 4–6.

    Google Scholar 

  • Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, 178, 471–479.

    Google Scholar 

  • Garfield, E. (1994). The Thomson Reuters impact factor. http://thomsonreuters.com/products_services/science/free/essays/impact_factor/.

  • Garfield, E. (1996). How can impact factors be improved? British Medical Journal, 313, 411–413.

    Google Scholar 

  • Garfield, E. (1999). Journal impact factor: A brief review. Canadian Medical Association Journal, 161, 979–980.

    Google Scholar 

  • Garfield, E. (2006). The history and meaning of the journal impact factor. JAMA, 295, 90–93.

    Google Scholar 

  • Glänzel, W. (2009). The multi-dimensionality of journal impact. Scientometrics, 78, 355–374.

    Google Scholar 

  • Glänzel, W., & Moed, H. F. (2002). Journal impact measures in bibliometric research. Scientometrics, 53, 171–193.

    Google Scholar 

  • Gluud, L. L., Sorensen, T. I. A., Gotzsche, P. C., & Gluud, C. (2005). The journal impact factor as a predictor of trial quality and outcomes: Cohort study of hepatobiliary randomized clinical trials. American Journal of Gastroenterology, 100, 2431–2435.

    Google Scholar 

  • Glynn, R. W., Chin, J. Z., Kerin, M. J., & Sweeney, K. J. (2010). Representation of cancer in the medical literature: A bibliometric analysis. PLoS ONE, 5(11), e13902.

    Google Scholar 

  • Godlee, F. (2004). Dealing with editorial misconduct. BMJ, 329, 1301.

    Google Scholar 

  • Goldstein, H., & Maier, G. (2010). The use and valuation of journals in planning scholarship: Peer assessment versus impact factors. Journal of Planning Education and Research, 30, 66–75.

    Google Scholar 

  • Gollogly, L., & Momen, H. (2006). Ethical dilemmas in scientific publication: pitfalls and solutions for editors. Rev Saúde Pública, 40, 24–29.

    Google Scholar 

  • Gøtzsche, P. C., Kassirer, J. P., Woolley, K. L., Wager, E., Jacobs, A., et al. (2009). What should be done to tackle ghost writing in the medical literature? PLoS Med, 6(2), e1000023. doi:10.1371/journal.pmed.1000023.

    Google Scholar 

  • Greenwood, D. C. (2007). Reliability of journal impact factor rankings. BMC Medical Research Methodology, 7, 48.

    Google Scholar 

  • Gwilym, S. E., Swan, M. C., & Giele, H. (2004). One in 13 ‘original’ articles in the journal of bone and joint surgery are duplicate or fragmented publications. The Journal of Bone and Joint Surgery, 86-B(5), 743–745.

    Google Scholar 

  • Ha, T. C., Tan, S. B., & Soo, K. C. (2006). The journal impact factor: Too much of an impact? Annals of the Academy of Medicine, Singapore, 35, 911–916.

    Google Scholar 

  • Habibzadeh, F., & Winker, M. A. (2009). Duplicate publication and plagiarism: causes and cures. Notfall Rettungsmed, 12, 415–418.

    Google Scholar 

  • Hames, I. (2007). Peer review and manuscript management in scientific journals: guidelines for good practice. Oxford: Wiley. doi:10.1002/9780470750803.

    Google Scholar 

  • Hansen, H. B., & Henriksen, J. H. (1997). How well does journal ‘impact’ work in the assessment of papers on clinical physiology and nuclear medicine? Clinical Physiology, 17(4), 409–418.

    Google Scholar 

  • Haslam, N., & Koval, P. (2010). Predicting long-term citation impact of articles in social and personality psychology. Psychological Reports, 106(3), 891–900.

    Google Scholar 

  • Hernan, M. A. (2009). Impact factor: A call to reason. Epidemiology, 20, 317–318.

    Google Scholar 

  • Ho, D. (2011). Thomson Reuters: Cause for celebration! Journal of Laboratory Automation, 16(3), A7–A8.

    Google Scholar 

  • Holsapple, C. W. (2009). Journal self-citation II: The quest for high impact:Truth and consequences? Communications of the Association for Information Systems, 25, 11–20.

    Google Scholar 

  • Horner, J., & Minifie, F. D. (2011). Research ethics III: Publication practices and authorship, conflicts of interest, and research misconduct. Journal of Speech, Language, and Hearing S346 Research, 54, S346–S362.

    Google Scholar 

  • Hunt, G. E., Cleary, M., & Walter, G. (2010). Psychiatry and the Hirsch h-index: The relationship between journal impact factors and accrued citations. Harvard Review of Psychiatry, 18(4), 207–219.

    Google Scholar 

  • Jacobs, G., & Ip, B. (2005). Ring fenced research: The case of computer-assisted learning in health sciences. British Journal of Educational Technology, 36, 361–377.

    Google Scholar 

  • Jacso, P. (2009). Five-year impact factor data in the journal citation reports. Online Information Review, 33, 603–614.

    Google Scholar 

  • Jahangiriana, M., Eldabi, T., Garg, L., Jun, G. T., Naseer, A., Patel, B., et al. (2011). A rapid review method for extremely large corpora of literature: Applications to the domains of modelling, simulation, and management. International Journal of Information Management, 31, 234–243.

    Google Scholar 

  • Jeacle, I., & Carter, C. (2011). In TripAdvisor we trust: Rankings, calculative regimes and abstract systems. Accounting, Organizations and Society. doi:10.1016/j.aos.2011.04.002.

  • Johnson, C. (2006). Repetitive, duplicate, and redundant publications: a review for authors and readers. Journal of Manipulative and Physiological Therapeutics, 29, 505–509.

    Google Scholar 

  • Johnstone, M.-J. (2007). Journal impact factors: Implications for the nursing profession. International Nursing Review, 54, 35–40.

    Google Scholar 

  • Jones, A. W. (2002). JAT’s impact factor: Room for improvement? Journal of Analytical Toxicology, 26, 2–5.

    Google Scholar 

  • Jones, A. W. (2003). Impact factors of forensic science and toxicology journals: What do the numbers really mean? Forensic Science International, 133, 1–8.

    Google Scholar 

  • Jörgensen, P. (2005). Incorporating context in text analysis by interactive activation with competition artificial neural networks. Information Processing & Management, 41(5), 1081–1099.

    Google Scholar 

  • Kapeller, J. (2010). Citation metrics: Serious drawbacks, perverse incentives, and strategic options for heterodox economics. American Journal of Economics and Sociology, 69, 1376–1408.

    Google Scholar 

  • Ketcham, C. M. (2008). The proper use of citation data in journal management. Archivum Immunologiae et therapiae Experimentalis, 56, 357–362.

    Google Scholar 

  • Knothe, G. (2006). Comparative citation analysis of duplicate or highly related publications. Journal of the American Society for Information Science and Technology, 57(13), 1830–1839.

    Google Scholar 

  • Kotiaho, J. S., Tomkins, J. L., & Simmons, L. W. (1999). Unfamiliar citations breed mistakes. Nature, 400, 307.

    Google Scholar 

  • Krell, F.-T. (2010). Should editors influence journal impact factors? Learned Publishing, 23, 59–62.

    Google Scholar 

  • Kumar, V., Upadhyay, S., & Medhi, B. (2009). Impact of the impact factor in biomedical research: Its use and misuse. Singapore Medical Journal, 50, 752–755.

    Google Scholar 

  • Kurmis, A. P. (2003). Understanding the limitations of the journal impact factor. Journal of Bone and Joint Surgery: Series A, 85, 2449–2454.

    Google Scholar 

  • Kurmis, A. P., & Kurmis, T. P. (2006). Exploring the relationship between impact factor and manuscript rejection rates in radiologic journals. Academic Radiology, 13, 77–83.

    Google Scholar 

  • Kurmis, T. P., & Kurmis, A. P. (2010). Self-citation rates among medical imaging journals and a possible association with impact factor. Radiography, 16, 21–25.

    Google Scholar 

  • Laband, D. N. (1990). Is there value-added from the review process in economics?: Preliminary evidence from authors. The Quarterly Journal of Economics, 105, 341–352.

    Google Scholar 

  • Lange, L. L. (2002). The impact factor as a phantom: Is there a self-fulfilling prophecy effect of impact? Journal of Documentation, 58, 175–184.

    Google Scholar 

  • Larivière, V., & Gingras, Y. (2010). The impact factor’s Matthew effect: A natural experiment in bibliometrics. Journal of the American Society for Information Science and Technology, 61, 424–427.

    Google Scholar 

  • Lau, S. L., & Samman, N. (2007). Levels of evidence and journal impact factor in oral and maxillofacial surgery. International Journal of Oral and Maxillofacial Surgery, 36, 1–5.

    Google Scholar 

  • Lawrence, P. A. (2007). The mismeasurement of science. Current Biology, 17, R583–R585.

    Google Scholar 

  • Lehmkuhl, G., Petermann, F., & Warnke, A. (2009). Research in child and adolescent psychiatry, promotion of young academics and publication practice [Kinder- und jugendpsychiatrische Forschung, Förderung des wissenschaftlichen Nachwuchses und Veröffentlichungspraxis]. Zeitschrift fur Kinder- und Jugendpsychiatrie und Psychotherapie, 37, 93–96.

    Google Scholar 

  • Leutner, D., & Wirth, J. (2007). As mirrored by the journal: Themes and trends of educational psychology in the years 2005 to 2007 [Im Spiegel der Zeitschrift: Themen und Trends der Pädagogischen Psychologie in den Jahren 2005 bis 2007]. Zeitschrift fur Padagogische Psychologie, 21, 195–202.

    Google Scholar 

  • Leydesdorff, L., & Opthof, T. (2010). Scopus’s source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations. Journal of the American Society for Information Science and Technology, 61, 2365–2369.

    Google Scholar 

  • Lock, S. (1995). Lessons from the Pearce affair: Handling scientific fraud. BMJ, 310, 1547–1548.

    Google Scholar 

  • Lomnicki, A. (2003). Impact factors reward and promote excellence. Nature, 424, 487.

    Google Scholar 

  • Maier, G. (2006). Impact factors and peer judgment: The case of regional science journals. Scientometrics, 69(3), 651–667.

    Google Scholar 

  • McGarty, C. (2000). The citation impact factor in social psychology: a bad statistic that encourages bad science. Current Research in Social Psychology, 5, 1–16.

    Google Scholar 

  • McKeever, L. (2006). Online plagiarism detection services: Saviour or scourge? Assessment & Evaluation in Higher Education, 31, 155–165.

    Google Scholar 

  • McVeigh, M. E. (2004). Journal self-citation in the journal citation reports. Thomson Reuters, http://thomsonreuters.com/products_services/science/free/essays/journal_self_citation_jcr. Accessed 2011 September 7.

  • McVeigh, M. E., & Mann, S. J. (2009). The journal impact factor denominator: Defining citable (counted) items. JAMA, 302, 1107–1109.

    Google Scholar 

  • Medarević, M., Banković, S., Pantić, D. & Obradović, S. (2010) Effects of the control method (goč variety) in selection forest management in western Serbia. Archives of Biological Science, 62(2), 407–418.

    Google Scholar 

  • Mehrad, J., & Goltaji, M. (2010). Correlation between journal self citation with impact factor for the scientific publications in humanities published between 2001 and 2007 based on Persian journal citation report generated by Islamic science citation database. Information Sciences and Technology, 25, 189–206.

    Google Scholar 

  • Mehrad, J., & Goltaji, M. (2011). Correlation between journal self-citation and impact factor in ISC’s PJCR agriculture and veterinary science journals during 2001–2007. International Journal of Information Science and Management, 9, 75–87.

    Google Scholar 

  • Meneghini, R., Packer, A. L., & Nassi-Calo, L. (2008). Articles by Latin American authors in prestigious journals have fewer citations. PLoS ONE, 3(11), e3804.

    Google Scholar 

  • Metze, K. (2010). Bureaucrats, researchers, editors and the impact factor: A vicious circle that is detrimental to science. Clinics, 65(10), 937–940.

    Google Scholar 

  • Meyer zu Eissen, S., & Stein, B. (2006). Intrinsic plagiarism detection. Lecture Notes in Computer Science, 3936, 565–569.

    Google Scholar 

  • Minelli, C., Thompson, J. R., Abrams, K. R., Thakkinstian, A., & Attia, J. (2009). The quality of meta-analyses of genetic association studies: A review with recommendations. American Journal of Epidemiology, 170, 1333–1343.

    Google Scholar 

  • Moed, H., & Plume, A. (2011). Research assessment: The multi-dimensional research assessment matrix. Research Trends, 23, 5–7.

    Google Scholar 

  • Moed, H. F., Van Leeuwen, Th. N., & Reedijk, J. (1996). A critical analysis of the journal impact factors of Angewandte chemie and the Journal of the American Chemical Society: Inaccuracies in published impact factors based on overall citations only. Scientometrics, 37, 105–116.

    Google Scholar 

  • Moller, J., Retelsdorf, J., & Sudkamp, A. (2010). As mirrored by the journal: Themes and trends of educational psychology in the years 2008 to 2010 [Im Spiegel der Zeitschrift: Themen und Trends der Pädagogischen Psychologie in den Jahren 2008 bis 2010]. Zeitschrift fur Padagogische Psychologie, 24, 163–169.

    Google Scholar 

  • Monastersky, R. (2005). The number that’s devouring science. The chronicle of higher education 52(8):A12 http://chronicle.com/free/v52/i08/08a01201.htm.

  • Mullen, L. B. (2008). Increasing Impact of Scholarly Journal Articles: Practical Strategies Librarians Can Share. E-JASL: The Electronic Journal of Academic and Special Librarianship 9, 1, http://southernlibrarianship.icaap.org/content/v09n01/mullen_l01.html. Accessed 2011 September 1.

  • Neuhaus, C., Marx, W., & Daniel, H.-D. (2009). The publication and citation impact profiles of Angewandte Chemie and the journal of the American Chemical Society based on the sections of chemical abstracts: A case study on the limitations of the journal impact factor. Journal of the American Society for Information Science and Technology, 60, 176–183.

    Google Scholar 

  • Obremskey, W. T., Pappas, N., Attallah-Wasif, E., Tornetta, P., I. I. I., & Bhandari, M. (2005). Level of evidence in orthopaedic journals. Journal of Bone and Joint Surgery: Series A, 87(12 I), 2632–2638.

    Google Scholar 

  • Ogden, T. L., & Bartley, D. L. (2008). The ups and downs of journal impact factors. Annals of Occupational Hygiene, 52(2), 73–82.

    Google Scholar 

  • Owlia, P., Vasei, M., Goliaei, B., & Nassiri, I. (2011). Normalized impact factor (NIF): An adjusted method for calculating the citation rate of biomedical journals. Journal of Biomedical Informatics, 44, 216–220.

    Google Scholar 

  • Patterson, M. (2009). Is the end in cite? EMBO reports, 10(11), 1186.

    MathSciNet  Google Scholar 

  • Perneger, T. V. (2010). Citation analysis of identical consensus statements revealed journal-related bias. Journal of Clinical Epidemiology, 63(6), 660–664.

    Google Scholar 

  • PLoS Medicine Editors. (2006). The impact factor game. PLoS Med, 3(6), e291. doi:10.1371/journal.pmed.0030291.

    Google Scholar 

  • Poomkottayil, D., Bornstein, M. M., & Sendi, P. (2011). Lost in translation: The impact of publication language on citation frequency in the scientific dental literature. Swiss Medical Weekly, 141, w13148.

    Google Scholar 

  • Porter, M., & McIntyre, S. (1984). What is, must be best: A research note on conservative or deferential responses to antenatal care provision. Social Science and Medicine, 19, 1197–1200.

    Google Scholar 

  • Postma, E. (2007). Inflated impact factors? The true impact of evolutionary papers in non-evolutionary journals. PLoS ONE, 2(10), e999.

    Google Scholar 

  • Preti, F., Dani, A., & Laio, F. (2010). Root profile assessment by means of hydrological, pedological and above-ground vegetation information for bio-engineering purposes. Ecological Engineering, 36, 305–316.

    Google Scholar 

  • Pudovkin, A. I., & Garfield, E. (2004). Rank-normalized impact factor: A way to compare journal performance across subject categories. Proceedings of the ASIST Annual Meeting, 41, 507–515.

    Google Scholar 

  • Racki, G. (2009). Rank-normalized journal impact factor as a predictive tool. Archivum Immunologiae et therapiae Experimentalis, 57, 39–43.

    Google Scholar 

  • Ramsden, J. J. (2009). Impact factors: A critique. Journal of Biological Physics and Chemistry, 9, 139–140.

    Google Scholar 

  • Reedijk, J., & Moed, H. F. (2008). Is the impact of journal impact factors decreasing? Journal of Documentation, 64, 183–192.

    Google Scholar 

  • Reller, T. (2011). Elsevier announces 2010 journal impact factor highlights. Press release, 14 July 2011. http://www.marketwatch.com/story/elsevier-announces-2010-journal-impact-factor-highlights-2011-07-14. Accessed 2011 August 25.

  • Resnik, D. B., Peddada, S., & Brunson, W., Jr. (2009). Research misconduct policies of scientific journals. Accountability in Research, 16, 254–267.

    Google Scholar 

  • Resnik, D. B., Patrone, D., & Peddada, S. (2010). Research misconduct policies of social science journals and impact factor. Accountability in Research, 17, 79–84.

    Google Scholar 

  • Rezaei-Ghaleh, N., & Azizi, F. (2007). The impact factor-based quality assessment of biomedical research institutes in Iran: Effect of impact factor normalization by subject. Archives of Iranian Medicine, 10, 182–189.

    Google Scholar 

  • Rieseberg, L., & Smith, H. (2008). Editorial and Retrospective. Molecular Ecology, 17, 501–513.

    Google Scholar 

  • Rieseberg, L., Vines, T., & Kane, N. (2011). 20 years of molecular ecology. Molecular Ecology, 20, 1–21.

    Google Scholar 

  • Robert, C. P. (2010). On the relevance of the Bayesian approach to statistics. Review of Economic Analysis, 2, 139–152.

    Google Scholar 

  • Roberts, J. (2009). An author’s guide to publication ethics: A review of emerging standards in biomedical journals. Headache, 49, 578–589.

    Google Scholar 

  • Roosendaal, H. E., Geurts, P. A. T. M. (1997). Forces and functions in scientific communication: An analysis of their interplay. Cooperative Research Information Systems in Physics, August 31–September 4 1997, Oldenburg. http://www.physik.unioldenburg.de/conferences/crisp97/roosendaal.html.

  • Rossner, M. (2006). How to guard against image fraud. The Scientist, 20(3), 24–25.

    Google Scholar 

  • Rossner, M., Van Epps, H., & Hill, E. (2007). Show me the data. Journal of Cell Biology, 179, 1091–1092.

    Google Scholar 

  • Rousseau, R. (2002). Journal evaluation: Technical and practical issues. Library Trends, 50, 418.

    Google Scholar 

  • Rousseau, R., & Van Hooydonk, G. (1996). Journal production and journal impact factors. Journal of the American Society for Information Science, 47, 775–780.

    Google Scholar 

  • Ruiz, M. A., Greco, O. T., & Braile, D. M. (2009). Journal impact factor: This editorial, academic and scientific influence [Fator de impacto: Importancia e influencia no meio editorial, academico e cientifico]. Brazilian Journal of Cardiovascular Surgery, 24, 273–278.

    Google Scholar 

  • Saha, S., Saint, S., & Christakis, D. A. (2003). Impact factor: A valid measure of journal quality? Journal of the Medical Library Association, 91, 42–46.

    Google Scholar 

  • Sample, I. (2002). People weigh less on a hard surface. New Scientist http://www.newscientist.com/article/dn2462-people-weigh-less-on-a-hard-surface.html.

  • Saunders, R., & Savulescu, J. (2008). Research ethics and lessons from Hwanggate: what can we learn from the Korean cloning fraud? Journal of Medical Ethics, 34, 214–221.

    Google Scholar 

  • Schoonbaert, D., & Roelants, G. (1996). Citation analysis for measuring the value of scientific publications: Quality assessment tool or comedy of errors? Tropical Medicine and International Health, 1, 739–752.

    Google Scholar 

  • Schopfel, J., & Prost, H. (2009). Comparison of SCImago journal rank indicator (SJR) with JCR journal impact factor (IF) for French journals [Le JCR facteur d’impact (IF) et le SCImago journal rank indicator (SJR) des revues françaises: une étude comparative]. Psychologie Francaise, 54, 287–305.

    Google Scholar 

  • Schubert, A., & Glanzel, W. (1983). Statistical reliability of comparisons based on the citation impact of scientific publications. Scientometrics, 5, 59–73.

    Google Scholar 

  • Schumm, W. R. (2010). A comparison of citations across multidisciplinary psychology journals: A case study of two independent journals. Psychological Reports, 106, 314–322.

    Google Scholar 

  • Scully, C., & Lodge, H. (2005). Impact factors and their significance; overrated or misused? British Dental Journal, 198, 391–393.

    Google Scholar 

  • Seglen, P. O. (1989). Use of citation analysis and other bibliometric methods in evaluation of the quality of research [Bruk av siteringsanalyse og andre bibliometriske metoder i evaluering av forskningskvalitet]. Tidsskrift for den Norske Laegeforening, 109(31), 3224–3229.

    Google Scholar 

  • Seglen, P. O. (1992). Journal impact: How representative is the journal impact factor? Research Evaluation, 2, 143–149.

    Google Scholar 

  • Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating research. BMJ, 314, 497–502.

    Google Scholar 

  • Simkin, M. V., & Roychowdhury, V. P. (2003). Read before you cite! Complex Syst, 14, 269–274.

    Google Scholar 

  • Simons, K. (2008). The misused impact factor. Science, 322(5899), 165.

    Google Scholar 

  • Skovsgaard, J. P., & Vanclay, J. K. (2008). Forest site productivity: A review of the evolution of dendrometric concepts for even-aged stands. Forestry, 81, 13–31.

    Google Scholar 

  • Smith, D. R. (2007). Historical development of the journal impact factor and its relevance for occupational health. Industrial Health, 45, 730–742.

    Google Scholar 

  • Sombatsompop, N., & Markpin, T. (2005). Making an equality of ISI impact factors for different subject fields. Journal of the American Society for Information Science and Technology, 56, 676–683.

    Google Scholar 

  • Sombatsompop, N., Markpin, T., & Premkamolnetr, N. (2004). A modified method for calculating the impact factors of journals in ISI journal citation reports: Polymer science category in 1997–2001. Scientometrics, 60, 217–235.

    Google Scholar 

  • Sonderstrup-Andersen, E. M., & Sonderstrup-Andersen, H. H. K. (2008). An investigation into diabetes researcher’s perceptions of the journal impact factor: Reconsidering evaluating research. Scientometrics, 76, 391–406.

    Google Scholar 

  • Soreide, K., & Winter, D. C. (2010). Global survey of factors influencing choice of surgical journal for manuscript submission. Surgery, 147, 475–480.

    Google Scholar 

  • Starbuck, W. H. (2005). How much better are the most-prestigious journals? The statistics of academic publication. Organization Science, 16, 180–200.

    Google Scholar 

  • Statzner, B., & Resh, V. H. (2010). Negative changes in the scientific publication process in ecology: Potential causes and consequences. Freshwater Biology, 55, 2639–2653.

    Google Scholar 

  • Stein, R. J., Haddock, C. K., Poston, W. S. C., Catanese, D., & Spertus, J. A. (2005). Precision in weighing: A comparison of scales found in physician offices, fitness centers, and weight loss centers. Public Health Reports, 120, 266–270.

    Google Scholar 

  • Stiftel, B., & Mukhopadhyay, C. (2007). Thoughts on Anglo-American hegemony in planning scholarship: Do we read each other’s work? Town Planning Review, 78, 545–572.

    Google Scholar 

  • Straub, D. W., & Anderson, C. (2009). Journal self-citation VI: Forced journal self-citation: Common, appropriate, ethical? Communications of the Association for Information Systems, 25, 57–66.

    Google Scholar 

  • Tal, A., & Gordon, J. (2010). Carbon cautious: Israel’s afforestation experience and approach to sequestration. Small-Scale Forestry, 9, 409–428.

    Google Scholar 

  • Taylor, M., Perakakis, P., & Trachana, V. (2008). The siege of science. Ethics in Science and Environmental Politics, 8, 17–40.

    Google Scholar 

  • Thomson Reuters (2011). Journal citation reports. Thomson reuters products and services. http://thomsonreuters.com/products_services/science/science_products/a-z/journal_citation_reports/. Accessed 2011 August 16.

  • Timuralp, B. (2010). Our first impact factor. Anadolu Kardiyoloji Dergisi, 10(4), 297.

    Google Scholar 

  • Tobin, M. J. (2002). AJRCCM’s policy on duplicate publication. American Journal of Respiratory and Critical Care Medicine, 166, 433–437.

    Google Scholar 

  • Todd, P. A., & Ladle, R. J. (2008). Hidden dangers of a ‘citation culture’. Ethics in Science and Environmental Politics, 8, 13–16.

    Google Scholar 

  • Todd, P. A., Guest, J. R., Lu, J., & Chou, L. M. (2010). One in four citations in marine biology papers is inappropriate. Marine Ecology Progress Series, 408, 299–303.

    Google Scholar 

  • Ugolini, D., Bogliolo, A., Parodi, S., Casilli, C., & Santi, L. (1997). Assessing research productivity in an oncology research institute: The role of the documentation center. Bulletin of the Medical Library Association, 85, 33–38.

    Google Scholar 

  • Van de Sompel, H., Payette, S., Erickson, J., Lagoze, C., & Warner, S. (2004). Rethinking scholarly communication: Building the system that scholars deserve. D-Lib Magazine, 10, 9.

    Google Scholar 

  • Van Driel, M., De Maeseneer, J., De Sutter, A., De Bacquer, D., De Backer, G., & Christiaens, T. (2008). How scientific is the assessment of the quality of scientific output using the journal impact factor? [Hoe wetenschappelijk is het beoordelen van wetenschappelijk werk aan de hand van impactfactoren van tijdschriften?]. Tijdschrift voor Geneeskunde, 64, 471–476.

    Google Scholar 

  • Van Gyseghem, E., Baert, L., Van Remoortere, P., van ‘t Klooster, G., Rouan, M.-C., Voorspoels, J., de Kock, H., Schueller, L., Rosier, J., Grooten, L., & Van den Mooter, G. (2010) Co-administration of darunavir and a new pharmacokinetic booster: Formulation strategies and evaluation in dogs. European Journal of Pharmaceutical Sciences, 41, 193–200.

    Google Scholar 

  • Van Leeuwen, T. N., Moed, H. F., & Reedijk, J. (1999). Critical comments on institute for scientific information impact factors: a sample of inorganic molecular chemistry journals. Journal of Information Science, 25(6), 189–198.

    Google Scholar 

  • Vanclay, J. K. (2009). Bias in the journal impact factor. Scientometrics, 78, 3–12.

    Google Scholar 

  • Vlosky, R. P., & Ozanne, L. K. (1997). Forest products certification: The business customer perspective. Wood and Fiber Science, 29, 195–208.

    Google Scholar 

  • Wager, E., Fiack, S., Graf, C., Robinson, A., & Rowlands, I. (2009). Science journal editors’ views on publication ethics: Results of an international survey. Journal of Medical Ethics, 35, 348–353.

    Google Scholar 

  • Wagner, A. B. (2009). Percentile-based journal impact factors: A neglected collection development metric. Issues in Science and Technology Librarianship 57.

  • Walter, G., Bloch, S., Hunt, G., & Fisher, K. (2003). Counting on citations: A flawed way to measure quality. Medical Journal of Australia, 178(6), 280–281.

    Google Scholar 

  • Wang, S., Wang, H., & Weldon, P. R. (2010). Effect of cooperation between Chinese scientific journals and international publishers on journals’ impact factor. Journal of Informetrics, 4, 233–238.

    Google Scholar 

  • Weale, A. R., Bailey, M., & Lear, P. A. (2004). The level of non-citation of articles within a journal as a measure of quality: A comparison to the impact factor. BMC Medical Research Methodology, 4, 14.

    Google Scholar 

  • Weller, A. C. (2001). Editorial peer review: Its strengths and weaknesses. American society for information science and technology monograph series, Information Today, Inc (p. 342). ISBN 9781573871006.

  • Williams, G. (2007). Should we ditch impact factors? BMJ, 334, 568.

    Google Scholar 

  • Winkmann, G., & Schweim, H. G. (2000). Biomedical databases and the journal impact factor [Medizinisch-biowissenschaftliche datenbanken und der impact-faktor]. Deutsche Medizinische Wochenschrift, 125, 1133–1141.

    Google Scholar 

  • Winkmann, G., Schlutius, S., & Schweim, H. G. (2002). Publication languages of Impact factor journals and of medical bibliographic databanks [Publikationssprachen der Impact Faktor-zeitschriften und medizinischer literaturdatenbanken]. Deutsche Medizinische Wochenschrift, 127, 131–137.

    Google Scholar 

  • Woelfel, J. (1993). Artificial neural networks in policy research: A current assessment. Journal of Communication, 43, 63–80.

    Google Scholar 

  • Woelfel, J., Woelfel, J. (1997). ThoughtView version 2.0, Galileo Corporation.

  • Woolgar, S. (1991). Beyond the citation debate: towards a sociology of measurement technologies and their use in science policy. Science and Public Policy, 18, 319–326.

    Google Scholar 

  • Wulff, J. L., & Nixon, N. D. (2004). Quality markers and use of electronic journals in an academic health sciences library. Journal of the Medical Library Association, 92(3), 315–322.

    Google Scholar 

  • Xiao, H., Yuan, F., & Wu, J.-G. (2009). Factors affecting citations: A comparison between Chinese and English journals in ecology. Chinese Journal of Applied Ecology, 20, 1253–1262.

    Google Scholar 

  • Yu, G., & Wang, L. (2007). The self-cited rate of scientific journals and the manipulation of their impact factors. Scientometrics, 73, 321–330.

    Google Scholar 

  • Yu, G., Yang, D.-H., & He, H.-X. (2011). An automatic recognition method of journal impact factor manipulation. Journal of Information Science, 37, 235–245.

    Google Scholar 

  • Yue, W., Wilson, C. S., & Boller, F. (2007). Peer assessment of journal quality in clinical neurology. Journal of the Medical Library Association, 95, 70–76.

    Google Scholar 

  • Zetterstrom, R. (1999). Impact factor and the future of Acta Paediatrica and other European medical journals. Acta Paediatrica, International Journal of Paediatrics, 88, 793–796.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jerome K. Vanclay.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Vanclay, J.K. Impact factor: outdated artefact or stepping-stone to journal certification?. Scientometrics 92, 211–238 (2012). https://doi.org/10.1007/s11192-011-0561-0

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-011-0561-0

Keywords

Mathematics Subject Classification (2000)

JEL Classification

Navigation