What determines the citation frequency of ecological papers?

https://doi.org/10.1016/j.tree.2004.10.010Get rights and content

Citation frequencies of scientific articles are increasingly used for academic evaluation in various disciplines, including ecology. However, the factors affecting citation rates have not been extensively studied. Here, we examine the association between the citation frequency of ecological articles and various characteristics of journals, articles and authors. Our analysis shows that the annual citation rates of ecological papers are affected by the direction of the study outcome with respect to the hypothesis tested (supportive versus unsupportive evidence), by article length, by the number of authors, and by their country and university of affiliation. These results cast doubt on the validity of using citation counts as an objective and unbiased tool for academic evaluation in ecology.

Section snippets

Citation rates and journal impact factors

The papers used in our analysis were published in 53 ecological journals with different impact factors (Box 1) and so we examined the relationship between the citation rates of individual papers and the impact factor of the journals in which they were published. The journal impact factor describes the mean citation rate of articles published in a given journal [25] and is calculated by dividing the number of citations received in the current year (e.g. 2003) for articles published in the

Citation rates and individual study characteristics

We examined the association between citation rates and two types of study characteristic: study outcome and article length. Several studies in medicine and psychology have found that the outcome of studies with respect to the hypothesis being tested influence citation rates, with either supportive or unsupportive results receiving more citations depending on the research area 14, 15, 16, 26, 30, 31. We found that the direction of study outcome with respect to the hypothesis tested might also

Citation rates and authorship characteristics

A recent survey [36] revealed that, of the authors publishing in five leading ecological journals, only 6% were females, which suggests a gender bias among senior ecologists. The range of ecological journals examined in our survey was much broader (53 versus 5) and, as a result, the proportion of articles written by females was much higher (30%). We found that gender of the first author had no effect on the citation rates of individual ecological papers (Box 3). This is in agreement with the

Conclusions

Our report is, to our knowledge, the first attempt to examine systematically the importance of various factors associated with the citation rates of ecological studies. The correlative nature of our study and the complex nature of the explanatory characteristics used make it difficult to interpret unambiguously the ultimate causes behind the observed associations. Nevertheless, the patterns that we detected suggest that factors other than the scientific utility of a study affect citation rates

Acknowledgements

We thank Erkki Haukioja, Chris Lortie and three anonymous referees for constructive comments on the earlier versions of the article, and Ellen Valle for checking the English. The study was supported financially by the Academy of Finland.

References (48)

  • L.L. Kjaergard et al.

    Citation bias in hepato-biliary randomized clinical trials

    J. Clin. Epidemiol.

    (2002)
  • T. Tregenza

    Gender bias in the refereeing process?

    Trends Ecol. Evol.

    (2002)
  • U. Ravnskov

    Quotation bias in reviews of the diet-heart idea

    J. Clin. Epidemiol.

    (1995)
  • R.K. Merton

    Science and technology in a democratic order

    J. Leg. Polit. Soc.

    (1942)
  • E. Garfield

    Is citation analysis a legitimate evaluation tool?

    Scientometrics

    (1979)
  • E. Garfield et al.

    Citation data: their use as quantitative indicators for science and technology evaluation and policy-making

    Sci. Pub. Pol.

    (1992)
  • D. Adam

    The counting house

    Nature

    (2002)
  • R.H. Peters

    A Critique for Ecology

    (1991)
  • R.M. May

    The scientific wealth of nations

    Science

    (1997)
  • D.A. King

    The scientific impact of nations

    Nature

    (2004)
  • M.H. MacRoberts et al.

    Citation analysis and the science policy arena

    Trends Biochem. Sci.

    (1989)
  • T.A. Brooks

    Private acts and public objects: an investigation of citer motivations

    J. Am. Soc. Inf. Sci. Technol.

    (1985)
  • W.R. Shadish

    Author judgements about works they cite: three studies from psychology journals

    Soc. Stud. Sci.

    (1995)
  • N. Gilbert

    Referencing as persuasion

    Soc. Stud. Sci.

    (1977)
  • P.O. Seglen

    Citation rates and journal impact factors are not suitable for evaluation of research

    Acta Orthop. Scand.

    (1998)
  • D.O. Case et al.

    How can we investigate citation behaviour? A study of reasons for citing literature in communication

    J. Am. Soc. Inf. Sci.

    (2000)
  • P.C. Gøtzsche

    Reference bias in reports of drug trials

    Br. Med. J.

    (1987)
  • U. Ravnskov

    Cholesterol lowering trials in coronary heart disease: frequency of citation and outcome

    Br. Med. J.

    (1992)
  • D.A. Wardle

    Journal citation impact factors and parochial citation practices

    Bull. Ecol. Soc. Am.

    (1995)
  • G. Paris

    Region-based citation bias in science

    Nature

    (1998)
  • E. Davenport et al.

    Why cites women? Whom do women cite? An exploration of gender and scholarly citation in sociology

    J. Doc.

    (1995)
  • S. Baldi

    Normative versus social constructivist processes in the allocation of citations: a network-analytic model

    Am. Sociol. Rev.

    (1998)
  • J. Kotiaho

    Papers vanish in miss-citation black hole

    Nature

    (1999)
  • J. Kotiaho

    Unfamiliar citations breed mistakes

    Nature

    (1999)
  • Cited by (0)

    View full text