The research assessment exercise (RAE) forms the basis for determining the funding of higher education institutions in the UK. Monies are distributed according to a range of performance criteria, the most important of which is “research outputs”. Problems to do with publication misconduct, and in particular, issues of justice in attributing authorship, are endemic within the research community. It is here argued that the research assessment exercise currently makes no explicit attempt to address these concerns, and indeed, by focusing attention on research outputs, may actually be fostering such ethical problems.
- Research assessment exercise
- publication ethics
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
The forthcoming 2001 research assessment exercise (RAE) will form the basis for the distribution of about £5 billion pounds to higher education institutions (HEI) in the UK.1 Research evidence and the anecdotal accounts of many editors of biomedical journals suggest that publication misconduct is widespread among the scientific community; the power and economic benefits of authorship, it is argued, are the root cause of these problems.2 In this paper, I summarise some of the key ethical considerations in the ongoing debate concerning authorship, arguably the most pressing concern in publication ethics today, and explore the possible effects of the RAE on the vexed question of authorship
RAE: motives and mechanisms
The last two decades of the 20th century witnessed an explosion of activity that was concerned initially with recording and monitoring activities in the public sector, before shifting focus to initiatives designed to measure quality and performance.3 Fuelled by a combination of national economic shortfalls, innovations in quality assurance mechanisms, and increasing public demands for transparency and accountability, a “new public management” evolved, leading to the development of what Power has called “the audit society”.4 It was against this backdrop of a need for initiatives to improve service efficiency, and to enable UK research to compete better in the global market, that the RAE was conceived. With the declared objective of promoting research excellence, the scheme aims to rate research quality for the purpose of “enabling the higher education funding bodies to distribute public funds for research selectively on the basis of quality”.5 Such streamlining of finite resources, it is argued, is necessary to allow “institutions conducting the best research to receive a larger proportion of the available grant so that the infrastructure for the top level of research in the UK is protected and developed”.5
Ratings are produced using a peer-review mechanism, on a seven-point scale, grades awarded according to the extent to which research is judged as reaching national or international standards of excellence.6 Activity is considered under certain broad categories termed units of assessment with quality assessed on the basis of predetermined criteria, most important of which are up to four “research outputs” per member of staff considered “research active”.7
Assessments have been performed every four to five years since the scheme's inception in the late 1980s. For the 2001 round, final funding decisions are to be determined on the basis of a “quality times volume” formulation, where volume relates to the number of research-active staff submitted.5 Since the reputation, funding, and in some cases, the very viability of research groups, is dependent on achievement in the RAE, institutions have understandably scrutinised closely their research outputs, which, in the context of biomedical research, almost invariably equate with publications. Inevitably, the publication records of staff have also come under close inspection, with less productive researchers increasingly being seen as a liability to the rating likely to be achieved by a research grouping.
Contemporary issues in publication ethics
Journal editors have suspected for many years that research and publication misconduct is widespread among the medical community, a view that appears to be supported by what little empirical research has been performed in the field.2 The widely held belief is that as pressure to publish increases, reflected in the often referred to principle of “publish or perish”, misconduct increases. Problems include frank fabrication of data, issues to do with failure to obtain the informed consent of participants, plagiarism, duplicate, and redundant, publication. The most common and pressing concerns facing editors however, are disputes to do with authorship.8
Who has a moral claim to authorship?
“Prints should be prized as authors should be read” reads the inscription on William Hogarth's moral engraving, A Midnight Modern Conversation, 1753.9 In Hogarth's time, and indeed until the middle of the last century, notions of authorship were well understood. For who could deny the likes of Anthony Trollope, the esteemed 19th century novelist, or Enid Blyton, responsible for many a childhood memory, their undeniable claim to authorship? Spending many hours a day, alone, seeking to perfect the human word, they truly satisfied “the writer” definition of authorship. Scientific authorship, in which writing is often the least valued role, is, however, another matter entirely.
The majority of medical papers published today are multiauthor affairs.10 A large multicentre randomised controlled trial for example, is likely to have significant input from several tens of people, including those writing the original research proposal, the principal and local investigators, research assistants involved with data collection, statisticians and health economists, to mention but a few. To muddy the waters further, a professional writer may have written the final paper. Who then has a moral claim to authorship? And does the question of authorship really matter?
To the latter question the answer must surely be “yes”, for as the anthropologist Mark Gruber reminds us, “the integrity of a body of literature is itself our society's ultimate temporal forum for negotiating life and death, suffering and wellness… the medical well-being of the society it serves is dependent on the question of who stands behind the word”.9 Derived from the Latin “auctor” and etymologically related to the term “authority”, authorship, as its roots and relationships imply, must bring with it responsibility as well as reward. At the heart of the question of authorship then, is the perennial concern of editors, researchers, and the public at large, the issues of accountability and trust for what has, and has not, been published.11 It may be argued that only those in a position to take such responsibility have a just claim to authorship, and this is indeed what is central to the consensus statement on authorship reached by the International Committee of Medical Journal Editors (see box 1).12 Judged by this standard, flagrant abuse, most commonly manifest as “gift” authorship, is commonplace among the medical research community.13, 14 Gift authorship refers to listing as a co-author someone who has insignificant input, defined as a failure to reach the criteria stipulated in box 1, to the research project. Motives vary, ranging from nepotism on the part of senior researchers, to reciprocating favours for previous co-authorships, to increasing the likelihood of publication by adding names perceived to carry gravitas. Gift authorship, as the name implies, typically brings (undeserved) credit to the recipient. Many editors have called for its elimination, appealing to researchers both on deontological grounds, that the practice is fraudulent, and on consequentialist considerations, that the practice may be something of a poisoned chalice, as was the case with Geoffrey Chamberlain's much publicised fall from grace.15, 16 Researchers have typically based their defence on the morally suspect argument that the utilitarian ethic is at work and that the end (publications leading to improved patient outcomes) justifies the means. More difficult to contend with however, is the widespread problem of researchers being unfamiliar with the operational definition of authorship, and that, among those who are aware of this definition, there is a sizeable minority who feel that the current definition is either unworkable, inappropriate, or both.17
Box 1: International Committee of Medical Journal Editors:
All persons designated as authors should qualify for authorship. Each author should have participated sufficiently in the work to take public responsibility for the content.
Authorship credit should be based only on substantial contributions to:
Conception and design, or analysis and interpretation of data and
Drafting the article or revising it critically for important intellectual content and
Final approval of the version to be published.
Conditions 1, 2 and 3 must all be met.
At least equally as serious, and the cause of much discontent, among junior researchers in the main, is the issue of “ghost” authorship.18 This refers to the practice of not citing as co-authors those who satisfy the criteria of authorship. Clearly an unjust practice, there is the wrong of usurping the aggrieved person's intellectual rights and utilising data over which one does not have a moral (and possibly legal) claim to ownership. Between both these extremes, lie the everyday tussles to do with ordering of authors, in particular the first and last position, because of the particular recognition of these positions among the academic community.19
Is contributorship the answer?
Standardisation can only work in practice if all relevant bodies, including researchers, and their employing and funding bodies agree a definition, and it could be argued, a joint strategy to deal with violations of such an agreement. No such moves are likely in the foreseeable future, which has led some editors to argue that we should no longer be thinking of authorship but rather contributorship, where, somewhat analogous to film credits, the contributions of each individual are listed separately.20, 21 Support has been forthcoming, particularly from the aggrieved,22 in the belief that such transparency is essential to restoring professional and public confidence in the scientific process.
Contributorship has been criticised via the argument that such “tinkering at the edges” is the product of a shallow ethical analysis, and if anything is likely to be counterproductive by adding yet another layer of obscurity and serving further to conceal the true nature of authorship. What then is the true nature of authorship? Scott, for example, believes that authorship is in essence a political process that involves “staking and maintaining territorial rights, colonisation and empire building”.23 Added to this dissenting voice is the concern of the reader, echoed in the tongue-in-cheek comments of Albert, who opines that editors “have become so bogged down with considerations of authorship that it never occurs to them that they might be boring the pants off any readers still hanging around”.24 Whilst moves towards contributorship are broadly to be welcomed, in the absence of widespread recognition of contributorship, this proposal is likely to have little impact on curbing the problems highlighted.
Impact of the RAE on authorship
Opinions about the scheme's success remain deeply divided (as one would expect with reputations and large amounts of money at stake), with valuable and considered debate on the ethics of introducing a mechanism that places fellow institutions in positions of rivalry and competition when what is perhaps needed is greater collaboration. There has also been useful discussion about: the relative merits of using outcome measures that are far from reliable; the possible impact of the scheme on promoting or stunting genuine scholarship; reducing gender discrimination, and facilitating a culture of meritocracy.25 It is surprising, however, that there has been no empirical research to date, exploring the impact of the scheme on issues of authorship; nor has there been any detailed critique of this relationship.26
It is of particular concern that although research outputs (publications in the present context) are the declared principal basis for assessment, the published RAE literature provides no conceptual or operational definition of authorship. Furthermore, the RAE appears not to recognise the concept of contributorship, since neither the submission guidance notes nor the draft assessment criteria circulated to review boards make any mention of this notion. How then will individual contributions to a multi-author project (where authors may commonly have been drawn from different institutions) be assessed and rated in a fair manner?
Given the lack of such clarity over authorship criteria, from a body that is primarily charged with ascertaining quality, consequentalists may argue that the scheme allows abuses of authorship to continue unchecked. Furthermore, in view of the RAE increasing pressure to publish, the scheme may actually be creating disincentives to abide by definitions of authorship currently in use. Essential to assessing quality of individual contributions, and apportioning merit and blame accordingly, is the need for panels to be able to determine which researcher contributed what. The present system of authorship makes it impossible to distinguish the person who “made the project happen” by being involved in all stages from conception to project completion and publication from the statistician or health economist who may have performed some relatively straightforward analysis and contributed and verified the final draft of the paper. Clearly these contributions are not equal, and this needs to be reflected in the relative quality ratings awarded to these individuals for this particular work. In the absence of such detail in assessment it seems likely that researchers will continue to vie for first and last positions, assuming that these will carry particular weight with the assessment panels. Such considerations may also encourage researchers to limit the number of authors, so that credit is not shared too thinly, by unjustly preventing colleagues being co-authors.
Research-active staff are allowed to submit a maximum of four pieces of work over a four or five-year period,7 those individuals for whom less than the maximum are submitted may be penalised unless an acceptable explanation for their relative lack of productivity can be provided. The RAE guidance notes make clear that committees must be willing to consider a range of research outputs and must not automatically penalise work that has not been subject to peer review. In practice, however, only a fraction of work is scrutinised by assessment panels (how many panel members would be willing to read a three-hundred page book for example?), and hence recourse to established peer review mechanisms such as impact factors (despite their limitations) to guide quality ratings appears to be commonplace.27, 28 In view of the many barriers to successful publication, including the very high rejection rates for journals with the best impact factors (90% for the Lancet for example), the pressures to submit many papers remains high; pressures likely to militate strongly in favour of gift authorship and publication misconduct.
Official guidance states that a certain proportion of returns will be audited to check for verification of information. The guidance carries the warning that institutions must be willing to provide more detailed information if requested; should they be found wanting there is the threat that this information will be relayed to the funding councils.5 In practice, these checks have been limited to ensuring that financial statements are in order and that personnel counts are accurate. No mention is made, for example, as to how attempts will be made to determine whether gift authorship has occurred.
But surely it is bizarre to claim that publication misconduct is promoted by a scheme that has as its declared aim the importance of actually improving research quality. After all, publication misconduct predates the RAE, and even if the scheme were to fold tomorrow, abuses of authorship would probably continue unabated. By limiting the number of papers that can be submitted over a four or five-year period to four, and with the very real possibility of detailed checking of the paper, it could also be argued that the scheme actually promotes responsible authorship. Any more detailed assessment, such as the scrutiny of individual contributions, is likely to be too time-consuming and expensive to justify its routine use. Furthermore, the scheme as it stands is an essential strategy to identify individuals and research groups that at taxpayers' expense are failing to produce the outputs needed to maintain Britain's position as a major player in the international research arena. Such non-productive researchers are a liability and need to be removed from the payroll, as would be the case if they were employed in many other industries. These counterarguments have their merits, though they do not detract from the central argument developed here. The ethics of an ethical RAE demands a consistent and transparent approach that allows first-rate researchers to be reliably differentiated from their less able colleagues.
The overall aim of the RAE, namely to improve research quality and excellence, is worthy and is indeed difficult to disagree with on ethical grounds. However, a scheme that proposes to achieve these aims by focusing its assessment on research output provides yet another incentive for publication misconduct, adding economic considerations to the existing rewards of power and prestige that authorship brings. It is also of concern that the mechanism to measure the outcome of publications is simplistic, failing to allow the quality of individual contributions to be independently assessed in a meaningful way. With no definition on the central question of authorship offered, and no explicit measures to safeguard against misconduct, it is likely (though far from proven) that the net effect of the RAE will be to foster problems to do with authorship and publication ethics. A radical shake-up of the RAE is proposed after the 2001 round.29 It is imperative that the new assessment model has built into it, mechanisms to ensure that the contributions of individuals can be accurately assessed and a healthy publication ethic flourishes, for assessing quality in the absence of responsibility and justice is at best of dubious worth, and at worst meaningless.
I thank Brian Hurwitz, Michael Parker and Paquita de Zulueta for their constructive comments on an earlier draft of this paper. I also wish to express my gratitude for the support provided by my NHS R&D National Primary Care Training Award.
References and notes
Aziz Sheikh, BSc, MBBS, MRCP, MRCGP, DCH, DRCOG, DFFP, is NHS R&D National Primary Care Training Fellow, Department of General Practice & Primary Health Care, Imperial College School of Medicine, Norfolk Place, London W2 1PG.