by
|
by |
|
| This article will appear in a forthcoming issue of Trends in Ecology & Evolution. | |
Abstract
Scientific journals receive many more manuscripts than they can publish. Although acceptance depends on the merit of the work, does it also depend upon the personal biases and preconceptions of the referees and editors involved? "Irrelevant" issues, such as a person's status, sex, and institutional affiliation, do influence the evaluations of their scientific work [1-5]; however, another potential bias involves the taxonomic identity of the study species. Scientists who study "unconventional" or "npopular" study organisms often complain that their papers are rejected for publication because they are seen as lacking general interest, whereas equally narrow studies on "popular" or "model organisms" are accepted. The alleged mechanism generating this bias is a simple one: researchers are most interested in papers on their own study organisms. Thus, papers based on model organisms will be handled more sympathetically during refereeing and editorial decision-making. By contrast, a manuscript describing studies on an unpopular organism will often be judged by people who do not work with that kind of organism, and who will thus see less general interest in the study.
This complaint predicts that, to be accepted for publication, a paper based on unpopular organisms will need to be framed in broader conceptual terms than will a paper based on model organisms. For example, a paper on birds might be accepted even if it defines its aims only in terms of bird biology, whereas a paper on snakes would be rejected as being of too little general interest if it were focussed only in terms of snake biology. Thus, papers on unpopular organisms will be judged "of general interest" only if the work is well embedded in theory - a filter that will apply less strongly to papers on model organisms.
Such a difference should be evident in the introductions to published papers. We surveyed 1,171 papers published in 1992, 1996, and 2000 in nine leading journals whose ISI impact factors in 1997 were >1.8, including both American and European publications and spanning the range from behavioural through to evolutionary ecology (Table 1). We selected only articles with a discrete "introduction." Displaying our own chauvinism, we arbitrarily restricted the study to papers on vertebrates.
Are Some Taxonomic Groups Strongly Over Represented?
Our null hypothesis was that vertebrate classes that are more species rich should feature more commonly in the papers surveyed. However, although there are more than twice as many ectothermic species (fish, amphibians, squamate reptiles, turtles and crocodilians) as endothermic (avian plus mammalian) species (∼31,000 versus 13,000), >71% of the papers analysed dealt only with endotherms.
Birds were highly over-represented (n = 513; 44% of papers versus 20% of species), as were mammals (n = 320; 27% of papers versus 9% of species). Other groups attracted less attention than was expected given their species richness: fish (n = 165; 14% of papers versus 48% of species), reptiles (including lepidosaurs, turtles and crocodilians: n = 87; 7% of papers versus 14% of species), amphibians (n = 85; 7% of papers versus 9% of species).
How Can We Measure "Generality" of an Introduction?
We scored the length of each introduction (number of printed lines) and the "line of first mention" of the study organism. That is, how far into the introduction is the study species (or higher lineage) mentioned for the first time? A more conceptually framed introduction will develop ideas before mentioning taxa, thus delaying the first mention of the study organism.
As expected, "narrow" introductions (those lacking even a single sentence devoted entirely to general concepts) mentioned the study organism sooner than did "general" ones (ANOVA, F1,1167 = 221.68, P <0.0001); the line of first mention correlated with the journal's impact factor [i.e. journals with higher ISI impact factors in 1997 exhibited longer "delays" to first mention of the study organism (n = 9, r = 0.80, P = 0.009); and the line of first mention is typically much later in concept-oriented journals than in taxon-oriented journals]; and the line of first mention has increased over the last decade, as have rejection rates of high-impact journals. These patterns suggest that the line of first mention does reflect the degree to which the introduction is framed in conceptual terms.
Are Endothermic Study Organisms Mentioned Sooner and More Often?
On average, the Class of the study organism was first mentioned at line 16 of the introduction in 833 papers on endotherms (mammals and birds), versus line 32 (on average) in 338 papers on ectotherms (reptiles, amphibians and fish: one-factor ANOVA, F1,1169 = 122.8, P <0.0001; figure 1). Average values for line of first mention did not differ between the two endotherm classes, or among the three ectotherm classes (Fisher's PLSD, P >0.18 in all comparisons). However, the three ectotherms all differed significantly from the two endotherms (P <0.0001; figure 1). Thus, organisms from more popular Classes were mentioned earlier (regressing line of first mention versus "% of papers devoted to that Class," n = 5, r = -0.91, P <0.035), a difference in position that was evident in all surveyed journals. A two-factor ANOVA (with the factors ectotherm/endotherm and journal, and the dependent variable position of mention of study organism) confirmed that study organisms were mentioned earlier in some journals than in others (F8,1152 = 4.15, P <0.0001) and that endotherms were mentioned much earlier than were ectotherms (F1,1152 = 54.63, P <0.0001), with no significant interaction term between these factors (F8,1152 = 1.04, P = 0.40). To control for differing lengths of introductions, we also calculated the line of first mention as a proportion of the length of the entire introduction. The result was unchanged.
Endothermic study organisms were cited more often in their relevant introductions than were ectotherms in theirs. The means (adjusted for length of the introduction) were 6.58 mentions for endotherms and 5.44 mentions for ectotherms (F1,1021 = 4.99; P <0.026).
Are Endotherm Papers Framed Less Broadly?
We classified 25.2% of the endotherm introductions as focussing exclusively on the study organism; this proportion was only 6.5% for ectotherm studies (χ2 = 52.7, df = 1, P <0.0001). Therefore, ectotherm papers were focussed more broadly than were endotherm papers.
Have These Patterns Changed through Time?
Because we sampled papers from 1992, 1996, and 2000, we could look for any temporal shift in the generality of introductions. No such change was evident. The proportion of narrow introductions has decreased (figure 2a), but the taxonomic difference remains (figure 2). The proportion of ectotherm-based studies relative to endotherm-based studies has been stable (29% versus 71% in all three years).
Are there Alternative Interpretations of these Patterns?
There are many reasons why research focuses on model organisms [6], and the over-representation of endotherms in ecological and behavioural studies is unsurprising. More worrying is whether this taxonomic bias prejudices the decisions of editors and referees. The publication differences between papers on ectotherms and endotherms are very clear (figure 1), so the only plausible challenge to this conclusion involves interpretation. In particular, do attributes of published introductions (line of first mention of study organism, number of times that study organism was mentioned and the presence of conceptual phrases) estimate the degree to which a paper is framed in terms of ideas, versus attributes of the specific study organism? In keeping with our interpretation, the various measures were intercorrelated, and showed patterns that we would expect of a measure of conceptual generality.
Are these differences in generality of approach caused by bias from editors and referees? Alternatively, papers on "model organisms" may not need to include as much conceptual justification, because the significance of the new data is already well understood. However, this interpretation predicts that introductions to endotherm papers should be briefer (which was not the case) and does not explain why the endotherm papers frequently mention the study organism, and rarely mention any general concepts. Thus, "taxonomic chauvinism" offers the likeliest explanation for these patterns.
How Can We Redress this Bias?
We doubt that an "ornithological Mafia" has conspired to suppress other disciplines. Herpetologists are equally passionate about their study animals, and a brave new world in which reptiles replaced birds as model organisms would see a reversal, rather than a disappearance, of the existing biases. The personal interests of ornithologists and mammalogists have influenced the structure of published papers only because their study animals have dominated ecological research, and thus people interested in these organisms have come to dominate the ranks of referees and editors (even those dealing with papers on other kinds of organisms). Similar prejudices probably apply to all researchers.
Most professional scientists acknowledge that their personal proclivities influence their scientific judgements. As fallible humans, we are often unaware of the nature of such prejudices [2,3]. Our data highlight the need for editors and referees to consider their own biases - including their level of interest in different kinds of organisms - when they evaluate manuscripts during the peer-review process. None of us are immune to such prejudices, but we need to be aware of their influence on processes that we optimistically contend to be "objective" evaluations of the quality of science. Our study suggests that such decisions (whether they involve publications, grants or promotions) should be made by groups of people that encompass a diversity of taxonomic and conceptual expertise.
Snakes: A New "Model Organism" in Ecological Research? - examines the growing popularity of snakes as study organisms. From Trends in Ecology & Evolution, 2000, 15:6:221-222. Full text available from BioMedNet.
Sanger Centre Model Organism Database - includes links to sites pertaining to some lesser-known model organisms.
Peer Review: Do Studies Prove Its Effectiveness? - offers highlights from the Fourth International Congress on Peer Review in Biomedical Publications. From the October 29, 2001 issue of The Scientist.
Peer Review and Quality: A Dubious Connection?, NSF Scores Low on Using Own Criteria, and NIH Wins an Exemption From HHS Peer-Review Overhaul - three recent articles from Science magazine. Registration required to view full text.
Peer Review Process - a collection of some of the best articles on the peer review process.
Masking, Blinding, and Peer Review: The Blind Leading the Blinded - an Annals of Internal Medicine editorial on the peer review process.
Related HMS Beagle articles: