There’s a lot of comment floating around about the peer review process. For readers who aren’t academics (I hope that there are some!), peer review is the process whereby researchers submit their work to scholarly journals, which then send it to scholars in the field to evaluate, before a decision is made on whether or not to publish. Reviewing other researchers’ work is part of our service to the whole research enterprise – almost all scientists review other people’s work, as well as having their own research reviewed. Reviewing can be rewarding, but it is also often challenging. Being reviewed is always a bit stressful, because one is being judged and the criteria for judgement are not always obvious. Owing to a general lack of clarity about the purposes of review (is the aim to improve the work or to keep “bad” science from getting published?), and lack of information about how an editor will use reviews to make editorial decisions (do we have to do exactly what the reviewers say to get the paper published? are the reviewers making the decision or the editor?), many people are not very happy with the peer review process (although I suspect most researchers are less unhappy about how they review other people’s work than how other people review their work). I’m not going to rehash this argument because I just read a really great post on the topic over at Small Pond Science – check it out.
I suspect that one reason many of us don’t think the peer review process is working well is that we get most of our information about it from our own experience. It’s difficult to assess how knowledgeably or fairly your papers are being reviewed when you only ever see reviews of your own work or hear complaints from others about how their work was revealed. As an editor, I do get to see mostly constructive reviews, but I suspect the authors’ reactions are more negative than mine, especially their initial responses (I don’t think anyone can respond happily to criticism of their work, no matter how justified or gently phrased). I do try very hard to be fair and to word my comments positively. Unfortunately, it is certainly true that reviewers are sometimes harsh, rude, or unfair in their criticisms. This is especially hard on student authors. About 5 years ago, a review came back on the first paper ever submitted by one of my students. It started out by noting that our study species, Xylocopa virginica, is indeed fascinating, more research on carpenter bees is needed, etc., which was very nice. The next comment definitely wasn’t: the reviewer described us as “incompetent”, the evidence for this being that we had done our observations in June rather than February. Apparently the reviewer was unaware that in eastern Canada, carpenter bees (or almost any insects for that matter) do not fly in conditions more commonly associated with ice skating and hypothermia than with pollination and brood-rearing. The grad student who wrote the paper was stunned and justifiably upset, and I was pretty mad. It was the first and only time that I have challenged an editorial decision – eventually the paper was resubmitted and published, with an extra figure demonstrating that in southern Ontario, spring bee emergence begins in April or May. I wish I could find that review and post it, together with a scathing rebuttal.
So instead, I’ve decided to emulate some other scientists in posting the reviews of my papers, together with my responses to the reviewers. I hope that my students and others will learn from them. I hope some of our undergraduates read these comments and realize that even professors are subject to critique and revision (lots and lots of revision!). I really think that the more open the review process is, the more constructive (and less prescriptive) it is likely to be. I think it is also time that I start signing my reviews.