The art of peer review

As an early career researcher, writing a peer review can be as daunting as receiving one. It carries the responsibility for judging other people’s work, and thereby, being judged in the process. Over the last few years I’ve started to collate a few tips on how to make the peer review process easier and nicer, for all involved. Then I found Bourne and Korngreen’s 2006 paper, “Ten Simple Rules for Reviewers“, which summarises most of them nicely. So here I’ll just summarise their paper, and add a few notes of my own, with particular reference to my experience in conservation science. I don’t want to call them “rules” though, because we all know, rules are meant to be broken!

I’ve also noted a bunch of other resources at the end, which explain in much more depth what the peer review process entails.

Peer review as running a gauntlet.
Nick D Kim’s take on the peer review process, but one shared by many. We can make the process better though, and less painful for all involved. Cartoon by Nick D Kim (http://lab-initio.com/).
  1. Learn to say no (and yes). If you really don’t have time to give it a prompt and fair review, then don’t accept it – it’s not fair on the authors or the journal staff. But, I’d add also to remember that peer review is part of the job – so make sure you make time for it. To be fair, you ought to be reviewing at least 2-3 papers for every one paper you submit (or will submit). Somewhere, someone told me to maybe aim for one every 1-3 months. Of course, at first you might not receive that many requests, so I’d suggest just taking (almost) all of them – it is a great experience to see the breadth of research (and research quality) out there.
  2. Enjoy and learn. Peer review is a great way to become familiar with the upcoming literature. But sometimes you might not feel you have the interest or the experience to review a particular paper. I’d possibly suggest if you have no interest in a manuscript that is within your field, then perhaps you are in the wrong job. But if you lack the experience, this is not necessarily a problem. If you are familiar enough with the field of research, not knowing a particular technique they use is ok – they should be explaining their paper adequately enough for a broad audience (though it might require some extra background reading). Just be clear to the editor, at least, why your review might be lacking in some aspects.
  3. Write reviews you would like to receive. This doesn’t mean “Accept, no changes”, more that the criticism is constructive, and supported with logical argument. Stay neutral, logically criticise the work (not the authors), and be patient. Bourne and Korngreen (2006) note that while your comments are often blinded to the author, your name (and thus reputation) is known to the editors (often people you might like to impress).
  4. Reviewing methods are personal. A common strategy involves several readings, interspersed with becoming familiar with the journal scope and style, the topic, or methods you might need a refresher on. Your review should cover a) the fit with the journal scope, b) the novelty and significance of the topic, c) if the science is rigorous, d) if the literature cited is current and comprehensive, and e) your recommendation. Try and clearly identify the major issues, then note minor issues. Before you start though, it is useful to check if the journal has any specific questions or formats for the review. It helps the editors if you follow these, and makes it easier on you. When you get more familiar with doing peer reviews, they might take about 3 hours to do, some much more, some much less. But expect the first few to take substantially more.
  5. Your comments should lead to a better paper. The most important part of a review is to suggest how the authors can improve their manuscript. So try to make your recommendations as clear and specific as possible (and reasonable – they are unlikely to want to write a whole new paper, and often little further analysis is possible).
  6. Tailor your review effort to the quality of the paper. If it is really a manuscript that is substantially lacking, or is very much not within the scope of the journal, then it is ok to concisely, but encouragingly, detail the main issues and reject it, without spending the next 3 weeks detailing everything that is wrong with the manuscript. But, I’ll add here that sometimes language can make a great paper difficult to understand… and it is much easier to fix bad language than a bad idea! So try not to let bad communication cloud your judgement of otherwise good work.
  7. Your opinion on the manuscript is valued. Be confident, and give a clear recommendation. If it is hard – clarify the decision parameters for the editor (in the comments to the editor) – for example, suggest the paper ought to be accepted, if the editor accepts that it is sitting at twice the word limit.
  8. Maintain the anonymity of the review, if the journal requires it. There are arguments for and against anonymity. But the most important thing is that you should be willing to stand behind your review, so write it as such. Recommending the authors cite a whole bunch of your work is not just obvious, it is also obnoxious, and only serves to show that you are also unfamiliar with the literature!
  9. Avoid conflict of interest, and remember the manuscript is confidential. What is a conflict of interest in this context? Ask yourself, will you be able to give an unbiased review? Some cases where you might be biased include if the work is close to your own, e.g. done by your close colleagues (or yourself), or if its a very similar topic and approach to a paper you are preparing. Note, actual conflict (disagreement with the quality of the research or interpretation) is a useful part of the review (or subsequent comment papers).
  10. Comment to the editor. This can be used to highlight your relevant experience/inexperience in the themes of the paper, and to give your personal perspective/opinion of the paper. This can be really helpful for editors to evaluate your review in regards to others in a split decision.
  11. Keep a record. This last one is for you. You may want to refer to them later, but you’ll definitely want to note how many and what journals you’ve reviewed for in your CV. So make sure to save a record of the papers and reviews you’ve done.
  12. Ask for help. Finally, if you need, ask for help on your first few reviews, or for a second opinion if you are unsure.

For sure, the peer-review process is pretty flawed and can be perverted, particularly given the new plethora of unscrupulous open-access publishing. But I like to think that most people in conservation science, at least, are honest and do want the best for our science, and it is likely some form of peer review will remain a prominent part of science. If you have any other suggestions, or comments that might get an early career researcher get started in peer reviews, I’d be glad to hear them!

Thanks!

Resources and further reading

“Ten Simple Rules for Reviewers” by Phillip Bourne and Alon Korngreen is available, open access, from PLoS Computational Biology: http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.0020110

The British Ecological Societies “A guide to peer review in ecology and evolution” is a fantastically detailed resource, explaining the institution of peer review as well as detailing the questions you should be asking at each stage, highlighting ethical issues, and answering FAQ. Highly recommended: http://www.britishecologicalsociety.org/wp-content/uploads/Publ_Peer-Review-Booklet.pdf 

The BMC has also recently prepared a series of editorials dealing with reviewing articles with reference to statistics, randomised controlled trials, and systematic review. Details of these can be found in Alam and Patel (2015) Peer review: tips from field experts for junior reviewers. BMC Medicine 13:269

Times are changing: To get an idea of how peer review might change in the future, Nikolai Slavov discusses some of the issues with peer review, and proposes that open review or post-publication comments might be useful: “Making the most of peer review” is available at http://elifesciences.org/content/4/e12708

Interestingly, there’s a new push to give researchers more credit for their peer review. Publons is working to allow researchers to publish their activity (if not the reviews themselves) and gain credit or prizes. A write up in nature can be found here.

(Thanks to all of you who referred me to these resources!)

Advertisements

One thought on “The art of peer review

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s