Teaching peer review

Peer review is the horror of many a PhD student who submits his or her first paper. Seeing their colleagues’ frustrations and hearing about impossible reviews nurtures a fear that painstaking work will be crushed by that one email from the editor. Therefore, a PhD’s first submission shouldn’t be the first time they come in contact with the peer review system.

peer review


This post is part of the open draft for the Research Group Leader Book
[about] [read more]


Students will only get acquainted with peer review if it’s a topic in the lab. This has two sides: reviews we get, and reviews we write.

Learning to deal with the reviews we get

One of the most important things about reviews is to talk about what they (can) do to you. If you haven’t cried your own tears, I am sure you know someone who has. Reviews can be devastating. So, it’s helpful to talk about how one can deal.

  • Read it once. If it makes you choke, save it, and leave it for three days until you go back to it. As Bradley Voytek puts it in his lab philosophy: You are not your science. You are not your science. You are not your science.
  • Attack the review point by point: brake it down into its single points. If the reviewer did not structure his points well, do it yourself. Isolate each single point that requires some action. This reduces the overwhelm of the many points that seem to be wrong with the paper.
  • For each point, assess whether and how you can address it.
  • Start with one point. Then work yourself through the list.

There appears to be a trend for editorial decisions to become more dramatic. What used to be a major revision is now often a reject with permission to resubmit. And what used to be categorized as a minor revision is now often a major one. If you’ve written a number of papers, you don’t even notice these things anymore. But first timers do. Reject feels like a killer to them. So they should learn from their supervisor that the gravity of a decision letter can only be assessed by checking whether what the reviewers want can be fulfilled (or rebutted) or not.

Learning to write reviews

It helps a bunch to have seen other reviews and to have written some reviews to be able to put one’s own reviews in perspective.

Besides, being able to give scientific criticism in an adequate manner should be one of the major aims of scientific teaching.

There are two approaches I take in my lab: discussing reviews we get ourselves, and writing reviews.

Discuss the incoming: reviews we get

There are several things to learn from discussing reviews of the lab’s papers.

  • Content: Learning to expect what reviewers want to know. This is important when writing a paper: we try to foresee what may be critical points, and answer them before the question comes up. But it’s just as important for learning to write reviews: What kind of questions appear often? What do other reviewers look for? How detailed are they in their criticism?
  • Style: I can’t stress the style issue enough. All those reviewers who yell at us must have grown up somewhere. Make your lab a place that brings up a different kind. You can discuss positive examples — reviews that are clear, respectful, concise and precise. And contrast them with negative examples — reviews that are difficult to understand, lack respect, are lengthy, and don’t tell you what the reviewer wants changed.

Discuss the outgoing: reviews we write

I once read the suggestion that all reviews should be done by PhD students, because they know what they are doing, know the latest literature, and are motivated. I guess it would be rather demotivating if the boss always passed everything down. But more importantly, PhDs should not be left alone with their reviews.

I ask students to write their first review in the second year. For the first few, I do the entire review in parallel. I read their review draft and discuss points that were difficult or things that I will add that they didn’t see. Of course, all the style points are also part of the discussion. Many journals let you enter the student’s name in a designated field. If they don’t, it’s good style to acknowledge the student in the note to the editor.

When a student has more experience, both in her scientific field and with writing reviews, I interfere less with their work. They can then decide themselves whether you should read the paper and help them, and discuss points they are insecure about.

And finally, one great advantage when students write their reviews is that they can read what the other reviewers said. In my experience, this is a really important thing. It feels great to detect your own points in someone else’s review, too. It feels great to see that you did a better, more thorough job than other reviewers. These experiences build confidence.

If you have thoughts to share, or links to good resources to teach peer review, do leave a comment!

Resources

There are many resources that deal with peer review. These are a good start for students before they write their own:

Benos, D. J., Kirk, K. L., & Hall, J. E. (2003). How to review a paper. AJP: Advances in Physiology Education, 27(2), 47–52. http://doi.org/10.1152/advan.00057.2002

Some publishers have posts on how to do peer review, for instance BioMed Central (link leads to first of three posts) and PeerJ.

Jeff Leek has published his thoughts on Github.

And I really like Bradley Voytek’s lab philosophy. It’s not about peer review. But the flair of it is exactly what will make you confident even when you get smacked on the head by one.

Photo credit: Gratisography.com

 


This post is part of the open draft for the Research Group Leader Book [about] [read more].
Please share with friends and colleagues!

Leave a comment...