Reviewing for journals

Reviewers are the most important people in the whole academic publishing process. They are (theoretically) unbiased, impartial and are there to make sure the paper is correct, everything is clear and the science is of high quality. Reviewers are doing the work for free (or occasionally reduced publishing costs) on top of their usual work, so there is often a struggle for editors to get reviewers so there will come a time where all young academics eventually get asked to review for journals. The first time I got asked was a bit of a shock, and I turned down the first couple as I didn't feel qualified enough to review the papers. Looking back I probably could have done one of them, but the other I definitely wasn't able to do to a high standard. I'm still a newbie when it comes to reviewing having only reviewed for 6 journals so far (and turned down a couple of others).

The review process is much the same across journals (at least for the few I've done) and follow the following format:

A few sentences summing up the paper showing they have at least read some of it. e.g. The authors of this new paper on tyrannosaurs, using new phylogenetic methods to prove that tyrannosaur arms aren't really small. That sort of thing but obviously with some more details about all of it.

Next is major issues and this is normally in paragraph format. What is really wrong with the paper? What analyses are missing or are the analyses run badly? Is there missing data that means you cannot be sure of the results?

Then comes minor issues. How detailed you get depends on the journals and your willingness to catch all typos/grammar issues. This is line by line errors/typos/grammar/missed references etc. Those sort of things. E.g. P4 ln 10: Missing ref from Author, year (incredibly useful to also give title of the paper too if not the full reference. Amazing how many people have multiple papers from the same year).

There is also normally a section to the editor where you put in your comments that are not seen by the author. Is it novel and high enough impact for the journal? Are there things you cannot say about the paper to the authors or issues that need raising to the editor (e.g. plagiarism)? Increasingly journals are moving towards a series of tick boxes instead of this section, but many still have it.

What advice would I have for any person just starting to review?

1. Review a manuscript with a supervisor or colleague first to see how it is done. At university, we actually did reviews for published papers which was very informative. To review something for a journal, you've probably published before and seen your paper reviews to give you an idea of how it is done.
2. Make sure you are qualified to review the paper. Does the title/abstract read like something you can fairly review? If not, don't be afraid to turn it down, but be prepared to suggest some other reviewers that might be able to do it.
3. Read the manuscript, scribble down notes on things that immediately catch your eye, then walk away and come back to it. Big errors are easily caught, but taking some time away helps you think over aspects of the manuscript that you might miss first.
4. Remember this is someone's work, and be professional. There are huge swathes of the internet devoted to demeaning insults in reviews about the manuscript (e.g  this particularly famous tumblr) that are neither helpful nor clever.
5. At the same time, don't be afraid to be harsh. If something isn't right, call the people out. If the data doesn't support the conclusions say it.
6. But, don't be that reviewer. You know, reviewer 2. The one who hasn't read the paper fully, is pedantic about nothing, and writes a review that is longer than the paper because they've had a crappy day and need someone to take it out on.
7. Be helpful in your reviews! Sometimes people are working across fields. I am one of those who works on biomechanics, phylogenetics, physiology and anatomy in a vast array of groups. I will not know as much as a specialist in any one of those fields so despite my best efforts may miss papers, or might not have read the newest methods that came out last month.
8. Do it in the length of time requested. We've all sat around waiting for reviews (see this earlier post), and early career researchers in particular need published papers to advance their careers. Don't sit on your reviews until the prompts start arriving (I've not found out what those emails are like yet). Most journals give 3-4 weeks to review the paper although a few are 14 days. If you can't make that deadline, don't take the review.

Hopefully that gives a quick insight into the review process, and help anyone just starting. There undoubtedly are lots of other opinions on how exactly to do a peer review, although I'm optimistic that most of those will be the same as those stated above. If you have any suggestions of things to include/do differently/methods that work for you, do let me know and I will, as always, happily add them to the list.