As a young scholar I frequently struggle with knowing the extent to which I should be critical of particular components of papers I am reviewing. I can formulate a rough hierarchy between major concerns (things that need to be changed or this shouldn't be published) and relatively minor concerns (things I think would improve the manuscript, but aren't substantive enough to affect whether the paper is publishable or not).The question here concerns the latter, in essence I don't know where the cut-off in reasonableness should be for minor concerns.
For one example, being curmudgeonly I would say bad graphics are the norm rather than the exception in my field, although they aren't frequently so bad I can't figure out what the author is trying to say. Are minor critques of graphs appropriate (e.g. your gridlines are very obtrusive, the aspect ratio of your chart isn't appropriate, your colors/patterns are hard to distinguish, you should use a dot plot instead of a stacked bar graph, etc.) Frequently my suggested improvements would be somewhat arbitrary though, so I frequently hesitate to give such advice.
Is there any advice to guide the role of the peer-reviewer? Another side question too, does the scope change if I'm doing this for a colleague versus as an anonymous reviewer for a journal?
Answer
I think everything you're asking about is within the scope of the reviewer's role. Generally speaking, I divide my review up into three sections:
Summary: A summary and free-form critique. Here I communicate what I think the "gist" of the paper is about, its strengths, and offer some weaknesses that might exist in the paper as a whole. For example, if I think the authors were slightly too timid in not offering a interpretation of the data, or if they've missed or glossed over some major point.
Major criticisms: These are things that must be fixed in order for me to consider it a publishable paper, and if I have the chance to review the revisions (some journals do this), things I expect to either see changed, or have very good arguments for why they're not. Stuff in this category includes:
- Flawed or inappropriate methods
- Major intuitive leaps that aren't supported by the data or analysis
- Study design problems that need attention
- Major failings of interpretation
- Journal specific problems, like failing to properly report your protocol to established standards for a clinical trial, or the inappropriate use of p-values in some journals.
Minor criticisms: These are all things that are essentially "The advice of someone reading your draft with a critical eye". Importantly, these are things where, if they all made it in, while I would possibly be annoyed, I wouldn't be upset that the paper hit the press. This does include things like advice on graphics (my pet peeve is graphics that are unintelligible when printed), missing citations, etc. Things that are above the level of a copy editor, but aren't going to move my decision on the paper one way or another, unless there are a lot of them.
Occasionally I'll put in one or two small copy editing notes if something jumps out at me (insure vs. assure vs. ensure, etc.)
The cut-off I use is "Will this irk me when I see it in print, and will I think less of the authors that produced it?" To use your example of bad graphs, yes, this would annoy me, and it results in a less usable finding than one with the appropriate graphs - just like garbled language in the Results section might. If its minor stuff, like a turn of phrase I wouldn't have used, or a slight fondness for run-on sentences? That falls below the radar.
As for friends vs. anonymous reviews, I think the scope does change slightly. For a friend, you're helping them polish a paper - I think a great many more things fall under that umbrella, including things like "That's really not what a semi-colon is for" or fiddling with the graphics parameters on a plot. For a reviewer, you're one of the last gatekeepers before this goes out into the world - but you aren't an editor. Your focus should be on the research, and the appropriate presentation of it, unless the errors are so bad as to impede one of those.
In either case, you should be polite.
No comments:
Post a Comment