In this forum, I am reading this great question (What is required of a mathematics referee?) by a user with the name mathprofessor. There is an answer by a user with the name Buffy which starts with:
Sorry, but if that's all you do, then your reviewing career is likely to be short, ending the first time you approve a paper that is revealed to have an error.
I am wondering now: How exactly can somebody's career as a reviewer end after some paper they reviewed is revealed to have an error?
Sure, the editor who assigned the reviewer may not assign them ever again, but how exactly are other editors (maybe from different journals) notified not to take them as a reviewer ever again? Is there some way the editor who knows is allowed to reveal the reviewer's identity? Or some higher authority they can talk to? Or how does that work out in practice?
Let us assume the following: If the answer is field-specific, let us assume we are talking about math. Moreover, as in the other question, let us assume there is no fraud going on -- the author made a honest (but big) error in the paper, and the referee was too sloppy in their report and did not note the error.
Additional question: Are there known cases where reviewers had to end their reviewing career because they did not notice an error? Again, I am assuming no fraud is going on.
Edit: I want to say that the user with name Buffy edited the answer in question and made a much weaker claim. This solves my confusion. Thank you very much, Buffy!
Answer
Well, not feeling obliged to review other people's papers anymore sounds like a nice deal so if you figure out an answer tell me.
Unfortunately, there won't be one. Not only do I know plenty of mathematicians who have approved papers with errors in them but I've known a number of mathematicians who were well-respected in the community despite the fact that everyone figured there was something like a 1 in 8 chance that the main claim of any paper of theirs would turn out to be fatally wrong. I can't say for certain that I've done it since obviously I would have flagged the error if I'd seen it and errors are so common that no one even mails the author for non-critical errors and a reviewer likely won't even be told if a fatal flaw in the proof is later found.
Hell, I've been halfway through extending people's published work only to email the author a question and find out that the proof is in shambles and they are struggling to find a patch. So it's literally the exact opposite situation where the total absence of errors is what would be unusual.
Indeed, I don't know anyone who has reviewed more than one or two math papers who hasn't approved a paper with an error. Studies suggest that something like 80% of published math papers contain some form of error (that's not a fatal error but still). Sorry if I don't remember the source on that study but I'm sure if you google it you can find the relevant info.
Note that I think this is a compelling reason that mathematicians should completely abandon the blind peer review process in favor of something like a math social network with up and down votes. Yes, still have two independent individuals read the paper and submit comments and demands for clarification but don't throw out all that the reviewers have learned by collapsing the judgement down to accept/reject. The mathematician I was thinking of with the frequent errors still did good work but often pursued proofs that were particularly knotty and difficult to check. The reviewers were well aware that certain parts of these proofs raised yellow flags but they couldn't specifically show there were any flaws and, since tenured professors aren't always willing to break things down to a tedious level or formality, I agree publication was the right call. However, a math social network could have passed along the reviewer's sentiment that they still have some reservations about the argument in part X moreover, the initial review will matter less since the accumulation of comments and the ability to use all professional mathematical readers of the paper as a crowdsourced continuing review will do more to help us build a mathematical edifice we are sure is true.
No comments:
Post a Comment