peer review - What makes a research paper low-quality?

As a reviewer, what are some obvious signs to you that a paper is low-quality?

Answer

Kind of a broad question, but I'll play along.

It is easier to say what makes a paper high quality: I learn something new and interesting by reading it. A twinge of jealousy -- as in, "Gosh, I wish I had thought of / done this" -- is a good sign there. The absence of all this makes the quality of the paper not too high.

There are some papers that don't teach me anything new and interesting that I nevertheless feel are of medium (publishable) quality: sometimes I don't really understand the value of a result but can't claim (or can't justify a claim) that the results will not be of value to someone else or someone in the future. Sometimes the results are known to me in some form but don't appear in the literature. Sometimes the results are novel when viewed from the knowledge base and comfort zone of one subfield but less so from another (or perhaps from the vantage point of a true expert -- in a field where true expertise is rare), and these can be worth publishing.

Probably what remains is low quality. I am inclined to object a bit to the framing of the question in terms of "obvious signs": as a referee, I aspire to read a paper carefully enough to be confident in my evaluation of its quality. Obvious signs may inform my preliminary impression, but I will read further to confirm. But here are some things that would make me evaluate a paper as of low quality: as I go down the list, the quality decreases:

• The results of the paper already exist in the literature. Usually this means the author was not aware of them.


• The results of the paper already exist in the literature, in a better form than in the author's paper. Thus publishing the paper could actually be a step backwards.


• Same as the above, but the current paper omits a key technique or perspective that is part of the contemporary understanding of the topic. For instance, there are some expository papers, and these do not need to contain new results or even the sharpest possible form of the old results it exposes, but the author needs to have at least the level of mastery of the topics being exposed as an advanced student in the area, or the exposition could have a negative effect.


• The results are faulty, either in ways that are not easily corrigible, or are easily corrigible but when corrected yield absolutely nothing new.


• The results look fishy, but the exposition is so obscure that it is a chore to tell right from wrong from basically-right-but-highly-garbled from totally absent.


• The author cannot or will not make a clear distinction between what is attained in the paper and what is desired to be attained. For me and many others in my field (mathematics), this is certainly the worst. I saw an especially good example of this recently: a paper posted this month on the arxiv begins (immediately following an abstract which is at turns obscure and vacuous) by stating "Registration contains colored markers:" A red dot stands for "a fact which is not proven at present or an assumption". A yellow dot stands for "the statement which requires additional attention". A green dot stands for "statement which is proved earlier or clearly undestandable [sic]". And then immediately following that, still on page 1, there is a "THEOREM", which is a statement of the Riemann Hypothesis (for those who don't know, this is arguably the single most important unsolved problem in all of mathematics; inarguably solving it will get you $1 million)...marked with a red dot. Then we have "PROOF", followed by 28 more pages of colored-dotted mathematics, which I spent a minute flipping through out of sheer morbid curiosity. But if I were a referee then, rather exceptionally, this first page would have been all the sign I needed to be confident of the low quality of the paper.