publications - How to publish superseding results without creating enemies

While working on a research paper with a coauthor, I found an unpublished paper in arXiv and discovered out that the technique we developed in our paper can be used to improve a result in the unpublished one. Specifically, we can design a superior algorithm to solve the same problem solved in the arXiv paper. If we publish our results, then the author of that arXiv paper will have a hard time publishing his paper, since some journals will not consider his algorithm once a better one is available. Of course, we cite the arXiv paper and there is no law against publishing improved results, but this may harm the author of the arXiv paper. Is there a way to publish the new result without creating enmity?

One potential option could be to combine the work and submit a joint paper. However, the papers do not "mix" very well since our paper is mainly about a different technique - the improved algorithm is only a usage example of it. Combining the papers will create a very large and inconsistent paper.

Another option is to wait until his paper is accepted for publication and only then publish our paper with the improved result. However, it seems unscientific to withhold results that we already know.

Answer

I disagree with the premise of your question. “Publishing superseding results” is basically the same as what’s known as “publishing”, since all papers build and improve on the existing literature in some way and push some older work slightly toward obsolescence or irrelevance. The extra twist in your situation that you are improving on unpublished work is of hardly any consequence and simply not worth worrying about, for the following reasons:

1. 
   

   As others have said, the authors of the arxiv preprint still have precedence and will get credit for making their contribution at the time they did, before your subsequent improvement was discovered. Referees and journal editors should (and almost certainly will) take that into account, within reasonable limits.

   
   


2. 
   

   In general, when you improve on earlier work you show that it is interesting and relevant enough for other people to follow up on. This is actually flattering to the authors of the earlier work, even if you imagine that it is unflattering (it’s also true that an improvement can sometimes portray earlier work in a slightly unflattering light, but my point is there would still be a separate flattering aspect that I think you are ignoring, and which in most cases will far outweigh any supposedly unflattering aspects).

   
   
   


3. 
   

   The publication status of the earlier work is simply not your concern. It is up to individual researchers to submit their work for publication and otherwise promote it in a timely manner. If they fail to do so, they have only themselves to blame, and they should not expect others in the scientific community to delay their own follow up research, at the cost of hurting their own careers and slowing down scientific progress, out of pity or charity.

   
   

To summarize, I think the idea that you will make enemies by publishing honest work that you did in a STEM field, regardless of the specific circumstances, might make for a cute plot element in a satirical TV show or novel about academia, but is not a realistic thing to worry about in real life.