publications - What should you do if you cannot reproduce published results?

I am trying to reproduce published results in a paper. Those results come from numerical simulations. The original authors and I do not use the same software, and theirs is proprietary (I don't have access to it). I have tried to reproduce their results, and it works qualitatively but not quantitatively: the differences between their results and mine on typical quantities of interest are between 2 and 5 times the expected accuracy of the method.

So far, I have communicated with the original authors, trying to clear out all possible sources of error I could think of (checked that I got the tricky parts of the algorithms right, checked that the “usual” parameters that were missing from their paper had the “usual” values, everything I could think of). They are forthcoming enough, and reply to my questions quickly, but it's clear they don't want to invest time in doing any serious follow-up on their side. And without access to their software, it appears I'm stuck.

Now, my question is on how to proceed. The “ideal case” for unreproducible results is to make a detailed analysis of how and why they cannot be reproduced, and possibly find out a source of failure (or at least plausible issues). This advances the field, and is probably publishable (not in a very high-profile journal). Here, this is not possible.

I have, however, nice results that I have obtained (extending their work far beyond what was already published), and if I didn't have these differences with their paper, it would make a very attractive paper. What can I do with those? Is it possible to publish them, merely noting the different with their paper without more comment? Or are my results simply unpublishable? I welcome any comment, especially from people who have found themselves in such an uncomfortable situation!

Answer

Just publish. Publish your attempts to replicate the findings, documenting the discrepancies, together with the nice results you've obtained by extending their work. Consider sending a draft to the original authors for their comments.