As the title says.
My background is in Economics/Finance(mostly), many topics in those fields (and I am sure other fields) require fairly complicated programming, enough to where one can easily screw something up. How do Academic journals defend against results that are generated by bugs?
As far as I understand nobody ever sees my code, it could be hundreds or even thousands lines of garbage code without a single function in it (not to mention no unit tests) laced with bugs, and I kept "fixing" things until my results "made sense", and then happily reported them. How could a journal tell that my results are trash?
Answer
Journals never make any guarantees regarding the validity of the content published in them, though this may seem to be implied. Ideally, errors are caught during the review process. Note that this problem is not specific to programming bugs, subtle errors can occur in all kinds of settings including physical experiments. It is important to always be critical about any results in any article, even highly cited articles in top journals (though these are less likely to be wrong, they still might be).
Letters to the editor are not uncommon in my domain when questionable results are published. In a worst case scenario, published papers can get retracted after the validity of their results has been formally rejected. Retractions for this reason seem to be fairly rare, though.
This is one of the reasons why reproducible results are so important. If several independent researchers seem to reach similar conclusions, they are likely correct.
No comments:
Post a Comment