software - Why are papers without code but with results accepted?

I just started reading some papers (Computer Science, specifically Computer Vision) and thought "now let's look at the source code" and was quite astonished that most of the papers don't have any source code implementing the described methods to look at, while claiming some performance or being better than other papers.

How do these papers get accepted in journals / conferences? Do people have to submit their source code privately to the reviewers at least, so that they can reproduce the experiment if possible.

Do most journals / conferences just "trust" that people who submit the paper really implemented the theory and got those exact results?

I always had this idea that any experiment should be reproducible by others else it's not scientific justified. Been wondering about this the last few days.

Answer

For me it seems that the reasons are two:

• the belief that code is only a tool, a particular implementation being secondary to the idea or algorithm,


• the historical residue (it was unpractical to print a lot of pages; especially as no-one could copy-paste it).

Additionally:

Moreover, things related to current incentives in academia (where publications, not code, are related to one's career possibilities):

• sharing code may mean risk of being scooped (instead of milking the same code for years),


• cleaning up code takes time, which can be used for writing another publications.

Do people have to submit their code privately to the reviewers at least, so that they can reproduce the experiment if possible.

Typically - not. If the code is not made public, almost for sure no reviewer have checked its existence; much less - correctness.

However, many scientists are starting to notice the problem (and they see how open source culture flourishes). So, there are new initiatives addressing such issue, like Science Code Manifesto:

Software is a cornerstone of science. Without software, twenty-first century science would be impossible. Without better software, science cannot progress.

Or e.g. this manifesto. Try to search for reproducible research or look at things such as knitr for R and this intro to IPython Notebook, or read about using GitHub for science. And it seems it is taking off.