publications - Is it ethical to withdraw a paper after acceptance in order to resubmit to a better journal?

I think the title asks the question, but once again in more words:

Suppose you submit a paper to a journal and it is accepted by the journal. By this I mean that the journal tells you that they have decided to publish your paper but you have not submitted or examined final proofs and -- especially -- not signed any paperwork allowing the journal to publish the paper.

I wonder how people feel about the ethics of deciding not to go through with the publication because you now feel that the paper could be published in a better journal? This could happen either because of outside feedback you received in the meantime or because of feedback from the referees/editors of the journal itself. (Added: In case this was not clear, I am assuming that upon submission one was seeking to publish in that journal conditional on not learning that one has "shot way too low".)

As far as I can see, it is absolutely legal to do this, so I am not interested in the legality of it, but rather its ethics and ramifications as an academic practice. I also think I will get better answers if I do not telegraph my own feelings about this; I will be happy to document them later on. Let me just say (i) this is a hypothetical question, but (ii) based on my own experience it is not ridiculous that it might be in the interest of the author to engage in this practice, if it happened to be kosher to all parties involved.

Note: I had previously asked a related question, at the time wondering whether it would be better asked separately. Based on the way things have gone thus far, I now think it is better to post this followup question separately (not necessarily right away).

Answer

There are certainly cases in which withdrawing a paper after acceptance is reasonable, for example if the author feels misled or mistreated by the journal (as Benoît Kloeckner points out in his answer). However, when the journal and referees have behaved blamelessly, it's hard to justify unilaterally withdrawing the paper after acceptance. When you submitted the paper, you implicitly agreed to publish it there if accepted. This is part of the research community's norms: a submitted paper is a request for publication, not a request for the option to publish.

Of course it's not a legally binding promise, or even a particularly grave moral promise, but it's still not something you can reasonably violate without a compelling reason beyond self interest. I don't think this is even worth considering except in extraordinary circumstances, such as a paper that was submitted in good faith but turned out to be far more important than the author could have foreseen. Fortunately, there's a simple solution in these cases, namely asking the editor for permission. It's a little awkward, but certainly less so than just going ahead and withdrawing the paper without asking, and an apologetic explanation can help. If there's a truly compelling reason to withdraw the paper and resubmit elsewhere, then there's a good chance of getting the editor's blessing, which would resolve the ethical issues. If, on the other hand, the editor disagrees with the reasoning, then the author is stuck. But it's better to be stuck publishing in a low-prestige journal than to do something inappropriate or unethical, and being unable to convince the editor is a bad sign regarding the ethics.

This is a special case of a broader principle: if you believe unusual circumstances justify behavior that might otherwise be considered unethical, and there's someone who could in principle grant permission, then it's generally better to ask for permission than to take action unilaterally.