perception - How can you test what color different people perceive?

If I would show someone a yellow object and ask them, "is this object yellow?" That person would say "yes".

But I could never know if my perception of the color yellow is the same as that other person's.
Because he or she could actually be seeing, what I know to be the color green.

But then tells me that its the color yellow because that has been taught to him or her from young age.

So how can you test if people are really seeing the same color?
(originally posted this question on physics.stackexchange but was advised to try it here) https://physics.stackexchange.com/questions/48731/how-can-you-test-what-color-different-people-perceive

Answer

One way we can get evidence qualia are the same or very similar for different people is by reactions to it, beyond just the word.

For example, beyond the word "pain", we have other strong reactions to pain. So nobody suspects other people might experience pain as pleasure and vice versa. Obviously not! There are no obvious signs for colour qualia, so it makes sense to suspect some people experience red as blue and vice versa.

Still, there are weak reactions to colours, like sometimes certain colours are associated with certain emotions. But, this is very weak evidence and could easily mismeasured, or be bound up with culture rather than biology.

We could test it: take pairs of identical twins at birth (so that they haven't learned words for colours yet), re-wire optical nerves of one of each pair so that red and blue receptors were switched (and perform a placebo operation on the other one, so they don't know who is who), and see whether there is any statistically significant change in the attitudes to different colours or the like as they grow up.

The experiment has various technological and ethical problems :-) But it could be done in principle, so to me this shows the question isn't meaningless, just that it's hard to find the answer.