The question of two or more people “seeing” a particular dress differently must be based on some external factor(s). The medical scientists consider the lighting, and the cones and the rods in the eyeballs. But, maybe its not the actual subject; but, something in the mind of the viewer.
Rather than the dress, however, why not focus on the greater issue.
The key point here could be more one of perception, rather than of color. For instance, our race, income class, political leanings, past life experiences, etc, might cloud how we perceive things. Such external factors could explain why: blacks view the recent killings of black men by white police differently than some whites do; Jews and Muslims each believe that the other side is responsible for the violence in the Gaza Strip; and the fourteen hundred years of violence within Islam–between the Sunnis and the Shias--is perhaps classic example of mistrust gone awry.
The problem of people coming to different conclusions, when given the same set of facts, is certainly beyond my comprehension. So, I am just raising the point here, and I am happy to allow the experts to keep working to resolve the difference.