MVG says (Ch 12.1, p.311):
It is clear, particularly for projective reconstruction, that it is inappropriate to minimize errors in the 3D projective space, P3. For instance, the method that finds the midpoint of the common perpendicular to the two rays in space in not suitable for projective reconstruction, since concepts such as distance and perpendicularity are not valid in the context of projective geometry. In fact, in projective reconstruction, this method will give different results depending on which particular projective reconstruction is considered - the method is not projective-invariant.
On the second thought, I realized that the (minimizing the distance between the backprojected rays) idea might be equivalent to minimizing the distances between the points and corresponding epipolar lines.
Then... why the trivial solution does not occur in the epipolar case?
