I am trying to extract camera orientation from the modelview matrix.
I understand that in the 4x4 modelview matrix like below, the upper left 3x3 matrix represents rotation and the right-most column represents translation.
1 0 0 | 0
0 1 0 | 0
0 0 1 | 0
0 0 0 | 1
I am currently in a funny situation where I need to extract eye position, target position and the up vector from the modelview matrix for use in gluLookAt().
The reason being that the modelview matrix is set by one framework and I need to pass this to another framework, which does not allow me to load matrix directly but can only manipulate it through the camera’s position and orientation.
Yes, I believe that is correct, assuming you only have rotates and translates in your MODELVIEW matrix (no scales or shears, or odd perspective-like distortions, which are possible).
I am currently in a funny situation where I need to extract eye position, target position and the up vector from the modelview matrix for use in gluLookAt().
Keep in mind that gluLookAt() specifies only the VIEWING transform, whereas MODELVIEW (of course) is comprised of both the MODELING and VIEWING transforms. Can you grab the state of the MODELVIEW when you know the MODELING component is the identity?
If so, then you know the eye position in WORLD space (gluLookAt eyeX/Y/Z parms) from the translate, and you know the orthonormal basis vectors for eye space in WORLD from the upper-left 3x3 (or vice versa; orthnormal so inverse = transpose). Plug a few points and subtract and you have centerX/Y/Z.
Intuitively anyway. Might have missed a step or two.
Of course you can do this same thing even if you can’t get MODELVIEW with an identity MODELING transform in it, if your MODELING transform similarly contains only rotates and translates. Your resulting transform would then be the full MODELVIEW for that object space specifically, just not world-space generally.
