Implementing a "Pan Hand."

[Alterscapes]

I apologize in advance if this question is a bit muddled. I'm relatively new to advanced GL coding.

I'm using JOGL + the JOGL port of GLEEM to write an app. I want to implement a feature similar to the "Pan Hand" tool in Adobe Acrobat/Reader. I've defined a plane (right now it's the X,Y axes, but I'd like to keep my solution general). When the user clicks, I'm casting a ray onto this plane and determining the (x,y,z) ModelView coordinate that corresponds to the (x,y) screen position. When the user moves the mouse, I want to update the x and y entries of the base ModelView matrix s.t. the ray through the new (x,y) screen position still maps to the original (x,y,z) point on the plane in scene space.

My naive intuition about how to achieve this is to retrieve the ModelView, Projection, and Viewport matrices, then solve for the transform-x and transform-y entries in the ModelView matrix, given the new screen(x,y) and fixed scene(x,y,z) coordinates. Is there a way to exploit the capabilities of the fixed-function pipeline to do this without a bunch of math on the CPU?

[dletozeun]

My naive intuition about how to achieve this is to retrieve the ModelView, Projection, and Viewport matrices, then solve for the transform-x and transform-y entries in the ModelView matrix, given the new screen(x,y) and fixed scene(x,y,z) coordinates. Is there a way to exploit the capabilities of the fixed-function pipeline to do this without a bunch of math on the CPU?

Yes that's the idea. With the new (x,y) hand coordinates you can transform them back to object space then modelview space computing the inverse of the transformation: (Projection * Modelview). The function gluUnProject may help you.

[Alterscapes]

I can see how to do this, except I don't have the w component of the coordinates, needed to produce the "Perspective Division" portion of the projection (ref. Fig 3-2 in the redbook, at http://glprogramming.com/red/chapter03.html) ..

Ideally it seems like I want to solve so that Modelview(4x4) = worldCoords(4x1)^-1 * screenCoords(2x1) * Viewport(2x2)^-1 * PerspectiveDivision(4x1)^-1 * Projection (4x4)^-1

but I want to fix all elements of Modelview except [1,4](x) and [2,4](y) ...

Anyone have a code snippet, by any chance?

[edit/addition]

I'm looking at the source to gluproject on the advice of a friend. It looks like if I assume w is always 1 (and if it's good enough for GLUT, it's good enough for me), and then put everything in terms of modelview[12] (eye x transform) and modelview[13] (eye y transform), then solve the system of equations for those two terms, I should be there. I'd ideally like to come up with a more generalized system of equations (eg, for cases where the camera is constrained to a different plane) though..

[dletozeun]

I think you should be looking at the gluUnProject code but anyway, w is affected by the perspective transformation. After this transformation, perspective division occurs. So in object, world, eye, clip (after perspective division) or screen space you can consider that w is 1 for a position.

Now, write down all this on paper sheet and you should find easily the modelview matrix to apply to your object to follow the hand.