[Solved] Picking oddities

Hi,

Hope someone can help on my picking/selection problem.

I have some 2D geometries on the screen. I manage to do picking sucessfully with

viewport(0, 0, windowWidth, windowHeight) and
glOrtho(0, windowWidth, 0, windowHeight, anyNearZ, -anyNearZ) and default orientation looking down into negative z-axis.

Oddity #1:
Even though the documentation says glPickMatrix takes window coordinates, I actually need to convert the mouse coord to world coord to get it to work. Why?

Oddity #2:
If I change my glOrtho to center on (0, 0, 0) with glOrtho(-windowWidth/2, windowWidth/2, -windowHeight/2, windowHeight/2, anyNearZ, -anyNearZ) it doesnt work anymore. the GL_SELECT buffer gets drawn some offset distance away from the actual geometry i am trying to pick. e.g. say my 5x5 SQAURE is at (10, 10), i need to click somewhere at (30, 10) with only a 2x2 hit area to detect the hit. What can be wrong?


void select(int x, int y)
{
	// if there are no shapes, nothing to select
	if(shape.size() <= 0) return;

	int buffSize = shape.size() * 4;
	GLuint* buff = new GLuint[buffSize];
 	GLint hits = 0;
	GLint view[4];
 
 	glSelectBuffer(buffSize, buff);
	glViewport(0, 0, worldWidth, worldHeight);
 	glGetIntegerv(GL_VIEWPORT, view);
 	glRenderMode(GL_SELECT);
 	glInitNames();
	// Now fill the stack with one element
	// otherwise glLoadName will generate an error
	// TODO: Is this because glLoadName will PopName before PushName
 	glPushName(0);
 	glMatrixMode(GL_PROJECTION);
 	glPushMatrix();
 		glLoadIdentity();
 		gluPickMatrix(x, y, 1.0, 1.0, view);
		glOrtho(-worldWidth/2, worldWidth/2,
			-worldHeight/2, worldHeight/2,
			 worldWidth, -worldWidth);
		glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
		glMatrixMode(GL_MODELVIEW);
		glLoadIdentity();
		for(int i = 0; i < shape.size(); i++)
		{
			glLoadName(i);
			shape[i]->Draw();
		}

	glFlush();
	glutSwapBuffers();

 	glMatrixMode(GL_PROJECTION);
 	glPopMatrix();
	// total number of hits
 	hits = glRenderMode(GL_RENDER);

	delete [] buff;
 
 	glMatrixMode(GL_MODELVIEW);
}

OK… it happens all the time… right after I post in the forums, I manage to stumble upon the answers.

By trial and error, I manage to solve my problems. glPickMatrix takes window coordinates except y is flipped as in

y = windowHeight - 1 - y;

The everything else works alright. Thanks.