Issues with using ReadPixels

Hello,

I’m trying to do picking with openGL. In a first time, before trying ray tracing, I’m trying to use the alpha channel (I only have a few items in my scene) to select the item corresponding to the cursor.

I saw that ReadPixels allows to get back the colors from a pixel, so i tried it. Here is a part of my code (context initialization, mouseDown event function).


	  var lastMouseX = null;
	  var lastMouseY = null;
	  var buff = new Uint8Array(4);
	  var currentSelection=0.0;

	  function handleMouseDown(event) {
	    mouseDown = true;
	    lastMouseX = event.clientX;
	    lastMouseY = event.clientY;
	    gl.readPixels(lastMouseX,lastMouseY,1,1,RGBA,gl.UNSIGNED_BYTE, buff);
	    currentSelection=buff[3];
	  }

var gl;
    function initGL(canvas) {
        try {
            gl = canvas.getContext("experimental-webgl",{preserveDrawingBuffer: true}) || canvas.getContext("webgl",{preserveDrawingBuffer: true});
            canvas.mozOpaque=true;
            gl.viewportWidth = canvas.width;
            gl.viewportHeight = canvas.height;
        } catch (e) {
        }
        if (!gl) {
            alert("Could not initialise WebGL, sorry :-(");
        }
    }

So i define an item with a buffer color fulled of (x.x,y.y,z.z,0.6), and put a rotation conditioned by a “if (currentSelection==0.6 || currentSelection==153)”. But when I test it it doesn’t work. After a few tries, it seems than my readPixels() makes the handleMouseDown(event) bug, indeed every line I add after readPixel() doesn’t work. And I don’t understand where the issue is as my buffer has good format and as I make sure of keeping the drawing buffer.

Anyone has an idea ?

PS : the mozOpaque for the context is for Firefox, to the canvas remains opaque.

Well well… it seems i should get back to the beginner part of the Forum… I forgot a “gl.” before “RGBA” and that explains it blocks the following, and now it works even if it returns 255 everytime for alpha and 0 for every color (i.e. my background). I’ll try to fix it.

Sorry to bother you for that…

I recently implemented mouse picking in GlowScript (glowscript.org). For object with id number N, I create a false color like this:

function id_to_falsecolor(N) { // convert integer object id to floating RGBA for pick operations
    var R=0, G=0, B=0
    if (N >= 16777216) {
        R = Math.floor(N/16777216)
        N -= R*16777216
    }
    if (N >= 65536) {
        G = Math.floor(N/65536)
        N -= G*65536
    }
    if (N >= 256) {
        B = Math.floor(N/256)
        N -= B*256
    }
    return [R/255, G/255, B/255, N/255]
}

I want to render the false colors of the objects to a texture the size of the canvas (need not be powers of 2):

pick_texture = gl.createTexture()
gl.bindTexture(gl.TEXTURE_2D, pick_texture )

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE)

gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, width, height, 0, gl.RGBA, gl.UNSIGNED_BYTE, null)
gl.bindTexture(gl.TEXTURE_2D, null)

Set up a framebuffer to render to:

var pickFramebuffer = gl.createFramebuffer()
gl.bindFramebuffer(gl.FRAMEBUFFER, pickFramebuffer)

var pickRenderbuffer = gl.createRenderbuffer() // create depth buffer
gl.bindRenderbuffer(gl.RENDERBUFFER, pickRenderbuffer)
gl.renderbufferStorage(gl.RENDERBUFFER, gl.DEPTH_COMPONENT16, width, height)
gl.framebufferRenderbuffer(gl.FRAMEBUFFER, gl.DEPTH_ATTACHMENT, gl.RENDERBUFFER, pickRenderbuffer)

gl.bindRenderbuffer(gl.RENDERBUFFER, null)
gl.bindFramebuffer(gl.FRAMEBUFFER, null)

At the start of a render cycle I execute this:

gl.bindFramebuffer(gl.FRAMEBUFFER, pickFramebuffer)
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, pick_texture, 0)

After rendering all the objects, with no lighting calculations (so that all pixels of an object have the same false color), I get the id number of the object under the mouse like this:

var pixels = new Uint8Array(4)
gl.readPixels(canvas.mouse.__pickx, canvas.mouse.__picky, 1, 1, gl.RGBA, gl.UNSIGNED_BYTE, pixels)
var id = 16777216*pixels[0] + 65536*pixels[1] + 256*pixels[2] + pixels[3]

// A non-power-of-two texture has restrictions; see
// http://www.khronos.org/webgl/wiki/WebGL … ifferences
// Important note: This destroys antialiasing, so render to texture is
// fully useful only for doing some computations, unless we do our own antialiasing.
// We could render to a large offscreen texture, say twice the width and height.

Note that antialiasing being turned off is actually what one wants for mouse picking because you don’t want an object with id of 10 that is next to an object of id 30 to get picked with an id of 20, which could happen with antialiasing turned on, due to the averaging of neighboring colors.

Bruce,

Using a texture to render the picking color is better than setting a constant attribute?
I mean:


gl.disableVertexAttribArray(vertexColorAttributeLocation);
gl.vertexAttrib4f(vertexColorAttributeLocation, obj.pickColor[0], obj.pickColor[1], obj.pickColor[2], obj.pickColor[3]);

Thanks,
Everton

I’m sorry, but I don’t understand the question. Surely one needs to render to a texture in order to be able to read the pixel color from the texture? Can you explain more fully how you would implement picking?

Yeah, sorry, I had misunderstood your post.

By default WebGL clears the backbuffer after your render function exits back to the browser. In other words, if you something like this


function render() {
   gl.clearColor(1,0,0,1); // red
   gl.clear();
   requestAnimationframe(render);
}
render();

canvas.addEventListener('mousedown', function(event) {
   ...
   gl.readPixel(..., mouseX, mouseY,....);
}

You’ll get 0,0,0,0 because WebGL cleared the backbuffer after the function for requestAnimationFrame (or any event that updates WebGL) exits

There’s 2 ways around this off the top of my head

  1. Render to your own FBO, bind it when you read

WebGL won’t clear your FBO, only the backbuffer.

  1. Init WebGL with {preserveDrawingBuffer: true}

This effectively tells WebGL not to clear the backbuffer (or rather not to double buffer the backbuffer)