Perform stencil test only if depth test fails

OpenGL OpenGL: Basic Coding
1

Hello everyone
As in the topic title I’ve this simple question:
Is it possible to have the following rendering flow?
I need this for prototype my 2.5D test application (3D objects with 2D background). I already tried with depth texture but due to non linear z buffer is a bit difficult to manage.

In pratical what I need is a high resolution background, a very simple geometry for occludes objects in midground (so only depth is written and no color).
I already tried in the past with this way, but for occludes in a good way midgrouds objects I need a very precise geometry and this is not good because geometry must have same detail of final render.
So my idea was to render a very simple geometry for occludes a “superset” of midground object, and then a stencil to follow the 2D object boundary (for examole if I have a table in the room, the geometry can be a simple cube, and stencil follow the table details). In this way I have a precise rendering and simple geometry. My idea was this, is it possible?

  1. Render a quad with background
  2. Write only depth for geometry
  3. Render 3D actors.
    3.1. For each actor pixel, if depth test passes, so render the pixel, otherwise (3.2).
    3.2. make the stencil and test stencil, if passes render the pixel, otherwise NOT render.

Logic should be ok in my opinion, but I don’t how to do it in a smart way with good performances.

Thanks a lot.

Ray

2

If the depth test fails, the fragment colour and depth remain unchanged.

You can choose whether the fragment’s stencil value is updated if the depth test fails. Stencilling has 3 cases: stencil test fails, stencil test passes and depth test fails, stencil test and depth test both pass. But this only affects the stencil value; the colour and depth are only modified if all tests pass.

3

Hi GClements,
Second case should be what I need, so depth test failed and stencil, but sorry it’s still not so clear from my side if it’s possible to obtain my behavior, in a nutshell for what I need the actor pixels must be NOT written only if either depth and stencil failed, in all other case must be written.

4

Here’s what you can do:

  1. If the depth and stencil tests pass, you can write color/depth/stencil.

  2. If the depth test fails and the stencil test passes, you can update the stencil with a different operation from the above.

  3. If the stencil test fails (regardless of the depth test), you can update the stencil with a different operation from the above.

These are the only operations you can do in OpenGL. You can only update color/depth if both tests pass.

5

Thanks a lot for reply guys, so if I understood in a nutshell what I need is not possible to obtain if these 3 operations are the only allowed?

6

Correct.

From what I know of your application, you don’t need stencilling, you just need a sufficiently-accurate depth texture with which to initialise the depth buffer each frame.

7

Dear GClements,
thanks a lot for your support, yes, you understood well, infact filling the depth buffer with a depth texture was my first (and also favourite) idea, I tried this also trick because I had problems with first idea, but now that I have an example I can show you a bit better my problems on the original topic (just to don’t go OT)