Hi Folks:
I'm at the half way point in [these](https://learnopengl.com/#!Advanced-OpenGL/Stencil-testing) great tutorials. I've been refining a library of my own for rendering Blender models for the last several weeks.
I’ll use this model for a crash test dummy as I work my way through the second half of the tutorials.
Thanks to everybody in this forum who’ve helped me get this far.
I’ve been running into a problem with the “Stencil Test” tutorial, which the link points to.
Depth testings seems to go nuts when I try to do anything with a stencil.
Here are the models I’m rendering in a splash screen:
The program’s name “Net Results” should be drawn in the background, with the tennis court rotating in front of it. The giant golf ball is a temporary part of the tennis court model, to help me see if the lighting effects are working.
This looks right when running the splash screen code.
I’ve added that purple box and placed it on the playing surface for the stencil tutorial.
The tutorial’s exercise is to use the stencil test to draw a yellow outline around the purple box.
The OpenGL related include section looks like this:
#define GLEW_STATIC
#include <GL/glew.h>
#include <GL/wglew.h>
// GLM Mathemtics
#include <glm/glm.hpp>
#include <glm/gtc/matrix_transform.hpp>
#include <glm/gtc/type_ptr.hpp>
I’m using assimp, so there’s also a reference to that header somewhere.
I’ve tried this several times, and watched it fail. Before the stencil code is added I test everything but. The purple box is being drawn, the yellow box that will become the outline is being drawn around it. Everything looks good.
Let’s do stencils.
Here are the stencil related code fragments I found from the sample program, and how I fit them into my code:
In the setup function:
glDepthFunc(GL_LESS);
glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_NOTEQUAL, 1, 0xFF);
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
glDepthFunc(GL_LESS) seems redundant, but it was in the sample.
In the loop:
GL_STENCIL_BUFFER_BIT was added to the glClear() argument.
“Net”, “Results” and the tennis court are drawn.
Before the purple box is drawn:
glStencilFunc(GL_ALWAYS, 1, 0xFF);
glStencilMask(0xFF);
After the box is drawn, preparing to draw the outline:
glStencilFunc(GL_NOTEQUAL, 1, 0xFF);
glStencilMask(0x00);
glDisable(GL_DEPTH_TEST);
The depth test is disabled to draw the border so it will be visible even if it’s behind something.
I then select the outline’s shader, it’s fragment shader just returns yellow. Then scale the box’s model transform matrix into the outline’s model matrix.
Set some uniforms in the outline’s shader and draw it.
After drawing:
glStencilMask(0xFF);
glEnable(GL_DEPTH_TEST);
And the depth testing seems to be toast.
Here’s another shot:
“Net” and “Results” are drawn first, if no depth testing was in effect I believe these would be drawn behind everything else. Next the court is drawn, then the purple box and finally, with GL_DEPTH_TEST disabled, the yellow outline. Yet the outline is the element that’s written over by almost everything.
The first suspect was the enabling and disabling of GL_DEPTH_TEST. I removed all the stencil stuff and kept the depth test stuff, images displayed fine.
Commenting out the stencil statements, then uncommenting them one by one shows the depth testing goes bad with the glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE) statement.
I know this function uses the results of a depth test fail. Do I need a different argument for it?
Suggestions?
Thanks
Larry