Results 1 to 4 of 4

Thread: Stencil test problem

  1. #1
    Member Newbie
    Join Date
    May 2004
    Posts
    46

    Stencil test problem

    Hi

    1.I'm using stencil test to draw ceilings for non-convex polygons
    Sometimes I'm getting strange display ,like the attached image ,i.e. I can see transparent lines ,and also white stripes around the frame

    http://img238.imageshack.us/my.php?i...nter023eu1.jpg

    My code:

    glEnable(GL_STENCIL_TEST);
    glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
    glStencilFunc(GL_ALWAYS, 0x1, 0x1);
    glStencilOp(GL_KEEP, GL_INVERT, GL_INVERT);
    for (int ib = 0; ib < iNumBuildings;ib++)
    {
    glBegin(GL_TRIANGLE_FAN);
    for(int iw = 0;iw < buildings[ib].iNumWalls;iw++)
    glVertex4f(.vertex coordinates..);
    glEnd();
    }

    glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
    glStencilFunc(GL_EQUAL, 0x1, 0x1);
    glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);

    for (int ib = 0; ib < iNumBuildings;ib++)
    {
    glBegin(GL_TRIANGLE_FAN);
    for(int iw = 0;iw < buildings[ib].iNumWalls;iw++)
    glVertex4f(...vertex coordinates...)
    glEnd();
    }
    glDisable(GL_STENCIL_TEST);


    Any suggestions?

    Thanks

  2. #2
    Senior Member Regular Contributor trinitrotoluene's Avatar
    Join Date
    Sep 2008
    Location
    Montérégie,Québec
    Posts
    362

    Re: Stencil test problem

    After some simple testing, I found these stencil state setup.

    Code :
     //Draw the geometry only in the stencil buffer
     //to avoid writing some unwanted value in the depth buffer. 
     glEnable(GL_STENCIL_TEST);
     glDisable(GL_DEPTH_TEST);
     glColorMask(GL_FALSE,GL_FALSE,GL_FALSE,GL_FALSE); 
     glStencilFunc(GL_ALWAYS, 0x1, 0x1);
     glStencilOp(GL_KEEP, GL_KEEP, GL_INVERT);
     
     //Draw concave geometry 
     
     
    //Draw the geometry only where the mask is 1 with depth test //enable 
     glEnable(GL_DEPTH_TEST);
     glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
     glStencilFunc(GL_EQUAL, 0x1, 0x1);
     //Reset the stencil mask to 0 where it was 1. Useful if we want //to draw many concave geometry
     glStencilOp(GL_KEEP,GL_INVERT,GL_INVERT); 
     //Draw concave geometry again
     glDisable(GL_STENCIL_TEST);

    Hope this code help and is bug free.

  3. #3
    Member Newbie
    Join Date
    May 2004
    Posts
    46

    Re: Stencil test problem

    Both versions (your and mine) do the same,but thanks

    Problem solved ,it was because of
    glEnable(GL_POLYGON_SMOOTH);

    I think also enabling/disabling depth buffer doesn't matter here....( or does ?)

  4. #4
    Senior Member Regular Contributor trinitrotoluene's Avatar
    Join Date
    Sep 2008
    Location
    Montérégie,Québec
    Posts
    362

    Re: Stencil test problem

    I am glad to know that the rendering of your concave polygon now work.

    With the code you post, I have to modify the the depth func (GL_LEQUAL) to be able to see the concave poly. But if a draw some geometry behind, the triangle(s) part who are not inside the concave poly hid and I see the background color. It is for this
    problem I have to disable the depth buffer when writing the stencil buffer the first time.

Similar Threads

  1. Using the Stencil Test
    By Twanks123 in forum Vulkan
    Replies: 7
    Last Post: 10-28-2016, 09:19 AM
  2. Perform stencil test only if depth test fails
    By Ray1184 in forum OpenGL: Basic Coding
    Replies: 6
    Last Post: 09-23-2015, 01:21 AM
  3. FBO stencil test
    By Emiel in forum OpenGL: Advanced Coding
    Replies: 2
    Last Post: 07-08-2010, 04:10 AM
  4. How to clear stencil buffer after stencil test?
    By in forum OpenGL: Basic Coding
    Replies: 5
    Last Post: 06-14-2003, 09:04 AM
  5. need some help with stencil test.
    By MemLeak in forum OpenGL: Advanced Coding
    Replies: 2
    Last Post: 09-09-2002, 05:27 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Proudly hosted by Digital Ocean