This calls into question whether you are activating MSAA rendering properly (and/or whether your driver implements it properly). I would suggest that you read this wiki page: Multisampling
With normal single-sample alpha-blend transparency, what you are describing makes sense, if the closer translucent object renders first (with depth writes and depth test enabled). When it does, it blends with the background, and sets the pixel depth value. Then the more-distant object renders, fails the depth test, and never writes (or blends) its color with the current pixel color.
However, this is not how MSAA rasterization works.
With MSAA rasterization, depth (and stencil) values are stored per sample, not per pixel, so depth compares happen per sample. Further, the coverage mask (determined from GL_ALPHA_TO_COVERAGE
in this case) determines which subsamples within a pixel to write or to not write to in order to approximate that degree of transparency.
What this means is that, with MSAA rasterization with GL_ALPHA_TO_COVERAGE
, even if the closer translucent object renders first, its depth value was only written to the pixel subsamples in the coverage mask. Therefore, when the most-distant object renders, there are still samples within that pixel whose depth values still reflect the background depth value (and not the depth value of the closer translucent object), so that more-distant object will win the depth test against those.
Example: If your closer, translucent object was only 25% opaque, then only ~25% of the subsamples should have been written, so the further object should win the depth test against and write/blend color for the remaining 75% of the subsamples (assuming our simple example with just these two fragments involved).
Based on this, let’s see the code that you are using for allocating an MSAA framebuffer, binding it, enabling MSAA rasterization, and downsampling (resolving) it to produce a renderable single-sample result.
Verify that you are actually enabling MSAA rasterization (glEnable( GL_MULTISAMPLE )
), not just creating and binding an MSAA render target.