Hi,
I am trying to display a MJPEG video an an OpenGL texture in Android.
Until now I used libJPEG to decompress the MJPEG stream and then just copied the result onto a surface view after a few modifications (remap all colors seperately), but the modifications are too slow on the CPU and therefore I would like to do this with OpenGL.
Unfortunately I have not found any library that decompresses a MJPEG-data-buffer from memory to an OpenGL surface. The only way I see right now is to decompress the buffer with libJPEG and then load it into OpenGL, but this would write into memory and then copy to OpenGL, and I would like to avoid the copying step. Is there any library that accomplishes that?
You can’t map a texture’s pixels directly.
The closest you can get is to create a buffer, map it, decompress to the mapped region, then bind the buffer to GL_PIXEL_UNPACK_BUFFER before calling glTexImage2D(). But it’s unclear whether this will be any faster than simply creating the texture from CPU memory.
If copying is an issue, you need to move some or all the JPEG decompression onto the GPU, so you’re copying compressed data. If you can get the decoder to give you YUV420 data (i.e. YCrCb data with the chroma channels at half resolution), that will halve the size of the data being copied. You can either expand that to raw RGB on the GPU (with a compute shader or render-to-texture) or decode it on the fly during rendering.
Performing the inverse-DCT conversion on the GPU is reasonably straightforward, but doesn’t reduce the amount of data being copied. Performing the Huffman decompression on the GPU is fairly involved (and not something a GPU is particularly suited to). But if you can perform just the Huffman decompression on the CPU and transfer the decompressed data without the trailing zeros, that will reduce the amount of data copied.