i already posted this ineffectively to comp.graphics.api.opengl and on this board (but in the wrong section, they kindly directed me here).
I’ve got a strange problem that solely occurs
in my OpenGL app using glut (v3.7):
Whenever i scroll the mouse wheel forwards,
the program exits giving a segmentation fault.
When i scroll the wheel backwards, nothing happens. I have two subwindows and the mouse functions for each of them work fine, also the three buttons work as expected.
Libraries i use: X11, Xi ,Xm, glut, GL and GLU.
I’m on redhat9 using a MS Intellimouse (optical,usb) and scrolling with the mouse wheel works fine otherwise (see the mouse section of my XF86Cconfig below).
Since i can only use an usb mouse on my laptop, i have not yet tried other mice.
Has anybody got an idea what’s going on
or what i could do to solve this?
I’ve already looked at alternatives like glfw but hesitate before porting.
i didn’t debug yet, thanks for your suggestion. I’ll have to look how to do it.
I’m not quite sure what you mean with ‘off by one’: a unique error only occuring in my app?
“Off by one” is a “more common than it should” programming error where somebody created a memory buffer, say, from 0 to 9 - but the actual code slips and thinks it is from 1 to 10, bites at the end of the buffer and dies with a segfault.
I think that I have found the problem. The scroll wheel sends mouse buttons 4 and 5, but an array of menu ids in the window structure only allowed for a 3 button mouse. Scrolling forward in a subwindow will effectively try to use the window’s parent pointer as a menu id. Scroll back would try to use a child window pointer if any.
I sent a bug report and we’ll see what official fix comes of it. For a quick hack (if you don’t mind recompiling glut) you can either:
try changing the ‘#define GLUT_MAX_MENUS 3’ in glutint.h to 5
validate the index of menu[] in glut_event.c line 425 (menuNum = window->menu[event.xbutton.button - 1]
