Given a light source position, vertex position, and vertex normal, how do I calculate the rgb value to store in the color array for that vertex, for use with dot3 bumpmapping? I can do the calculations, but they don’t really take the texture mapping into account. If the texture is flipped on one face, it will get incorrect values that result in odd normal mapping.
Well there’s no standard way of doing this, the point is to support the same space in your vertex colors in tangent or object space that you do in your normal map, and derivatives of texture.
Things like mirroring can require an axis flip (because derivatives have been flipped).
Just be consistent and set up a simple test case to make sure you can get it right or at least can tweak your code until you do get it right.
I think the only possible way to do this must be to find another vertex this vertex connects to, and construct vectors from their positions and tex coords, then somehow construct a matrix from that. This is not a simple task. I was hoping that maybe there was a simpler method.
Are you basically asking how to compute the tangent space? If so, I have some code for that in my framework:
http://www.humus.ca/index.php?page=3D
On brushes, this is easy, because they have texture mapping axes and a face normal. On arbitrary triangle meshes, it is a little more difficult. How do you usually set this up on arbitrary triangle meshes?
Well, at least as far as I know, you can’t do any tangent space without having both normals and texture coordinates.
Right, of course you have normals and texcoords for each vertex in the triangle mesh. But it is quite a bit more complex than just setting it up for a flat coplanar surface with known texture mapping axes.
http://www.terathon.com/code/tangent.html
works fine, however be aware that using mirrored UVs will create problems, needs some special preparation.
the meshmender lib from nvidia can take care of that (I am sure there is other libs too)
Cool, that looks pretty simple. Thanks.