Skeletal Animation giving strange results (Collada imported file) when playing

Ok, guys, this is gonna be a loooong post. I know that. But I just cannot figure out where I am doing mistakes. I need your help!
I’ve been following this tutorial and I tried to convert it to my OpenTK C# engine:

Animation in Blender: Dropbox - File Deleted

Blender file: Dropbox - File Deleted

Animation in my engine: Dropbox - File Deleted
(don’t worry about the colors - it’s just for testing the normals)

I downloaded a rigged blender model and exported it to the collada file format. I then import this file with my engine (using the assimp library).
If I import a model that has no bones and no animations, the model gets rendered correctly. That’s why I assume, that there is a mistake somewhere in my animation code.

I need to find the answer to the following questions:

[ul]
[li]is my blender file ok? how do I need to export to collada? am I ticking the right boxes? do I always need to export the file with my mesh lying on the side (swapping y and z in blender)?
[/li][li]is the creation of my bone id and bone weight buffers ok?
[/li][li]is the calculation of my matrices ok?
[/li][/ul]

So here goes my code:

When importing a rigged model, I first create two additional buffers of Vector3 objects. One buffer holds the bone ids that affect each vertex and the other holds the weights. The GLBoneMapping instance gets stored in a dictionary. Every imported model filename gets an associated GLBoneMapping (if there are any bones in the model). The parameter “Scene result” is the return value of the assimp importer method.


    public GLBoneMapping(Scene result)
    {
        if(result.MeshCount < 1 || result.MeshCount > 1)
        {
            throw new Exception("Model contains more than one mesh. That is not supported.");
        }
        mBoneCount = result.Meshes[0].BoneCount;
        BoneTransformations = new Matrix4[mBoneCount];
        BoneOffsetTransformations = new Matrix4[mBoneCount];
        for(int i= 0; i < BoneTransformations.Length; i++)
        {
            BoneTransformations[i] = Matrix4.Identity;
            GLMatrixHelper.ConvertAssimpToOpenTKMatrix(result.Meshes[0].Bones[i].OffsetMatrix, out Matrix4 convertedOffsetMatrix);
            convertedOffsetMatrix.Invert();
            BoneOffsetTransformations[i] = convertedOffsetMatrix;
        }

        mBoneIDsBuffer = new Vector3[result.Meshes[0].VertexCount];
        mBoneWeightsBuffer = new Vector3[mBoneIDsBuffer.Length];
        for(int i = 0; i < mBoneIDsBuffer.Length; i++)
        {
            mBoneIDsBuffer[i] = new Vector3(-1, -1, -1);
            mBoneWeightsBuffer[i] = new Vector3(0, 0, 0);
        }

        for(int vertexindex = 0; vertexindex < mBoneIDsBuffer.Length; vertexindex++)
        {
            for(int boneindex = 0; boneindex < mBoneCount; boneindex++)
            {
                Bone currentBone = result.Meshes[0].Bones[boneindex];
                foreach(VertexWeight vw in currentBone.VertexWeights)
                {
                    if(vw.VertexID == vertexindex)
                    {
                        for(int slot = 0; slot < 3; slot++)
                        {
                            if (mBoneIDsBuffer[vertexindex][slot] < 0)
                            {
                                mBoneIDsBuffer[vertexindex][slot] = boneindex;
                                mBoneWeightsBuffer[vertexindex][slot] = vw.Weight;
                                break;
                            }
                        }
                    }
                }
            }
            for (int j = 0; j < 3; j++)
            {
                if (mBoneIDsBuffer[vertexindex][j] < 0)
                {
                    mBoneIDsBuffer[vertexindex][j] = 0;
                }
            }
        }

        int c = 0;
        foreach(Bone b in result.Meshes[0].Bones)
        {
            if(b.Name.Trim().Length == 0)
            {
                throw new Exception("Your model contains nameless bones. Please remodel.");
            }
            else
            {
                if(!BoneNames.ContainsKey(b.Name))
                {
                    BoneNames.Add(b.Name, c);
                }
                else
                {
                    throw new Exception("Your model contains 2 bones with identical names. Please remodel.");
                }
            }
            c++;
        }
    }

Further down the road, I then upload these buffers as attributes (in my Draw() method):


    GL.BindBuffer(BufferTarget.ArrayBuffer, mRenderer.GetBufferHandleBoneIDs());
    GL.BufferData(BufferTarget.ArrayBuffer, (IntPtr)(GetBoneIDsBuffer().Length * Vector3.SizeInBytes), GetBoneIDsBuffer(), BufferUsageHint.StaticDraw);
    GL.VertexAttribPointer(mRenderer.GetAttributeHandleBoneIds(), 3, VertexAttribPointerType.Float, false, 0, 0);
    GL.EnableVertexAttribArray(mRenderer.GetAttributeHandleBoneIds());

    GL.BindBuffer(BufferTarget.ArrayBuffer, mRenderer.GetBufferHandleBoneWeights());
    GL.BufferData(BufferTarget.ArrayBuffer, (IntPtr)(GetBoneWeightsBuffer().Length * Vector3.SizeInBytes), GetBoneWeightsBuffer(), BufferUsageHint.StaticDraw);
    GL.VertexAttribPointer(mRenderer.GetAttributeHandleBoneWeights(), 3, VertexAttribPointerType.Float, false, 0, 0);
    GL.EnableVertexAttribArray(mRenderer.GetAttributeHandleBoneWeights());


    for (int i = 0; i < mBoneTransformations.Length; i++)
    {
        GL.UniformMatrix4(mRenderer.GetUniformHandleBoneTransformations() + i, false, ref mBoneTransformations[i]);
    }

Before I draw the animation, I update the matrices (mBoneTransformations) that get sent to the vertex shader:


    internal void UpdateAnimationTransformationMatrices(ref Matrix4[] matrices, bool frameBased = true)
    {
        if(AnimationID >= 0)
        {
            GLBoneMapping currentMapping = ModelBoneMapping[GetType()];
            Scene scene = ModelDatas[GetType()];
            if (currentMapping == null || scene == null)
            {
                throw new Exception("No model data or bone mapping found. Cannot update animation matrices.");
            }

            Animation a = ModelAnimations[GetType()][AnimationID];
            Matrix4 identity = Matrix4.Identity;

            float timestamp = (float)(a.DurationInTicks * AnimationPercentage);
            ReadNodeHierarchy(timestamp, ref a, AnimationID, scene.RootNode, ref identity);

            Type t = GetType();
            for(int i = 0; i < scene.Meshes[0].BoneCount; i++)
            {
                matrices[i] = ModelBoneMapping[t].BoneTransformations[i];
            }
        }
        else
        {
            for(int i = 0; i < matrices.Length; i++)
            {
                matrices[i] = Matrix4.Identity;
            }
        }
    }

The actual calculation of the matrices is done in the ReadNodeHierarchy method:


    internal void ReadNodeHierarchy(float timestamp, ref Animation animation, int animationId, Node node, ref Matrix4 parentTransform)
    {
        string nodeName = node.Name;
        GLMatrixHelper.ConvertAssimpToOpenTKMatrix(node.Transform, out Matrix4 nodeTransformation);

        NodeAnimationChannel channel = null;
        Matrix4 globalTransform = Matrix4.Identity;

        if (AnimationMapping[GetType()][animationId].NodeDictionary.ContainsKey(nodeName))
        {
            channel = AnimationMapping[GetType()][animationId].NodeDictionary[nodeName];
        }
        if (channel != null)
        {

            CalcInterpolatedScaling(out Vector3 scaling, timestamp, ref channel);
            Matrix4 scalingMatrix = Matrix4.CreateScale(scaling);

            CalcInterpolatedRotation(out OpenTK.Quaternion rotation, timestamp, ref channel);
            rotation.Normalize();
            Matrix4 rotationMatrix = Matrix4.CreateFromQuaternion(rotation);

            CalcInterpolatedTranslation(out Vector3 translation, timestamp, ref channel);    
            Matrix4 translationMatrix = Matrix4.CreateTranslation(translation);

            // finally merge all matrices together:
            nodeTransformation = scalingMatrix * rotationMatrix * translationMatrix;
        }
                                               
        globalTransform = nodeTransformation * parentTransform;

        if (ModelBoneMapping[GetType()].BoneNames.ContainsKey(nodeName))
        {
            int boneIndex = ModelBoneMapping[GetType()].BoneNames[nodeName];
            ModelBoneMapping[GetType()].BoneTransformations[boneIndex] = ModelBoneMapping[GetType()].BoneOffsetTransformations[boneIndex] * globalTransform * mGlobalInverseTransform;
        }
       
        for (int i = 0; i < node.ChildCount; i++)
        {
            ReadNodeHierarchy(timestamp, ref animation, animationId, node.Children[i], ref globalTransform);
        }
    }

    internal void CalcInterpolatedTranslation(out Vector3 translation, float timestamp, ref NodeAnimationChannel channel)
    {
        if (channel.PositionKeyCount == 1)
        {
            Vector3D s = channel.PositionKeys[0].Value;
            translation = new Vector3(s.X, s.Y, s.Z);
            return;
        }
        for (int i = 0; i < channel.PositionKeyCount - 1; i++)
        {
            VectorKey key = channel.PositionKeys[i];
            if (timestamp < (float)channel.PositionKeys[0].Time)
            {
                Vector3 start = new Vector3(key.Value.X, key.Value.Y, key.Value.Z);
                translation = start;
                return;
            }
            else
            {
                if (timestamp >= (float)key.Time && timestamp <= (float)channel.ScalingKeys[i + 1].Time)
                {
                    VectorKey key2 = channel.PositionKeys[i + 1];

                    float deltaTime = (float)(key2.Time - key.Time);
                    float factor = (timestamp - (float)key.Time) / deltaTime;
                    if (factor < 0 || factor > 1)
                    {
                        throw new Exception("Error mapping animation timestamps. Delta time not valid.");
                    }

                    Vector3 start = new Vector3(key.Value.X, key.Value.Y, key.Value.Z);
                    Vector3 end = new Vector3(key2.Value.X, key2.Value.Y, key2.Value.Z);
                    Vector3.Lerp(ref start, ref end, factor, out translation);
                    return;
                }
            }
        }
        Console.WriteLine("Error finding scaling timestamp for animation cycle.");
        translation = new Vector3(0, 0, 0);
    }

The calculation methods for rotation and scaling are quite the same (rotation with Quaternion instead of Vector3).

Finally the vertex shader code:


    void main()
    {
        vec4 totalLocalPos = vec4(0);
        vec4 totalNormal = vec4(0);
        vec4 totalTangent = vec4(0);
        vec4 totalBiTangent = vec4(0);
        
        for(int i = 0; i < 3; i++)
        {
            totalLocalPos += aBoneWeights[i] * uBoneTransforms[int(aBoneIndices[i])] * vec4(aPosition, 1);
            totalNormal  += aBoneWeights[i] * uBoneTransforms[int(aBoneIndices[i])] * vec4(aNormal, 0);
            totalTangent += aBoneWeights[i] * uBoneTransforms[int(aBoneIndices[i])] * vec4(aNormalTangent, 0);
            totalBiTangent  += aBoneWeights[i] * uBoneTransforms[int(aBoneIndices[i])] * vec4(aNormalBiTangent, 0);
        }
        
        vShadowCoord = vec4(uShadowMVP * vec4(totalLocalPos.xyz, 1.0));
        vPosition = uM * vec4(totalLocalPos.xyz, 1.0); 
        vColor = vec4(aColor, 1.0); 
        vTexture = aTexture; 
        vNormal = normalize(vec3(uNormalMatrix * vec4(totalNormal.xyz, 0.0)));
                 
        vec3 tangent = normalize(vec3(uM * vec4(totalTangent.xyz, 0.0)));
        vec3 biTangent = normalize(vec3(uM * vec4(totalBiTangent.xyz, 0.0)));
        vec3 normal = normalize(vec3(uM * vec4(totalNormal.xyz, 0.0)));
        TBN = mat3(tangent.xyz, biTangent.xyz, normal.xyz);
                                                         
        gl_Position = uMVP * vec4(totalLocalPos.xyz, 1.0);
    }

BTW: I am pretty sure the multiplication order of my matrices is ok, because I always need to do: model * view * projection instead of projection * view * model.

Your help is greatly appreciated!

I also found this question:

I think this guy might have had the same prolem. But I do not understand the solution. :frowning: