I gave up on using Collada and fell back on the DirectX x file format. Although parsing an .x file without DirectX is no small task, it is possible, and I cannot say the same of parsing Collada files.
Whenever complaints about the complexity of this file format are made, it is usually pointed out that Collada is meant as an interchange format, not a game-ready format. Fair enough, I can understand that various applications use a lot of very complex data.
Why is there no game-ready equivalent to Collada? Don’t tell me that game engines require such different data that this is impossible, because that is not true. All we need is a model hierarchy with matrices, and vertex buffers for position, normals, uv coords, colors, and binormal/tangent arrays. Animation data per limb would not be difficult to add.
I would gladly adapt my engine to such a format if there were exporters readily available for every major modeling package. Frankly I do not see Collada as it is catching on. I could be wrong, but for developers like myself, the format is prohibitively complex. However, a simpler fast-loading binary format with broad export support would be very appealing to me.
The collada refinery, a tool released by sony, can be used to refine the data within a collada file so that it will be closer to what you need at runtime. In particular, the deindex filter will take separate streams for positions, normals, uvs etc and make them parallel. Refinery reads in a .dae file and outputs a .dae file. Unfortunately, the refinery doesn’t include a utility to generate the tangent space basis if there isn’t one. I haven’t tinkered with the source code to it so i wouldn’t know how hard that would be to add.
In any of the collada examples that I’ve seen on skinning, the rigging information is not part of the mesh. That makes sense from a DCC tool point of view. For example in 3dsmax, the skinning is a separate modifier that is applied to a geometry. However, within a game engine usually the bone weights and indices are part of the vertexbuffer and the associated base pose is typically stored in the same object as the mesh. Personally I’d like to be able to represent data from a game engine directly in collada with minimal translation. One minor issue is that source nodes in collada are arrays of a single type. With vertexbuffers containing ints (bone indices) intermixed with floats, I would want the file format to allow an array of structs where the struct is essecntially the vertex class used by the game engine. Someone please correct me if I have misunderstood something.
Instead of inventing a new file format, I’d prefer to see Collada evolve (as necessary) to support game engine needs. Hopefully with a standard set of common filters and tools, each game developer could easily get content into a form that is pretty close (if not one-to-one) with what is needed at runtime.
I think COLLADA is fine as a data exchange format, but definitely not as a game engine format, I think that should still be engine specific.
Loading and parsing a COLLADA file can take quite a lot of time if the file contains too much information due to text to binary translations and text parsing, if you were to load a whole game world (think GTA), you might as well leave the game loading and come the next day to play if you’re lucky.
Instead what you should do is write a converter or “compiler” of sorts that takes a COLLADA file and turns it into your own binary, proprietary format, that way you instantly support any DCC application that exports COLLADA files.
Leadwerks, from what I can tell your main objections to using Collada for your game are that it’s text-based and that it’s complex.
The first point has been discussed a lot. Game developers prefer to roll their own model formats for maximum efficiency. I don’t think Collada was ever in a position to compete against that; even if Collada were binary, game developers would still use their own formats. Consider the .X file format for example. It’s binary and has quite a few importers/exporters, but I don’t think many game developers use it as the model format for their games. Since Collada isn’t trying to be a runtime delivery format, it makes sense that it’s text-based. Working with text documents is much easier for development.
For the second point, you’re right that portions of Collada are somewhat poorly designed. Geometry in particular is needlessly complex. I don’t see why this would be a deal-breaker though. I implemented a Collada loader for my previous employer without too much difficulty. Just read the spec and post questions about any parts that aren’t clear. When you say things like “Although parsing an .x file without DirectX is no small task, it is possible, and I cannot say the same of parsing Collada files” you just sound silly.
I have the collada book advertised as well as used the official spec and I agree somewhat with the poster. The largest problem with all the texts available is that there’s very little discussion about intentions. Basicly what would be needed to ease the implementation of collada would be a very thorough text which discusses how collada is inteded to be parsed. Almost all my problems steems from the fact that you often have to reverse engineer the thoughts of the designer of the format ( “what’s the reason for this indirection? should I parse the libraries first or the scene graph first? how does this element relate to this other distant indirectly referenced element?” ). I understand that collada tries to be as flexible as possible in order to support a wide range of use cases, that however just increases the need for more implementation discussion.
I guess Collada is really meant as a format to convert to another format and then load. It does a good job of this, and I was under the wrong impression when I thought it was for loading directly into a game engine.
I have implemented a file format of my own (Game Model Format, or .gmf) with support for skinned animated models, as well as plain hierarchal models. The data is stored in such a way that it can be read straight into memory and uploaded right to the GPU. Here is a preliminary spec: http://www.leadwerks.com/post/GMFSpec.pdf
I really tried to focus on the model hierarchies, vertex arrays, and animation data. Because there is no possible way I could design a general-purpose materials data format, I just left it so that the material name can be defined in a custom “properties” block. If someone wants to extend this functionality to support their own engine, it can be done without breaking compatibility with old loaders.
At this point all I have is a .b3d to .gmf converter, but I would be interested in having a Collada to .gmf converter written. I believe this was the original intent of the Collada format, and I will share what I implement with everyone else, to take or leave as they see fit.
The Collada converter will be free and open-source, so if you want to use it, you can extend their functionality to meet your own needs. I will also have some sample code for reading and displaying the files.
That seems like a pretty sensible plan to me, and fits in line with how I imagine other game developers would use Collada. By writing a Collada converter for your format, you could support many modeling packages (like Max, Maya, and Softimage) “for free”, without having to write any plugins. In practice you should test these conversion paths, as Collada exporters for modeling packages tend to very greatly in quality.
At my last company we used Collada as the end delivery format, sort of like Google does with Google Earth. Our software ran on mostly high-end PCs though, and we weren’t making a game, so our requirements were very different.
Hah, yes, I’ve definitely felt that way at times. Especially when writing the SID ref resolver in the DOM.[/quote]
And along those lines, I suggest that if you make a spec, you really think about data types, and document failure or unsupported conditions. For instance, what if subnodes is negative? What if length is negative or has a 0 value? Is the format Little Endian only? And does the “official” author of this type of data know how to really write portable structures? Don’t laugh. If you write an entire class or structure to disk, instead of EACH of the elements individually, this is NOT always portable between compilers or platforms. File Offsets listed in OpenFlight docs for instance were incorrect in later versions due to alignment & padding produced by newer compilers. Our company’s software ran on four different platforms once upon a time, and our own file formats were portable among all platforms and compilers.
I’ve probably written over 50 different data readers in my lifetime, and those little minor details DO make a big difference!
For physics middleware it makes sense to provide a binary game-ready representation, closely related to COLLADA. Many professional game companies load the physics assets in-game directly from the Havok binary .hkx format instead of rewriting their own.
For Bullet, we are going to provide a binary chunk-based format using IFF, Interchange File Format, a standard back from 1985. It has all that is needed, including handling big and little-endian, easy extensibility and it has been used in various derived formats (IFF ILBM, AIFF, RIFF etc). The COLLADA URI and SID can be stored as properties in the IFF format.
IFF is like binary XML, so it matches COLLADA quite well. IFF is more compact, faster to load and trivial to implement a reader/writer (a few hundred lines of code). Obviously, we try to keep the data chunks close to the COLLADA spec, and use the extension .CLIFF, as a hint from COLLADA and IFF.
The Dynamica Maya plugin currently exports physics information in COLLADA .dae files. As a first step we provide a converter between COLLADA physics .dae and .cliff. Later we directly export in .cliff from Dynamica, next to .dae. Apart from physics data, it would be trivial to add binary chunks for other COLLADA nodes such as geometry and animation.