My Collada wishes (game dev)

Hi there, my wishes:

  • Another layer for faster/easy/clean access to geometry/skin/animation/etc. Yes, Collada is a complex format and very good implemented, but writing a decent importer is just painful. Also could be nice to have some features like convert axis, reduce number of influences, and so on. Honestly, importing a model right now, require more work that if you do it in DCC native API. Personally, I don’t care how much memory, disk space use the Collada files, or the execution speed, as I am using the format for converting only, into my desire format. I think almost everybody do that. I prefer functionality first.
  • Better documentation. Well, Collada specification is clear, to bad almost nobody take care of it. But I think that’s just temporary, as much peoples will start using it, they will fix the problems and will respect more the specifications.

Thanks for reading.

IF there is a import or exporter wizard, would that help?

We are trying to get the tools maker to maintain their own COLLADA exporter.
We manage to get someone to maintain Maya, Max, and XSI, probably more tools makers will follow.

We are still trying to find the best way to help game developers to adapt COLLADA and to make importer. The main problems are the learning curve of COLLADA Schema and the time it takes to make a COLLADA to you binary exporter.

We definately will consider your suggestions on making more higher level middewares and better documentation, and make the importer writing experience less painful. Any other comments are welcome.

Please note that the higher an api level reaches a platform or engine, the less audience and reflexbility it get.

BTW, we encourage asset refining though COLLADA conditioners. Convert axis and reduce number of influences are good examples for conditioner modules. Please search for COLLADA Refinery and conditioners. Existing conditioners include AxisConverter, Triangulation, CacheOptimizer, etc.

Herbert

I am in the process of implementing an import for Collada. And it is indeed painful.

First, there is the issue of deciding whether to use FCollada, Collada_dom/RT, or heaven forbid write something from scratch.

Second, there is getting the data properly traversed and converted from the complicated Collada format into the desired, by default, more simple format.

I chose Collada_dom/RT, but I’ve had to make a dozen fixes to the build environment and Makefiles to get a successful compilation for MinGW on WinXP. And, Collada_dom/RT admits that it is not 100% percent supporting the format. There are also 6 or 7 external third party libraries that are required.

It seems to me that for a format to become widely adopted there should be an officially sanctioned library (or 3) that parses the format and loads it into memory for converting. For example, if I want to use PNG or JPEG, I don’t have to cobble together a patchwork of semi-supported code, I just use the official library for the format.

But, I will admit that FilmBox (FBX) has such support, and it has not taken over the universe. But, 3ds had/has such a library and I think it greatly aided in its adoption as a standard format. Lastly, I am very interested in getting Collada fully supported for import into my middleware, because the Exports seem to be the best possible.

Not trying to be negative, just wishful…
Chris

We are really aware of the difficulties that a new COLLADA adoptor has by choosing and implementing the “right” libraries. It gets even worse as there are even more (not yet official) libraries we are working on within the netxgen 3ds Max and Maya Importer/Exporters :shock:

COLLADA StreamWriter
COLLADA Sax Parser
COLLADA Framwework

At this moment we are joining all our efforts to prepare, test and deliver a complete set of developer tools and implementation samples that are reachable from one single source.

Wizard? You talk about the DCC plugins or?

I am sure the specifications will be respected in the future. BUT, I am afraid some part of the problem is in the API, maybe peoples find it difficult to use or maybe isn’t enough documentation?

Well, in my opinion, Collada format is excelent how is structured. And I think is dont hard to be understand. Again, I think the API is the main problem here, I mean, is way to complex for simple things. Indeed, is doing a great job, but I still think to a high level layer more that welcome.

When I think at a layer, I really like to see it as a high level layer, and with possibility to access the normal API data also. So, I don’t see any lose here, if you need something more detailed, you use the normal classes for you needs via a pointer maybe.

I can detail here. And i am pretty sure are other game devs on forum who can give some opinions about the high level layer and what they need.

Well, I personally still think axis converter or reducing the number of influences could be nice to have it inside collada-dom.

but writing a decent importer is just painful.

Agreed, I used FCollada but it wasn’t 100% compatible, and lost information while roundtrippping, at that time (a few years ago).

Then I switched to COLLADA DOM, but it was a painful process. I created new build systems, including Visual Studio projectfiles, jam and CMake support for all platforms, and fixed/removed clutter and external dependencies. I still ended up using libxml, but I rather used tinyxml. I never upgraded beyond COLLADA 1.4 because of all this pain and wasted time. Hopefully this work could help people with build problems.

The DOM API is not intuitive at all, so for the Bullet physics library we provided a simple interface to load/save collision/physics information.

Unfortunately, the DOM is way too big and memory/processor intensive.

A COLLADA board (if it still exists) could learn the following from the FBX approach:

[ul]
[li] FBX has 1 official SDK, supported by Autodesk[/:m:337xc6dm][/li][li] This FBX SDK is easy to use[/:m:337xc6dm][/li][li] FBX binary files are almost always compatible. No need for a conformance test when you have one official SDK[/*:m:337xc6dm][/ul][/li]
If only FBX was open source… :slight_smile:

Erwin, did you check in your changes to sourceforge?

All the changes are available in the current Bullet physics sdk source repository at http://bullet.googlecode.com ,under Bullet/Extras/COLLADA_DOM and Bullet/Extras/libxml. The build system is based on CMake, http://cmake.org. Other changes are not suitable to propagate upstream, because it disables several features. In particular, exernal URI resolving has been switched off, it required libraries with a viral (GPL/LGPL) license, which are avoided in Bullet.

COLLADA DOM 1.5 introduced Boost, which is way too bloated to be included in the Bullet source tree (and we want to stay self-contained), so no upgrade is made.

One option is to switch to OpenCollada SAX-based parser, in combination with expat XML library. It uses the MIT license, and hopefully it is more lightweight and enjoyable to use.
Thanks,
Erwin

The base of the parser is an auto generated, validating, COLLADA specific, xml parser, that alternatively uses libxml2 (fast) or Expat (little bit slower), both of them are MIT licensed. The C++ code for this parser is generated from the COLLADA XSD by a Java program. This parser has callback functions for each COLLADA element (SAX style parsing). One for the beginning of each element, one for the end of each element and, if character data is allowed inside this element, also one for that data. The begin functions parse the attributes, if any, convert them to the corresponding c++ data type according to the XSD and provide them in a struct. The names of the members in this struct correspond to the attribute names in COLLADA. Similarly, the parser parses the character data, if any, inside elements, converts it to the corresponding c++ data type according to the XSD and calls the data function with these values stored in an array. Since sax based xml parser do not necessarily pass all character data at once, that data function might be called multiple times for one element and provide the data in pieces (e.g. large float arrays are passed during several calls).
The end function is called when the element is closed and has no parameters.

On top of this auto generated parser we have implemented two more components. A data model (currently called COLLADAFramework) and a COLLADA parser (loader) which fills that model (currently called COLLADASaxFrameworkLoader).
The data model consists of atomic objects (Geometry, Scenegraph, LibraryNodes … ) which are passed to any software that wants load a dae file using this loader. To receive the objects, this software needs to implement the COLLADAFW::IWriter interface. The loader then calls the methods of this interface, as soon as an atomic object has been loaded completely, e.g. the writeGeometry method is called as soon as the end_geometry function of the generated
sax parser has been called. Therefore, the atomic objects are passed in the order they appear in the dae file, which requires that implementers must be able to handle them in arbitrary order. If this is not possible, one can parse the dae file more than once. For instance the COLLADAMaya plug-in uses this technique, since it needs to know the scene graph, before it can handle geometries. The methods of the interface pass pointers to the objects that are only
valid inside the call of this method. As soon as this method returns, the object might be deleted and the pointer becomes invalid. This approach reduces the memory consumption, since it needs to hold only one atomic object in memory a at time. Each atomic object has a unique id used to match objects and references to them. The COLLADASaxFrameworkLoader already resolves urls in COLLADA and uses them to assign these unique ids.

There is a very simple sample implementation for the Sax Parser available that converts DAE files into binary Ogre3D meshes that might server as a starting point for any game development.
http://opencollada.svn.sf.net/viewvc/opencollada/trunk/dae2ogre/

These libraries are all extremely fast (500MB DAE file parsing takes ~2 sec) and have no memory overhead. The SAX based approach is a bit more difficult to handle but the framework allows you to create an importer more reliable and faster than with the COLLADA DOM libraries. Not as simple as FBX yet but available under the MIT license.

Sebastian

Hi,

Your project is interesting but there is only a Visual studio version on sourceforge and I work with MinGW (gcc).
Where can I find a tutorial to compile it properly ?

Thanks.