'ownership' tag

We’ve been discussing the difference between an interchange format and an export format. I think Collada ideally would do both jobs. To make that possible, I think there’s a necessary extension. First, here’s my premise:

To be an interchange format, Collada needs to express as much of the data as possible in a generic way (for example, vertex position as a 3d float vector is pretty universal! and animation can be a sequence of keyframes with orientation, position, and time). However, most art tools have data that embodies concepts that are not at all universal - like a modifier stack in MAX. To be an import/export format, Collada needs to support those as well. This will result in files with a lot of data that any 3D tool could read, and a lot of data in <Extra> tags that’s needed by the tool that created the file.

If Collada files are to serve as import/export files for a given art tool, they need to have -all- the data that tool requires. A lot of it will be in <Extra> tags, or tags not understood by a generic Collada parser.

If multiple art tools are to be able to use a Collada file as an interchange format, though, as much data as possible must be in common format. The problem arises when that common format isn’t really a reflection of how the generating art tool stores that data.

For example, I’m sure Collada will support data describing generic bone & keyframe-based animations. Any art tool should be able to display such animations. However, let’s say I have an art tool that generates animations by imposing physics on a hierarchical mesh. If I save that file as a Collada file, I’d like to be able to reload it and recover the data that describes the forces that drive my animations - <Extra><Force> etc. I’d expect a good art tool to put both common and tool-specific representations of the data in the same file. That way any tool could view my animations.
The problem comes when a different tool, with a different view of how to animate bones, is used to modify the file I created. Clearly, it won’t understand my <Force> tags.

To solve this, we could have an attribute or tag that claims ownership of data. This tag would declare that a section of Collada data is generated by a specific tool and is dependent on data that other tools may not recognize. In effect, it declares certain data “readonly” within a Collada file. This allows for interchange between a lot of tools.

The spec should also allow for a tool to override the ownership; if it does so, it must add a tag of its own so the original creation tool can detect that its custom data has been invalidated.

Here’s a potential way to do it:
<mesh tool=“parametricModeler” source=“sphereGen”> …
<Extra><sphereGen> <param > …

this implies that the “ParametricModeler” tool was used to describe a sphere; it’s saved as a <mesh> so other tools can display it, but that’s not its real format. If another tool modifies the mesh, it needs to tag it:

<mesh>
<Extra><sphereGen overridden=“PolyModeler”><param> …

so now if it’s loaded back into the ParametricModeler, it knows it can’t use the <sphereGen> representation any more.

At the point the data is overridden, it seems the data has become invalid. What is the point of saving it back into the file? There may be certain occasions on which this data is still valid, but I can’t think of any.

The ownership thing sounds similar to the <technique profile="blah> element already present in the spec. You can have multiple techniques per element.

Rob

You’re right, it doesn’t really need all the original data. By keeping some record of the change, it allows the original creator some measure of control - it may not be useful.
In my example, if someone opened up the file in a 3D paint program and modified the RGB channels of the verts, they would be obliged to signal that the parametricTool’s data is no longer valid. But if they haven’t changed the topology, it’s possible that the parametricTool has the capability of merging the RGB data into its own original data. Admittedly, this is a bit forced and may lead to more problems than it’s worth.

I’m aiming for a workflow where, for example, a model comes from Max and goes to Maya for animation; it will also be processed by a 3D lighting & texturing tool. We don’t want to lose the rigging that Maya has put into the file; we don’t want to lose Max modifiers, and we don’t want to lose the extra data from the texturer. Currently we must do it in an assembly-line fashion, since each tool loses some data from the previous one, or have each tool export some subset of the data that our engine will merge later.

I’m writing a reader (and later writer) which loads Collada files into a generic internal binary format to potentially allow modifications to the data. I’m supporting something similar to what you suggest (keeping around data for profiles I don’t understand) by duplicating verbatim the XML tree in memory for the unknown techniques. Then I can write the data back out unchanged. My assumption is that if I were to change the data under a specific profile, I would then discard all profiles my reader doesn’t understand, as Rob suggested.

I had also thought of the exact same idea you suggested (marking certain profiles as “overridden”) rather than just deleting them. This would allow the tool in question to decide whether it was possible to “merge” changes back in from the common profile into its specific profile or not. Whereas if you just delete it, the information is permanently lost. For example, in the “multitexture.xml” example file, you could have a tool that modifies the “TEXCOORD” array in the “COMMON” profile. Obviously the tool might not understand the “UV” arrays in the “EQUINOX-3D” profile, but it could still leave them around.

Then “EQUINOX-3D” could load in the file and it would realize that the texture coordinates in the common profile changed. It could decide that it can actually rebuild the “UV” coordinates by applying an inverse of the “U-V” transform to the “TEXCOORD” array. Then the potentially valuable U-V transform information is maintained. The default behavior in most tools would be to just discard overridden data, but it would always be possible to add a “smart” behavior. If you think your tool has completely clobbered a specific data block, you also have the option to nuke the other profiles as if they were never there.

I think it’s a good idea. I also like the idea of including utilities in the COLLADA source to support verbatim copying and storage of XML subtrees (I had to write this code myself), since it would be useful for any tool that wanted to be non-destructive of unknown data.

I’m writing a reader (and later writer) which loads Collada files into a generic internal binary format to potentially allow modifications to the data. I’m supporting something similar to what you suggest (keeping around data for profiles I don’t understand) by duplicating verbatim the XML tree in memory for the unknown techniques. Then I can write the data back out unchanged.

Actually, this is required from conformant import/export tools. [We will soon publish a conformance test as part of the Collada package].

Another way to support this feature is to remember to copy/paste the info from the input to the output when exporting the file. The data does not have to be loaded in memory.

This would allow the tool in question to decide whether it was possible to “merge” changes back in from the common profile into its specific profile or not.

And in order to detect when multiple representation of the same data is out-of sync, you have to use the <asset> tag, So your tool can detect which version of the data is the most up to date.
This is the basis to implement build systems / makefile and asset management.

I also like the idea of including utilities in the COLLADA source to support verbatim copying and storage of XML subtrees (I had to write this code myself), since it would be useful for any tool that wanted to be non-destructive of unknown data.

Yes, good idea.
This can probably be handled as a constant string.