Maya Importer - Handling Floats

I’ve tried using the Maya importer. Overall its looks good, but it appears to have an achilles heel in its decoding of floats.

It decodes the array by finding the next token in the string and converting this substring into a float using its own ‘strtofloat’ routine. This routine doesn’t handle exponents.

Is this intentional or will it be fixed at some point?

It is for speed reasons. Using sscanf can be 5000 (yes that’s 5 an 3 zeroes!) slower than implementing your own float parser.
Even if you don’t use sscanf, adding support for the scientific notation will slow down the importer.
When you read 10s of millions of floats (e.g. a game scene) from an XML file, it makes a huge difference.
For this reason, the reference importer doesn’t support exponents either and I’m guessing that the Maya importer cut-pasted the converter routine.
It is up for discussion whether it is worth supporting exponents, at the cost of giving up importing speed.

You’re both very right - it doesn’t handle scientific notation, and it was cut and paste from the collada reference code. My personal view is that we will need to support this notation given we’re an interchange format - but I think the decision on whether or not we outlaw it for performance reasons needs to be made at the COLLADA specification level, not by individual importer/exporters or we’re going to lose interchangability. So, assuming we don’t decide to outlaw exponents, I’d say Gabor needs to add it to the standard code so I can cut and paste a new version =)

You’re right Gordon. We need to decide on this at the spec level. That’s what I meant by “up for discussion”.
We need to do some tests, because we need “hard numbers” to make that decision.
I’ll do some tests with the reference code…

Is it possible to have another option:

use format “float” for data written without an exponent, and another format “float_exp” or something for applications where it is felt necessary?

I like that.
What do you think Gordon?

I’d rather not create new primitive type beyond what XML Schema Language defines for numbers. In this case the type is xs:double and that is IEEE 754 double precision numbers including exponent notation. For this speedy implementation to be valid the schema who need to change the type to be xs:decimal.

My version of the Collada spec lists ‘int, float, Name and token’ as the only valid types for array elements. Should ‘double’ be included as well then?

No because COLLADA float is typed as xs:double. As a Unicode format, COLLADA doesn’t define many sizes of primitive types since that has little meaning in textual representation. A floating point number is the main distinction we make for data validation purposes.

How about atof and family…? It has WAY less overhead than sscanf, and should be somewhere equivalent in speed to something you could do yourself…

Just a suggestion

Adruab

[quote=“marcus”]

No because COLLADA float is typed as xs:double. As a Unicode format, COLLADA doesn’t define many sizes of primitive types since that has little meaning in textual representation. A floating point number is the main distinction we make for data validation purposes.[/quote]

I’m probably being a bit slow here but does this mean:

Decoders for COLLADA “float” should handle exponents as its defined as xs:double

For speedy implementation the option to use another type based on xs:decimal should be available instead?

Yes, COLLADA floats can have exponents now.

I’m not suggesting that xs:decimal be an option for COLLADA floats, but rather a replacement of xs:double in the <array> schema. If we all agree to do this then no implementation needs to parse engineering notation for COLLADA floats.