messing with materials

I am having some troubles dealing with the materials used by collada… the problem is quite complex, so I’ll extend a bit, but first, an introduction to the problem:

basically, my 3d engine uses a custom multipass rendering system, it’s architecture is somewhat like the DirectX FX file format, and the big problem is that every material requires much more data than that defined in the standard collada 1.4 specs, but also that exported by any current plugin… well… I’m not only talking in terms of data complexity, but also in terms of data structure.

For example, in my engine I can define a material that has a procedural texture (eye-normal envmap) multiplied by a decal texture, then the result added to the screen with additive blending… so, how can I design a collada material that can be interpreted as this one, in a generic (non cgfx) way?

One thing that would really save my life would be to be able to assign a text script to a 3dsmax material (in the materials GUI), so I could set custom properties there), then, collada would export it as a “material_script” or something like that, then my engine would get that script and interpret it as it needs.

but, to begin with, I have no idea how to assign a random text to a max material, and, even I am able to, I don’t think any current plugin would be able to export it…

I bet many others are also thinking in this problems, so any ideas will be welcomed!

Those two things seem contradictory. Either you can use common/standard descriptions to represent the data that you need, and then use the standard tools, or you want to have your own custom representation, and have to extend the format, and extend the tools.

One way is to use the <extra> element, that enables you to add any XML formatted data at specific points in the COLLADA document. In theory you should be able to add <extra> elements within the DCC tool, and get those properly exporter/imported by COLLADA, but in practice some DCC tools are not designed to support this feature very well. We are aware of this and have made support of user data the highest priority for the plug-in work. I suggest you contact directly the plug-in provider, through their support system.

Another way is to use one of the standard FX profiles in COLLADA, such as the CG profile, and make sure that the information you need for your engine can be extracted from the information in the CG profile. This is probably the easiest way forward, since more and more tools are supporting those FX profiles.

I hope this helps

That’s exactly what I mean, theidea would be to do the latter (to use a CG profile), but in a generic way, instead of using the CG profile as a proxy.

Imagine the collada plugin adds a text edition window on every material, pretty much like the objects user data… there, the user could write custom material attributes, some shader code, or anything else required by the current project, at export/import time, this would show in a collada document as a simple <user_data> …text… </user_data> inside the material definition.

The advantage of this is that it is a generic solution, and developers do not need to reengineer their tools for every new project that needs new custom material properties.

I’ll give you a simple case scenario: what if I want to make a simple game, in which the player character looses 1p life every time it steps in a certain material? I can certainly do it by detecting the material the player has stepped…

But it is easier to leave these decisions to the level designer, who can effectively decide that the “Lava” material kills the player when it touches it, or the player is dragged to the left when it steps on a “left arrow” material… and the level designer can do it by just typing in the “material user data”.

What’s important to me (and one of the reasons I choose collada) was to not have to rewrite my tools for every new project…

Hope I have clarified a bit my questions… and thanks for your reply!

Hey vic, I wanted to say that I completely agree that custom data needs to be handled more intelligently by the tool vendors and plugin writers. I’ve looked into this issue a bit and will share my experiences.

How does the Collada plugin add stuff to Max’s material interface? By writing a Max material plugin and requiring artists to work with those? Max doesn’t have a nice way of extending an object’s interface without creating a whole new plugin for that object type, which is clumsy and laborious.

Here are a few possible solutions for what you’re looking for:

[ul]- Name hacks. If a material should represent lava that hurts the player when he touches it, require that the material name end in “_Lava”. Clumsy, hackish, non-scalable, but it works.

  • The last time I checked, a node’s user data area gets mapped to an <extra> block on the node in the Collada file. You could take advantage of this by doing the following:[list:1nhm7pw5]- Provide a utility plugin (written in MaxScript perhaps) that allows the artist to browse all the materials in the scene and assign custom properties to them. For example, have a button for “lava” that marks the currently selected material(s) as being lava materials.
    • When the user modifies some of these custom properties, don’t store them with the material itself (you’d have nowhere to put them), but instead create a hidden dummy node and store the data in that node’s user data area. Give the node a specific name (eg “MyCustomDataNode”) so your Collada importer can find it. For example, if the user selects the material named “Material1” and clicks your “Lava” button, dump the following data into the custom data node’s user data area:
      <Material1> Lava </Material1>
    • When importing a Collada document into your app, look for the custom data node and retrieve the custom data from the <extra> block it’ll contain.
    • Some warnings/caveats regarding this method:[list:1nhm7pw5]- The last time I checked (which was a few months ago), the ColladaMax importer didn’t import the <extra> block that it exported from the node’s user data area. It just threw the data out. A bug was reported on this and it may be fixed now.
      • The data in the node may become out of sync with the model. For example, you might have custom info pertaining to Material1, but what happens if the user renames Material1? Your custom data stored in the node won’t be updated appropriately.
      • Just to be clear, writing a utility plugin to provide a nice interface for your artist to generate the custom data isn’t strictly necessary. You could just require them to explicitly create the custom data node and modify its user data area.[/ul][/list:u:1nhm7pw5][/list:u:1nhm7pw5]Man that’s going to be formatted so poorly… oh well.

The nice thing about the methods above is that they don’t require you to modify the Collada importer/exporter in any way.

Most of the issues with importing/exporting custom Collada data from Max are due to the fact that, frankly, Max doesn’t have a very elegant architecture that’s easy to extend. Consider as a counter-example SoftImage and their architecture. SoftImage allows just about any object in their system to be extended by attaching custom parameter sets to the object. A parameter set is made up of a list of parameters, and a parameter has (amongst other attributes) a name (“Lava”), a type (“bool”), and a value (“true”). These parameter sets are dynamically creatable/modifiable/queryable, and can be created directly by the artist within SoftImage. Because the parameters are so self-descriptive, it’s easy for SoftImage to generate a gui for your custom parameter sets, and that’s exactly what it does: as an example, your “Lava” parameter would automatically be assigned a checkbox that the artist can toggle on/off as he pleases. This is possible because SoftImage knows that Lava is a boolean value.

SoftImage supports <extra> Collada data in both their importer/exporter by mapping <extra> blocks to their custom parameter sets, and this custom data has automatic gui support (described above), automatic save/load, automatic script access, etc etc. There were some bugs in the <extra> <–> parameter set mapping the last time I looked into it (see one of my links above), but they may have fixed the issues since then.

Anyway I don’t mean to sound like a SoftImage fanboy, but I definitely think they have the right idea as far as how to properly support custom data in their tool. Good luck with your project.

  • Steve

Hi sthomas, thanks for your sharing your thoughs, this is getting interesting :slight_smile:

yes, that’s the good, old, overused trick. Certainly, I could use it, but then, what’s the point of using collada if the format can’t solve these problems? I personally think that collada was designed to solve these problems, among others

I’ve read all your post about using the user data of hidden dummy nodes for storing custom properties, the idea is interesting, but I have done some research today, that might enlighten things a bit:

it is, in fact, possible to extend max material parameters using maxscript, and amazingly, it is a lot easier that I thought at first, being a complete n00b at maxscript, and using the maxscript plugin of the guys of Ogre3D project, I was able to design a completely new material inside max.

I have to say that the Ogre3D scripts only replace the material user interface with a new one, but they don’t add custom features.

Basically, what I did was a script that inherits the standard MAX material, in that class, you can completely rebuild / rearrange the material interface of MAX, also, you can add new custom properties, I verified that MAX properly loads/saves these custom properties correctly, when you load/save the project.

The difficult thing is to retrieve that custom properties. For that, there’s another script that loops all the materials in the scene, in the script, you can detect the type of material, and if you detect your custom material, you can read all it’s properties, including the inherited properties of the standard material AND your custom material properties.

I’m going to refine the scripts a bit, and I plan to upload them in a few days, so people interested will be able to discuss them.

Again, I am new to maxscript, but I think, the solution for all is this:

use a maxscript derived class to extend standard material

use another script with the next methods:

  • getParamsCount
  • GetParamName(idx)
  • GetParamValue(idx);

this script would be coupled with the material script… the collada plugin, upon finding this extended material, would simply call GetParamCount(), and then loop over all the params, and dump them to the EXTRA node.

this way, we could extend MAX materials using maxscript, with as many custom parameters as we wish, without having to rewrite the collada plugin

thanks for your replies!

Hey vic, I’m familiar with the approach you’re describing, in fact that’s what I was referring to when I said the following:

The benefits as I see it to the plugin approach are as follows:[ul]- You can provide a custom data interface directly with the object that’s meant to have the custom data associated with it. In my suggestion above you’d have a separate interface in the utility panel, which is less direct.

  • Since the custom data is stored directly with the object, you don’t need to update the data when the name of the material changes or the material is deleted, etc. In my approach, the data can become out of sync with the model.[/ul]But there are many drawbacks as well. Consider the following:[ul]- Since you’ve written a new material type (let’s refer to this new material type as a “ColladaMaterial”), those objects are totally distinct from normal Max materials. So you have to tell your artist to always create and work with those, which doesn’t conform to their normal workflow, or you tell your artist to create ColladaMaterials only when he wants to do any custom data. Imagine the artist creates a normal Max material and spends several minutes setting the material up and getting it right, but then he decides he wants to mark it as a lava material. Well, he can’t convert that material to a ColladaMaterial, so he has to create a new ColladaMaterial from scratch and redo the work he did to set up his material parameters. In the approach I described the artist would always be working with Max’s materials, so this wouldn’t be an issue.

  • What if you then want some custom data to be stored not with materials, but with textures? Or with animation controllers? Or with objects? Or with… well, I’m sure you get the idea. You’d need to write plugins for all of those types (and Max’s base “object” type isn’t pluginable iirc), which is very laborious. The plugin approach doesn’t scale well when you want to start storing custom data elsewhere. The approach I described is much more scalable.

  • The plugin approach requires directly modifying the Collada importer/exporter to get it working with your custom material type. Since the ColladaMax plugin is open-source that isn’t necessarily a deal-breaker, but it’s definitely a drawback. Also, you may need to modify the import/export code every time you change your custom data (say, add a new parameter). If you structure your custom data just as a big text field that the artist writes into, then you should be able to avoid having to modify the import/export code every time you change the custom data, but then the artist’s interface is sub-optimal (eg if it’s a bool parameter it should have a checkbox, not a text field). In my solution you don’t have to modify the ColladaMax plugins and can still provide a more natural user interface.[/ul]
    So both methods have benefits/drawbacks. At my company we needed custom data on nodes, materials, scenes, and maybe in other spots as well, so the lack of scalability of the plugin approach made it unsuitable for us. But if all you need is custom data on materials then the plugin approach should work well. Good luck, and be sure to let us know how it goes.

well, I agree with you in that both systems have benefits and drawbacks, also notice that I put the “lava material damage” as a sample of a material property that must be forcefully a custom property, but it is not my case right now.

Actually, my main problem right now is about defining materials for special effects, which require multiple passes, and they can’t be defined in MAX with its current standard format.

Another problem of letting artists use the standard MAX material “as is”, is that they are continuosly asking me which material properties are supported by the engine, and which not. I understand that not all artists must be software engineers, and sometimes I have a hard time trying to explain to an artist that he can’t simply use all the features of the standard material, because the engine does not support them. So, I think it is better to have a custom material that has exactly what the engine supports, so artists can know what they can do and what not.

from what i’ve been testing these days, it is possible to create a new material with custom properties, and still have the full properties set of the standard material, so, if I create a new custom material, the basic features of MAX will still show correctly in MAX, and theorically, these standard properties should still be readable by any plugin, only the custom features will be lost… (although it could be nice if the plugin could be able to detect them and export them too, I’ve noticed that maxscript has capabilities to allow it)

And, personally, I don’t need extra features beyond objects and materials… so, if this issue is solved, I’m served :slight_smile:

Vic