Collada as a good format for (semi)automatic semantic segmentation

Hi all!

I am a Physics researcher involved in a project for Cultural Heritage 3D-scene fruition. I am interested in the (semi)automatic semantic segmentation of monument scenes created by photogrammetry.
While I am getting my first results (automatic identification of columns, at the moment, and sub-segmentation into shaft, capital etc), I am investigating on the file formats to use for my output.
I need to save labeled geometries (e.g., point clouds labeled as column shaft, column capital…) at different LODs, with hierarchy and relationships (the column is composed of shaft, capital…; something is over something else…) and attributes (materials etc).
At present the output files should be on disk, but later I’ll put them on remote databases, so that the good LOD of the desired parts can be retrieved.

I am absolutely new to Collada, and I am wondering if this file format can easily accommodate my needs (object labels with hierarchy, LODs, relationships, attributes).
Any opinion, hint, link, information is welcome!
Thanks
Best regards
Giorgio


Dr Giorgio De Nunzio
University of Salento
Lecce - Italy

You should have no issues to store that data in a valid COLLADA document. I don’t think you even need an extension.

The only thing is that you will be using a semantic that you cannot expect generic Collada applications to recognize. For example, you can store LOD information, but most applications would just load and display all the versions of the object. So you’d have to add extra scripting to tell the application what to do based on the information stored in the document - unless you can use this extension directly Instance node OpenCOLLADA extension - COLLADA Public Wiki

I rarely read this forum, but I’ll be happy to help if you reach me on email remi (at) acm (dot) org

Hi Remi, thank you very much for your kind reply!
I’ll distinguish the two problems, that is:

  1. storing semantic information according to a thesaurus I wish to use (e.g. labels such as Column Shaft, Column Capital, and so on, still to be defined, in a hierarchic approach)
  2. storing multiple versions of the objects, at different LODs.

As to problem (1), a visualizer should retrieve the information and assign it to the object to visualize; I imagine that a hierarchy can be inserted in a COLLADA file, and each level of the hierarchy may have a label. So, supposing a column is to be visualized, it should be partitioned into shaft, base, capital, each with the corresponding label attached. Is COLLADA able to to this out-of-the-box? Where can I find an example for dummies?
Sorry, what do you mean with “you will be using a semantic that you cannot expect generic Collada applications to recognize”? It is important that a COLLADA-aware application recognizes the hierarchy of objects/subobjects, and attaches my labels, but I don’t understand what other kind of “recognition” should be necessary.

As to problem (2), I suppose I can use the extension you pointed me to. Do you mean that - being it an extension - existing software is not necessarily able to interpret it somehow? I should take some COLLADA plugin and modify it accordingly?

As you can see, of course my first problem is to understand how COLLADA works starting from scratch:

  • a “hello world” example, how to write a file with a simple object in it (say a cube)
  • how to visualize it: is there some free software I can use?
  • libraries that use the format, e.g. in C++
  • how to insert a point cloud instead of the mentioned extra-simple cube
  • how to describe hierarchy
  • how to attach labels to objects (semantics)
  • how to manage extensions

This is quite a large program, I see, and I really need someone to point me to the right starting point. Of course I’ll begin to search myself, as soon as I can sit down and really work, because I am now out of office for research, but any hint is welcome!
Than you very much!
Best
Giorgio

@remi, I’m so glad to see you here,

@giorgio.denunzio, if you have Blender then you can export a simple CUBE or other scene as COLLADA, COLLADA is default exporter of Blender.

Most modeling and viewer softwares support COLLADA e.g. Blender, Maya, Apple Preview, FBX Review…

Also there are some open source libraries; OpenCOLLADA (C++), COLLADA-DOM (C++), AssetKit (C99, In Progress), Mick P. is working on COLLADA-DOM actively and I’m working on AssetKit actively, I hope a few month later COLLADA will have a new library (AssetKit) and improved COLLADA-DOM library

You can buid hierarchy by <node> elements, if your shaders need to access a label then you could use <input> to send label to shader, if not then I think giving a name, id or something else to <node> would be enough, but remi is author and he can say more

Hi recpas,
thank you very much for replying!
I’ve just downloaded Blender, so I’ll try what you suggest and begin experimenting. In the meantime I’ve also found a couple of “hello world” examples.
Thanks for all the other hints and information! I am looking forward to testing the existing libraries and your future one!
Best
Giorgio

Recpas asked that I reply.

To be honest, my advice to anyone is to only use COLLADA if you think there is no alternative but to. And if you feel you have no alternative, it shouldn’t be on the grounds of technical capabilities-rather, it should be on the grounds that you cannot see your project using a more proprietary product instead of a nominally non-proprietary product like COLLADA. And if you think you are in that boat you need to think long and hard about it because you’re venturing into a no man’s land.

If you use C++ it’s probably worth your while to check out the code I developed last year. There’s a snippet here (ColladaDOM 3 - COLLADA Public Wiki) where you can see an example that looks something like this:

//Example. If any successive node does not exist, then use Y_UP.
RT::Main.UpAxis = COLLADA->asset->up_axis->value->*RT::up::Y_UP;

What you can see in this example is in C++ you’re almost working natively in the XML data structure’s representation. A line of code like this can replace 20 or more lines of old code that would use to access this relatively simple variable. If the COLLADA element in this example wasn’t a “const” object then the <asset> and <up_axis> variables will have been created automatically.

I think COLLADA doesn’t seem like a natural fit for what you are doing, but I cannot tell you what would be. I would only recommend it for very basic video game and robotics modeling applications, and only then if it works. In its present state I’d only recommend it to masochists and idealists and maybe then only if they want to see COLLADA’s situation improved. The path to improvement could not be less clear either.

Even under the best circumstances you’ll find yourself trying to piece together what the initial designers were thinking. There are not central tenets to COLLADA. Sometimes it seems like there is 20 different addressing modes and none of them are ever used more than once. It’s maddening to be perfectly honest. Khronos has absolutely dropped the ball this time, and I think I see a lot of interest in the idea of COLLADA right now, if not the implementation in practice. I really wish Khronos would make an effort to resurrect this black sheep in this decade.

Good luck.

(I’m working on a reference implementation right now that visualizes rich COLLADA documents to the greatest degree COLLADA allows. Most software implements about 2% of COLLADA and calls it a day. It’s really only meant as a dialect of a language for clients and subcontractors to make bespoke modules talk to each other. As a kind of archival storage medium or exchange format–as people imagine it is–it has a long way to go. There is no ecosystem.)

[QUOTE=Mick P.;42097]Recpas asked that I reply.

To be honest, my advice to anyone is to only use COLLADA if you think there is no alternative but to. And if you feel you have no alternative, it shouldn’t be on the grounds of technical capabilities-rather, it should be on the grounds that you cannot see your project using a more proprietary product instead of a nominally non-proprietary product like COLLADA. And if you think you are in that boat you need to think long and hard about it because you’re venturing into a no man’s land.

If you use C++ it’s probably worth your while to check out the code I developed last year. There’s a snippet here (ColladaDOM 3 - COLLADA Public Wiki) where you can see an example that looks something like this:

//Example. If any successive node does not exist, then use Y_UP.
RT::Main.UpAxis = COLLADA->asset->up_axis->value->*RT::up::Y_UP;

What you can see in this example is in C++ you’re almost working natively in the XML data structure’s representation. A line of code like this can replace 20 or more lines of old code that would use to access this relatively simple variable. If the COLLADA element in this example wasn’t a “const” object then the <asset> and <up_axis> variables will have been created automatically.

I think COLLADA doesn’t seem like a natural fit for what you are doing, but I cannot tell you what would be. I would only recommend it for very basic video game and robotics modeling applications, and only then if it works. In its present state I’d only recommend it to masochists and idealists and maybe then only if they want to see COLLADA’s situation improved. The path to improvement could not be less clear either.

Even under the best circumstances you’ll find yourself trying to piece together what the initial designers were thinking. There are not central tenets to COLLADA. Sometimes it seems like there is 20 different addressing modes and none of them are ever used more than once. It’s maddening to be perfectly honest. Khronos has absolutely dropped the ball this time, and I think I see a lot of interest in the idea of COLLADA right now, if not the implementation in practice. I really wish Khronos would make an effort to resurrect this black sheep in this decade.

Good luck.

(I’m working on a reference implementation right now that visualizes rich COLLADA documents to the greatest degree COLLADA allows. Most software implements about 2% of COLLADA and calls it a day. It’s really only meant as a dialect of a language for clients and subcontractors to make bespoke modules talk to each other. As a kind of archival storage medium or exchange format–as people imagine it is–it has a long way to go. There is no ecosystem.)[/QUOTE]

Hi Mick,
thanks for replying!
I admit I’m quite… frightened…! I am not masochist, and I cannot take the wrong way because I’d have no time to recover.
There are alternatives, one of them being CityGML with appropriate extensions (because CityGML does not consider, per se, LODs as detailed as I need: many people are working on this, but - again - there is no standard).
Perhaps at the moment I need a simpler graphic format containing some hierarchy and semantics metadata, later embedding my files in more complex and complete environment, so deferring the decision to the future.
Thanks again
Best regards
Giorgio

Giorgio, there is also glTF format for runtime (GitHub - KhronosGroup/glTF: glTF – Runtime 3D Asset Delivery),

AFAIK, Cesium.js uses glTF to render 3D tiles (glTF + CityGML or more sources): http://cesiumjs.org/2015/08/10/Introducing-3D-Tiles/
I’m not expert of CityGML so I can’t go further, but it seems it fits with your needs

[QUOTE=giorgio.denunzio;42098]Hi Mick,
thanks for replying!
I admit I’m quite… frightened…! I am not masochist, and I cannot take the wrong way because I’d have no time to recover.
There are alternatives, one of them being CityGML with appropriate extensions (because CityGML does not consider, per se, LODs as detailed as I need: many people are working on this, but - again - there is no standard).
Perhaps at the moment I need a simpler graphic format containing some hierarchy and semantics metadata, later embedding my files in more complex and complete environment, so deferring the decision to the future.
Thanks again
Best regards
Giorgio[/QUOTE]

I may anger people to say so, but I think that’s almost definitely the right call.

How I would characterize COLLADA in its present state, is if you are hiring a contractor to develop a module just for internal use in your company; in which case you already have a good deal of money invested. Then what COLLADA can do for you in the best case scenario is it can save you and the contractor from having to agree on a language for representing the information you are communicating. Then the two parties can agree on internal extensions to COLLADA to fill in the holes in COLLADA. Then what you end up with is a dialect of a language that only two parties can speak in. And these parties are private companies, so the outside world will never learn of their software.

In that scenario COLLADA can save time and money by avoiding the first step of creating a language for software to speak in. That’s about it though.