Options for flattening node hierarchy?

[donmccurdy]

One area with room for performance improvement: glTF results in a fairly deep node hierarchy, even for simple (single-mesh) models. For example, Box.gltf only contains a single node entry, but results in the following three.js node tree:

• Scene (userland scene)
  • Scene (glTF scene root)
    • Object3D (root node)
      • Object3D (not obvious where this came from...)
        • Group (derived from glTF mesh entry)
          • Mesh (derived from glTF primitive entry)

I think that 2nd Object3D is not required by glTF, and may be a bug in THREE.GLTF2Loader. But the rest is derived directly from the glTF file, and could have name, extras, or transforms attached, so it's not trivial to throw any more away.

Userland code can certainly treat the glTF as a container, and pluck the Mesh out from the bottom of the tree. But ideally viewers and frameworks like A-Frame would be able to treat the glTF as a black box, and expect reasonably efficient results.

Questions for discussion, then:

• Is it allowed/correct for a loader to discard nodes that don't have useful data attached, like this?
• Do we want to make any allowances (implementation notes, future spec metadata) that would help enable delivering flatter node trees?

[stevenvergenz]

Ultimately, I think this is an implementation detail. If consumers potentially want the raw data out of a glTF document, those could be supplied out-of-band to the scene hierarchy, like the camera list is. It's also going to depend on how geometry is represented in the engine. Not all engines support transforms on mesh objects.

Something we did in UnityGLTF was to try to do smart primitive detection. So if a node has a mesh, and the mesh has only a single primitive, the geometry is attached directly to the node (i.e. node is THREE.Mesh not THREE.Object3D). Similarly, on export, if a node has any children with identity transforms and meshes, they are represented as primitives in a collective mesh instead of meshes themselves.

So bottom line, optimize importers/exporters, not the spec. Maybe it merits an implementation note?

[donmccurdy]

I agree there is no easy answer, but in the Box.gltf example the exporter has already done everything possible to minimize the node tree (there is only one node), and the mesh comes out 5 levels deep despite that, because of the various places name, extras, or transforms could potentially be attached. Perhaps this is overly simplifying, but it would be ideal if there were something an exporter could do to indicate that it intends to simply deliver an un-transformed mesh.

If not, then yes I can change the three.js loader implementation and throw out all of those intermediate nodes, just returning a Mesh, or as near as I can get. For example, merge the mesh and its parent node into a single output. But, a user might intentionally include nested nodes with no transforms, with the intention of creating pivoted transforms at runtime.

So having a loader automatically flatten things seems tricky, borderline non-conformant with the spec, and I would like to get feedback here if there are other options.

[stevenvergenz]

To clarify, the loader doesn't (and must not) flatten any glTF nodes. It just doesn't generate additional game objects for the logical objects mesh and primitive. At least, not if it can help it.

That's a good point about extras and name, those wouldn't be present in the scene hierarchy in my system. But I think that's okay. I would expect those fields to be used by the loader, and not by the consumer app.

[pjcozzi]

I created a best practices label for this (and any other issue) so we can track ideas that may go into a best practices guide or appendix as @sbtron once suggested. Please label any other issues accordingly.

[zellski]

@stevenvergenz I don't quite understand the distinction between eliminating glTF nodes and opting not to create game objects for them. Are you presuming an internal model that sits between the glTF and the resulting model, or perhaps a loader that takes care of traversal but offloads actual in-game object instantiation to the caller? For myself, I just want to be able to optimize the runtime hierarchy as much as possible, while respecting the user's reasonable expectations of consistency. It's unfortunate if an optional but very-rarely-used extra field, for example, means that a loader has to choose between either inflating the scene graph depth for all loaded models or producing surprising topologies depending on the existence or non-existence of extras.

[stevenvergenz]

I'm arguing against the common practice of creating hierarchy levels that represent glTF meshes and primitives. While these structures are defined separately in the glTF description, they do not describe the same thing as nodes or scenes. No one is proposing adding materials as nodes in the hierarchy, are they? As such, UnityGLTF produces one GameObject for each glTF scene/node, and if a node has a mesh defined, that GameObject has a MeshFilter component representing the mesh. No extra hierarchy levels there.

The same idea can be applied to OP's Three.js. Generate Object3Ds for all scenes and nodes, but if a node has a mesh defined, generate a Mesh to represent it instead. Meshes inherit from Object3D anyway, so no data lost, and no extra hierarchy levels.

Similarly, multi-prim meshes can be represented using Unity submeshes or Three.js geometry groups. This is a little more complicated in practice, but seems to map conceptually pretty well to glTF primitives.

If these recommendations are implemented, OP's scene hierarchy would look like this:

• Scene (userland scene)
  • Scene (glTF scene root)
    • Mesh (glTF node)
      • BufferGeometry (mesh)
        • groups (primitives)
      • Material[] (primitive materials)

For the sake of compatibility, the loader could stick the full glTF descriptions into userData or something, in case extras are needed. Does this explanation make more sense?

[donmccurdy]

Similarly, multi-prim meshes can be represented using Unity submeshes or Three.js geometry groups.

I didn't realize three.js had geometry groups... that might help quite a bit here. 😀

The only issue I see with your proposed hierarchy is that both the glTF node and glTF mesh might have extras or name. We already try to preserve those as userData and name in THREE.GLTF2Loader, and i don't think BufferGeometry has those properties. So, options:

• (a) we still need one Object3D wrapping the Mesh.
• (b) forcibly put userData and name onto the BufferGeometry, even if it's non-default in three.js.
• (c) decide that three.js ignores extras and name on anything that isn't a node, which is already true-ish because things like glTF buffers (which could have extras too) are not exposed in the GLTF2Loader output at all.

[stevenvergenz]

I think a passable solution here is for the loader to return two things: a parsed glTF document with each item including a reference to its corresponding Three.js data structure, and the loaded scene. Heck, maybe just return the parsed document, and let the consumer application grab the scene they want out of it, e.g. parsedDocument.scenes[0].

[donmccurdy]

Heck, maybe just return the parsed document, and let the consumer application grab the scene they want out of it, e.g. parsedDocument.scenes[0].

We (sort of) do that already, returning the array of scenes, but we don't separately make the meshes and materials available. Not sure if that could be done without also potentially duplicating some data... But in any case, it seems like there are reasonable things we can do in the three.js implementation, without needing changes or clarification from the spec, so we can close this issue and continue discussion in mrdoob/three.js#11944.

Thanks all!