GLTFLoader: Extensibility

[Jeremy-Gaillard]

Hi,

We are currently implement b3dm (a format that encapsulates glTF) support in our project and have encountered a few problems with the current GLTFLoader implementation.

We need to support addition attribute (batchid, used to identify the different objects packed in the geometry), extensions (CESIUM_RTC) and potentially additional uniforms. However, there does not seem to be any way to add these new features without modifying the GLTFLoader file itself.

So my question is: how to support the extensible nature of glTF? Do we contribute directly to the source and add the attributes/extensions/uniforms that we need? Or does the loader need to be modified so extensions can be "plugged in" without changing the source?

Related question: when loading glTFs in a scene rendered using a logarithmic depth buffer, the model's shader are not modified to account for it, resulting in depth issues: http://jsfiddle.net/x6ufnz3y/. I implemented a small hack to fix the problem. Should this process be added to the loader natively? Or could Three provide a helper function to patch the materials?

[donmccurdy]

So my question is: how to support the extensible nature of glTF?

Hi @Jeremy-Gaillard — I like the idea of modifying the glTF loader to support user-defined extensions, rather than just the extensions we've implemented ourselves. This would make it much easier to support draft extensions like #11577. Definite +1 from me on that. Hopefully we could do this by having a common interface for all extensions with methods like onAfterTexture, onAfterNode, ....

However, I think we would be implementing that extensibility on GLTF2Loader, and not GLTFLoader — are you able to upgrade to the newer loader?

when loading glTFs in a scene rendered using a logarithmic depth buffer, the model's shader are not modified to account for it, resulting in depth issues...

This seems specific to glTF1, as glTF2 does not have custom shaders currently. If it's not too much trouble, it may be best to keep your hack as-is, and if it's still an issue when GLTF2Loader gets custom shader support later, we'll figure it out then. Most development work is on GLTF2Loader at this point.

Some background on the shader thing: custom GLSL shaders were part of the core glTF1.0 specification, and the default export format in COLLADA2GLTF. Since glTF2.0, they've been moved to an optional extension, and that extension hasn't been fully defined yet. The default now is metal-rough PBR, with options for spec-gloss PBR or blinn-phong.

[Jeremy-Gaillard]

Hopefully we could do this by having a common interface for all extensions with methods like onAfterTexture, onAfterNode, ....

Yes, that is the kind of feature that I need. But I wonder if it would be enough, especially for nested processes (like this switch). Maybe refactoring some of the code could help us. For example, we could replace the switch with something like this:

// Create a list of supported attributes
// The list can be modified by the user to add new attributes
this.supportedAttributes = {};
this.supportedAttributes['POSITION'] = {
   triangleAddAttribute: (geometry, bufferAttribute) => geometry.addAttribute( 'position', bufferAttribute ),
   lineAddAttribute: (geometry, bufferAttribute) => geometry.addAttribute( 'position', bufferAttribute ),
   replaceShader: (shaderText, regEx) => shaderText = shaderText.replace( regEx, 'position' )
};
...
// Replace switch l.1492 by this
this.supportedAttributes[semantic].replaceShader(shaderText, regEx);
// Replace switch l.2134 by this
this.supportedAttributes[attributeId].triangleAddAttribute(geometry, bufferAttribute);
// Replace switch l.2311 by this
this.supportedAttributes[attributeId].lineAddAttribute(geometry, bufferAttribute);

However, I think we would be implementing that extensibility on GLTF2Loader, and not GLTFLoader — are you able to upgrade to the newer loader?
Since glTF2.0, they've been moved to an optional extension

Good to know. Upgrading to GLTF2Loader should not be a problem.

[donmccurdy]

For custom attributes, a syntax like this might work?

class MyAttributeExtension extends GLTFExtension {
  constructor ( json ) {
    this.json = json;
  }
  onAfterGeometry ( primitive, dependencies, geometry ) {
    if ( primitive.attributes[ 'myAttribute' ] ) {
      geometry.addAttribute( 'myAttribute', bufferAttribute );
    }
  }
}

const loader = new GLTFLoader();
loader.registerExtension(MyExtension);
loader.load('foo.gltf', function () {
  // ...
});

As for patching the shaders, replaceShader is dead code in GLTF2Loader — there are no custom shaders in glTF2.0 yet, just PBR and blinn-phong, so no real way to test this. Until custom shaders are supported, you'd probably be better off writing an extension that creates a THREE.ShaderMaterial itself, like GLTFMaterialsPbrSpecularGlossinessExtension does now. Or just sit tight with what you have working already. 🙂

[donmccurdy]

@stevenvergenz I'd be interested in your thoughts on this sort of extension syntax, for making it easier to experiment on the AVR_* extensions.

[stevenvergenz]

@donmccurdy I really like the idea of having free-standing extension support, without having to mess with the core loader. If you can define enough functionality hooks to cover the extension surface, I'd be happy to use the AVR extensions as a test.

[donmccurdy]

Copying over @mrdoob's comment from #12761, we could consider returning the entire JSON object as part of GLTFLoader's response.. doesn't solve all use cases but does allow some self-service custom behavior.