WebGPURenderer: ShaderMaterial support prototype

[takahirox]

You might like to wait for node base material for WebGPU and/or WGSL, but let me share my prototype work because I want to hear your folks opinion and feedback.

This draft PR adds ShaderMaterial support to WebGPURenderer.

Background

We would like the usage of node based materials right from the beginning #20254 (comment). I agree with that, and we maybe shouldn't add new material support now.

But some basic features 3D graphics engine should have require some freedom of writing special shaders for them, for example textured background and post-processing.

We can't start to work on them until node based material system for WebGPU lands. But it may be a bit of unclear when it does. If we support ShaderMaterial we can move such basic features support forward.

And even if node based material lands I think there will still be some demands that user wants to directly write their shader codes. ShaderMaterial support can satisfy it.

Plus, currently node based material is compiled and it is handled as ShaderMaterial in (webgl) renderer. WebGL/WebGPU needs raw shader code so this design may not be so bad. If we do the same approach even in WebGPU renderer, we need ShaderMaterial support.

So I think adding ShaderMaterial support into WebGPU renderer now would be ok to me. What do you think?

API

We may need to brush up when node based material system for WebGPU lands.

Update: I noticed that @Mugen87 made UniformBufferObject support PR #15562 before (it hasn't been merged yet). Using it may be more right way?

// Based on existing ShaderMaterial API
const texture = new TextureLoader.load( url );
const material = new ShaderMaterial( {
  // No need of binding number in uniforms because
  // renderer parses shader code and analyzes uniforms
  uniforms: {
    // uniform.value takes an array for Uniform block
    timeUniforms: {
      value: [
        0
      ]
    },
    colorUniforms: {
      value: [
        new THREE.Color( 0x888888 )
      ]
    },
    mySampler: { value: texture },
    myTexture: { value: texture }
  },
  // common headers like position attribute declaration are added
  // to vertex and fragment shaders respectively in renderer.
  // Need idea: I don't really like that binding number in user shader code
  // needs to start with non-zero...
  vertexShader: `
    layout(set = 0, binding = 2) uniform TimeUniforms {
      float time;
    } timeUniforms;

    void main(){
      vUv = uv;
      float time = timeUniforms.time;
      gl_Position = cameraUniforms.projectionMatrix * modelUniforms.modelViewMatrix
        * vec4( sin( time * 0.5 + 0.5 ) * position, 1.0 );
    }
  `,
  fragmentShader: `
    layout(set = 0, binding = 2) uniform TimeUniforms {
      float time;
    } timeUniforms;

    layout(set = 0, binding = 3) uniform ColorUniforms {
      vec3 color;
    } colorUniforms;

    layout(set = 0, binding = 4) uniform sampler mySampler;
    layout(set = 0, binding = 5) uniform texture2D myTexture;

    void main() {
      float time = timeUniforms.time;
      vec4 texelColor = texture( sampler2D( myTexture, mySampler ), vUv );
      outColor = vec4( texelColor.rgb * colorUniforms.color * ( sin( time ) * 0.5 + 0.5 ), texelColor.a );
    }
  `
} );

function render() {
  // update uniform value
  material.uniforms.timeUniforms.value[0] += clock.getDelta();
  renderer.render( scene, camera );
}

Sample video

https://twitter.com/superhoge/status/1304299232182980608

Implementation note

I add inline.

[Mugen87]

So I think adding ShaderMaterial support into WebGPU renderer now would be ok to me. What do you think?

Um, I'm not sure about this. I was hoping we could go for a pure node based approach and focus the renderer's architecture on it. I think we should give @sunag a chance to implement this in a very basic manner. Especially since there is no need to rush with the WebGPURenderer implementation.

But some basic features 3D graphics engine should have require some freedom of writing special shaders for them, for example textured background and post-processing.

I have the feeling we can implement the JS logic for stuff like background and post-processing with temporary material definitions like we do now. The problem is that with a new ShaderMaterial class, the engine also requires a shader chunk system. I was hoping we could get away from the current one and build a new system based on node material.

[takahirox]

And even if node based material lands I think there will still be some demands that user wants to directly write their shader codes. ShaderMaterial support can satisfy it.

What do you think of?

[Mugen87]

I'm not very experienced with NodeMaterial but I've seen users like @martinRenou who actually write custom shader code with it. It would be great if the engine can provide the user a single tool (NodeMaterial) for their shader definitions.

Yes, it is a paradigm shift but I believe it is the more versatile solution than starting over with RawShaderMaterial and ShaderMaterial.

I'm not sure how @mrdoob and the others feel about this but I guess this is a good opportunity to discuss it 👍 .

[Mugen87]

BTW: I'm going to study your PR in detail over the weekend^^.

[martinRenou]

I'm not very experienced with NodeMaterial but I've seen users like @martinRenou who actually write custom shader code with it

I am a big fan of the NodeMaterial, especially because it provides a very nice API for building a shader easily. It is super user-friendly, and very modular. It does not provide as much freedom as the RawShaderMaterial and ShaderMaterial, but I think we can further improve the NodeMaterial to add more freedom. Following this path, I recently added a BasicNodeMaterial which is closer to what the ShaderMaterial is.

[mrdoob]

Um, I'm not sure about this. I was hoping we could go for a pure node based approach and focus the renderer's architecture on it.

Yes, as @Mugen87 already proposed in the first PR, better to focus on NodeMaterial for the new renderer.

[Mugen87]

@takahirox In any event, I see some bits of the PR we still might want to adapt to WebGPURenderer. I'll have more time over the weekend to give detailed feedback.

[takahirox]

Thanks for the comments. Good to know that BasicNodeMaterial offers similar freedom to ShaderMaterial. And if renderer is aware of node based material system, node material won't need to go through ShaderMaterial. So I withdraw my suggestion.

@takahirox In any event, I see some bits of the PR we still might want to adapt to WebGPURenderer. I'll have more time over the weekend to give detailed feedback.

Thanks. Even with node based material system, it finally needs to be compiled to raw shader code. So partly some codes or ideas in this PR can still be usable for example parsing and analyzing uniforms of shader code. And

we can implement the JS logic for stuff like background and post-processing with temporary material definitions like we do now.

this ShaderMaterial support (without code chunk system) might be good for renderer internal use, handling such temporal materials, until node based material system lands. Temporarily and internally using ShaderMaterial might be more flexible and simpler than defining such temporal materials each. ShaderMaterial specific code isn't so huge then replacing when node based material lands may not so hurt.

Anyways, I'm willing to explain my code if you couldn't get. Let me know anytime.

[takahirox]

Ugh, I need better name.

[Mugen87]

Let's do the review step by step^^.

This is very good suggestion which makes the vertex buffer setup more robust. Code like:

layout(location = 1) in vec3 position;
layout(location = 0) in vec2 uv;

now works too.

I've applied your code in dfe538e. However, it was necessary to use slot instead of shaderLocation (since it is undefined on attribute level).

[sunag]

To advance in NodeMaterial I think that we need create a ShaderBuilder to generate uniforms, in and out and prevent secondarys Regular Expressions.

Something similar to that

let builder = new ShaderBuilder();
let uv = builder.getInput( 'v2', 'uv' );
let position = builder.getInput( 'v3', 'position' );

console.log( uv ); // layout(location = 0) in vec2 uv;
console.log( position ); // layout(location = 1) in vec3 position;

This automatic manager the inputs of the shader, it is part of NodeBuilder.

[Mugen87]

In this case,ShaderBuilder would have similar logic which is currently implemented in WebGPURenderPipelines across various methods. So the idea is that ShaderBuilder provides the information which are necessary to build the render pipeline and bindings.

[Mugen87]

This was also a good inspiration! 😉

I've refactored the UBO updates based on your code here: d5c3089

The idea is to encapsulate each uniform type into a new class (check out WebGPUUniform.js) which encodes the byte length and item size. Besides, setValue() can be used to set the actual value of the uniform at runtime.

This means it's now nicer to compute the total byte size of the UBO and the offset value required for writing at the correct position in the buffer can be computed automatically.

[takahirox]

Did you adopt all the code you want? If so I close this PR.

[Mugen87]

Did you adopt all the code you want?

Yes. I'd like to wait for a basic node material before investing more time in this part of the renderer.

In any event, your code was a big help in pushing WebGPURenderer forward 👍 .

[takahirox]

I'm inspired by the API in #20390 and I locally made RawShaderMaterial support branch for personal shader testing purpose.

The changes are just +11 LOC.

In user code, set the following properties of RawShaderMaterial

vertexShader: vertex shader code strings,
fragmentShader: fragment shader code strings,
bindings: an array of webgpu bindings

then you can use your custom shader.

See webgpu_shader.html for more detail.

The advantage is you can test custom shader code without polluting WebGPU classes. It would be good for testing until node based material system lands.

Live demo: https://raw.githack.com/takahirox/three.js/WebGPUCustomShader/examples/#webgpu_shader

If you are interested in, please locally take this change.