NodeMaterial

[sunag]

Hi.

I started developing of a THREE.NodeMaterial to reconcile the differences materials between 3D authoring software. In SEA3D Studio has options to create layers in Albedo with mask and various blend modes, Rim shader and others without need custom shader code. I would like to bring this to Three.JS with node shader.

I think that MeshPhongMaterial, MeshPhysicalMaterial and others can easily be based on NodeMaterial through of a interface for backward compatibility or proxy only.

UPDATED
http://sunag.github.io/sea3d/Labs/Three.JS-NodeMaterial/webgl_materials_nodes.html
http://sunag.github.io/sea3d/Labs/Three.JS-NodeMaterial/webgl_postprocessing_nodes.html

Syntax example for uses UV1 or UV2 for texture:

var usesUv2 = true;
var isLightmap = false;

var t = new THREE.NodeTexture( texture, new THREE.NodeUV( usesUv2 ) );

var nodemat = new THREE.NodePhongMaterial();
if (isLightmap) nodemat.light = t;
else nodemat.color = t;
nodemat.build(); // build shader

I am making an editor too, currently this would be the interface. Color is albedo and transform is the vertex position.

I am also taking care that it can be used in a deferred shading. Now I will create reflection and refraction inputs.

Will be sharing the news to the PR, suggestions, tests and enhancement are welcome 👍

[mrdoob]

Interesting!

[bhouston]

@sunag We are very interested in something like this. How can I help? I can add support for Standard at least.

[bhouston]

I am interested in creating fairly arbitrary graphs, so that the intermediate nodes also take inputs. So you can have a graph that looks like this:

A Texture(tex1, uv1)
B Texture(tex2, uv2)
C Blend(A,B, mode)
D Noise(param1, param2)
E Blend(C,D, mode)

And then use that final node E, as an input to a Material.

So it is very arbitrary, not limited to just textures.

[bhouston]

My goal would be to help create this in the next few weeks, hopefully collaborating with you in the next week or so if possible. I was looking at doing this via the shadergraph library here: #7339 But I am find creating it within ThreeJS directly. Really any solution is good as long as it is flexible and it works.

[bhouston]

I'm reading through your code, it is quite nicely designed. I have some initial feedback. Could I start a PR using your code as a base start collaborating on it?

(1) I'd have the material resolve the references. Basically references would have names that they would ask their material to resolve, and the material would give back a snippest of code on how to access that data. This would also allow the material to know what variables (uniform/varyings) are used the nodes, so it can optimize appropriately. This also allows for different materials to resolve references differently, thus making the nodes more portable, rather than having to know how to the materials implement things, especially when there are differences between fragment and vertex implementations.

(2) I try to use the GeometryContext object I created in the lights refactor, it gives consistent access to a lot of the required local variables. But of course that can be resolved by the material itself.

(3) I've have UV just be another reference, which is resolved by the material. And I would have NodeTexture should actually take a Node Input, thus allowing for procedurally generated UVs.

(4) I would call NodeCube, NodeTextureCube to be consistent with the rest of Three.JS. And I would remove the logic on how to actually go the ray casts from it. But I like the idea of standard cube maps, so I would actually not put the environment query for specular or diffuse in the nodes, but have that in the base phong material, and you can only control the normal used for the query, or the cubemap result itself (thus allowing it to be a procedurally determined color.) Does that make sense? So I would have the normal pluggable in the material and the cube texture pluggable (queriable by a direction and bias/lod, and returns a color). Thus one can provide a cube texture to the irradiance map and another to the specular map. We can swap out true sampleCube with @tschw's cubeToUV2 function as just a node swap.

(5) I'd try to add a NodeFunction that allows one to call arbitrary functions with parameters as an addition to your NodeOp (or maybe they could be merged in some fashion.)

(6) I'd get rid of all of the verbose NodeNormal, NodeTransform, NormalMap, etc individual class and just have some simple constructors that create a NodeReference with a name that is resolved by the material as appropriate. NodeReference could resolve uniforms, varyings as well as computed values in the shader.

(7) I do not understand the difference between NodeEnvironment and NodeCube. I think NodeEnvironment may be incomplete?

(8) It is confusing to have NodePhong not be derived from NodeMaterial. Although I see that NodeMaterial is derived from ShaderMaterial. I wonder if you called the direct derivative from ShaderMaterial, GraphMaterial (or NodeGraphMaterial) that would make more sense -- because all together the nodes form a graph, and it is the graph that becomes the material, not an individual node.

(9) I would suggest maybe some more varied terminology. I'd call the root node, MaterialNode, and one could derive PhongMaterialNode from it. I've have Vector3Node, FloatNode, etc derived from ValueNode -- not necessarily constant, but just a value. Thus one could pipe in three FloatNodes to a Vector3Node. I think you can have a helper that would make declaring each of these one line or so rather than the 10 or so currently.

[bhouston]

(10) I would move the name "Node" from the start of the class names to the back because that is how it is in the rest of the ThreeJS project.

(11) I would create the new MaterialNode class and it would be initialized with a list of uniforms and varyings. It would be default be able to resolve this and also track which it has resolved so one can track which features are needed. One could thus have a limited resolve in the derived PhongMaterialNode that would resolve the special cases and rely on the underlying class to do the simple ones (varyings, uniforms.)

(12) I am sort of confused between the difference between NodePhong and NodePhongMaterial. I didn't realize there was both until now.

[bhouston]

(13) there is code like this:

THREE.NodeGLPosition.prototype = Object.create( THREE.Node.prototype );
THREE.NodeGLPosition.prototype.constructor = THREE.NodeGLPosition;

THREE.NodeGL.prototype.generate = function( material, shader ) {

But above this snippet you already defined generate for NodeGL and you didn't define one for NodeGLPosition -- thus I think it is a copy-paste-edit error.

[bhouston]

(14) I would get rid of NodeReflectUVW and NodeRefractVector and instead just make this something one can request from the material via a Reference resolve. Calculating a reflection vector is straight forward. I have added it to GeometryContext in my unmerged experimental ThreeJS branches.

[bhouston]

(15) The way I would implement reflection and refraction would be to have them as color inputs on the Material. One would resolve Refect, ReflectLOD, and Refract, RefractLOD in the simple way you would resolve any variable, and then pass them into one's texture cube equivalent (procedural or samplerCube-based) and then pass the resulting color into Material. Is that how you were doing it?

[bhouston]

(16) I'm confused about the light input -- usually one doesn't have lights being pluggable, rather the light parameters are fully defined in the light class. I guess you need this additional flexibility? How do you envision it.

[sunag]

@bhouston woow, thank you very much feedback.
I will need several posts to answer :)

I am interested in creating fairly arbitrary graphs, so that the intermediate nodes also take inputs. So you can have a graph that looks like this:
A Texture(tex1, uv1)
B Texture(tex2, uv2)
C Blend(A,B, mode)
D Noise(param1, param2)
E Blend(C,D, mode)

Currently the syntax like this. Uv1 offset animate example:
I think that NodeMaterial to MaterialNode and THREE.PhongMaterialNode It would be better too.

var uv2 = false;
var uv_offset = new THREE.NodeFloat(0);     
var uv = new THREE.NodeOperator( '+', new THREE.NodeUV( uv2 ), uv_offset);
var texture = new THREE.NodeTexture( imgTexture, uv );

nodematerial.color = t;

// onUpdate
uv_offset.number += .01;

I think reverse the order with your suggestion get better (mode,A,B) to (A,B,mode). I am in the process of creating the reflex maps, cubemap and others...

The environment and Cubemap are incomplete.

Currently the bugs it may happen more because of the format converter still unfinished. This is responsible by vector conversion. vec3 to vec4 or vec4 for example.

https://github.com/sunag/sea3d/blob/gh-pages/Labs/Three.JS-NodeMaterial/index.html#L365

A Blend "texture" for example: ( I have not tested this code )
It can be implemented in the same of a THREE.NodeOperator

https://github.com/sunag/sea3d/blob/gh-pages/Labs/Three.JS-NodeMaterial/index.html#L1105

THREE.NodeBlend = function( a, b, mode ) {

    THREE.NodeInput.call( this, 'blend' );

    this.mode = mode;
    this.a = a;
    this.b = b;

};

THREE.NodeBlend.prototype = Object.create( THREE.NodeInput.prototype );
THREE.NodeBlend.prototype.constructor = THREE.NodeBlend;

THREE.NodeBlend.prototype.generate = function( material, shader, output ) {

    var a = this.a.build( material, shader, output );
    var b = this.b.build( material, shader, output );

    switch(this.mode)
    {
        case 'multiply':

            return this.format( '(' + a + '*' + b + ')', this.a.type, output);

            break;
    }

    return a;

};

.generate() is the responsible for the code generator. The calcs codes are stored in a cache if you want to use in more than one input without losing performance.

Still I do not set up pointers or constant for optimization...

The compilation is done by propagation in build() for vertex and fragment code.

I can put you as a collaborator? If you want to edit the code in any way, I will be working on it as well.

[bhouston]

I can put you as a collaborator? If you want to edit the code in any way, I will be working on it as well.

Thanks! I'll make PRs to yours so you can approve the changes.

I've added you (as well as @mrdoob, @WestLangley and @tschw) to a side project of mine that is attempting to define a set of reusable nodes and material definitions that can be transferrable between various renders. It is mappable onto this shader graph system you've created.

[bhouston]

I do not think you have to pay attention to the repo I just gave you access to if you do not want to. It is what I am interested in implementing on top of this.

[sunag]

(2) I try to use the GeometryContext object I created in the lights refactor, it gives consistent access to a lot of the required local variables. But of course that can be resolved by the material itself.

I wish that the lights are one LightNode. My concern is to harness the code already developed for Three.JS.

(3) I've have UV just be another reference, which is resolved by the material. And I would have NodeTexture should actually take a Node Input, thus allowing for procedurally generated UVs.

You can replace UV to a vec2 would it be this?

(5) I'd try to add a NodeFunction that allows one to call arbitrary functions with parameters as an addition to your NodeOp (or maybe they could be merged in some fashion.)

this would be great. mainly for a BlendNode.

(6) I'd get rid of all of the verbose NodeNormal, NodeTransform, NormalMap, etc individual class and just have some simple constructors that create a NodeReference with a name that is resolved by the material as appropriate. NodeReference could resolve uniforms, varyings as well as computed values in the shader.

In this line of thought I think the MaterialNode could be a base of material Phong and Physical material.

(7) I do not understand the difference between NodeEnvironment and NodeCube. I think NodeEnvironment may be incomplete?

I still can not finish these Nodes.

(8) It is confusing to have NodePhong not be derived from NodeMaterial. Although I see that NodeMaterial is derived from ShaderMaterial. I wonder if you called the direct derivative from ShaderMaterial, GraphMaterial (or NodeGraphMaterial) that would make more sense -- because all together the nodes form a graph, and it is the graph that becomes the material, not an individual node.

NodeMaterial would be the root node material, it is necessary to use a node for vertex and fragment. NodePhong is hibrid and NodePhongMaterial is only a proxy class. This can then be merged.

(9) I would suggest maybe some more varied terminology. I'd call the root node, MaterialNode, and one could derive PhongMaterialNode from it. I've have Vector3Node, FloatNode, etc derived from ValueNode -- not necessarily constant, but just a value. Thus one could pipe in three FloatNodes to a Vector3Node. I think you can have a helper that would make declaring each of these one line or so rather than the 10 or so currently.

Sounds good.

(16) I'm confused about the light input -- usually one doesn't have lights being pluggable, rather the light parameters are fully defined in the light class. I guess you need this additional flexibility? How do you envision it.

This would be for the lightmap or a possible LightNode.