add equiangular EXR IBL (Image based lighting) example #13693

richardmonette · 2018-03-26T20:30:38Z

Summary

This PR adds an example of an IBL (Image based lighting) pipeline based around an equiangular formatted EXR.

Since we already have a nice PMREMGenerator, I have taken the approach of adding a EquiangularToCubeGenerator which uses the GPU to convert an equiangular format texture into a cube map. The highlight here is the equiangular EXR workflow, but I have also included in the example using a PNG equiangular map, just to demonstrate the EquiangularToCubeGenerator conversion is also possible in a low dynamic range context.

Performance

Test

console.time('EquiangularToCubeGenerator');

var cubemapGenerator = new THREE.EquiangularToCubeGenerator( texture, 512 );
var cubeMapTexture = cubemapGenerator.generate( renderer );

console.timeEnd('EquiangularToCubeGenerator');

Result

EquiangularToCubeGenerator: 28.334716796875ms

@mrdoob @WestLangley

looeee · 2018-03-26T21:11:46Z

Live version. Looks amazing btw 😍

Is there a reason why PNG seems so overexposed compared to EXR?

richardmonette · 2018-03-26T21:17:32Z

@looeee Without getting super into the details, perhaps suffice to say the PNG version is using LDR (low dynamic range) so its never going to look correct, you can mess with the exposure a bit to adjust, but the primary goal here is to establish the correct pipeline using HDR (high dynamic range) light probes as the basis of the IBL (image base lighting) pipeline.

WestLangley · 2018-03-26T22:03:51Z

examples/webgl_materials_equiangular_ibl.html

+
+				standardMaterial = new THREE.MeshStandardMaterial( {
+					map: null,
+					color: 0x000000,


Why black? I suggest white -- and reduce the default exposure, accordingly

WestLangley · 2018-03-26T22:05:19Z

examples/webgl_materials_equiangular_ibl.html

+				scene.add( torusMesh1 );
+				objects.push( torusMesh1 );
+
+				floorMaterial = new THREE.MeshStandardMaterial( {


I think MeshBasicMaterial is more appropriate, here, as the purpose is only to display the mips.

WestLangley · 2018-03-26T22:10:20Z

Sweet!

For a future PR, if you have any ideas on how to reduce the memory stats, that would be great.

memory:
    geometries: 318
    textures: 18

richardmonette · 2018-03-26T23:34:58Z

@WestLangley Thank you for your comments! I've made the changes you recommended and I think it even slightly improves the visual quality. Good call with the MeshBasicMaterial, its unlit rendering is exactly what we needed for the debug texture.

WestLangley · 2018-03-27T01:49:06Z

@looeee I took the liberty of editing your temporary live link as it was not accommodating the additional commits. :)

mrdoob · 2018-03-27T04:37:17Z

@WestLangley Thanks! The new link looks much better indeed!

Mugen87 · 2018-03-27T09:29:50Z

examples/js/loaders/EquiangularToCubeGenerator.js

+	];
+
+	this.camera = new THREE.PerspectiveCamera( 90, 1, 0.1, 10 );
+	this.planeMesh = new THREE.Mesh( new THREE.BoxGeometry( 1, 1, 1 ), this.getShader() );


I would choose BoxBufferGeometry here.

Mugen87 · 2018-03-27T09:30:21Z

examples/webgl_materials_equiangular_ibl.html

+					envMapIntensity: 1.0
+				} );
+
+				var geometry = new THREE.TorusKnotGeometry( 18, 8, 150, 20 );


And here: TorusKnotBufferGeometry

Mugen87 · 2018-03-27T09:41:32Z

I have a black screen and the following warnings in the console.

[.Offscreen-For-WebGL-0x55fb20407c10]GL ERROR :GL_INVALID_FRAMEBUFFER_OPERATION : glClear: framebuffer incomplete (check)
[.Offscreen-For-WebGL-0x55fb20407c10]GL ERROR :GL_INVALID_FRAMEBUFFER_OPERATION : glDrawArrays: framebuffer incomplete (check)

Tested with Chromium 64.0.3282.167 on Ubuntu 16.04. FF 59.0.1 shows a black screen, too (with similar warnings).

Other HDR examples (like https://threejs.org/examples/webgl_materials_envmaps_hdr.html) seem to work on my machine:

richardmonette · 2018-03-27T12:14:09Z

@Mugen87 Thank you for your review. I have made the following changes, based upon your suggestions:

Changed both of the suggested geometries to buffers
I have fixed the issue with FF 59.0.1 (referring to https://developer.mozilla.org/en-US/docs/Web/API/WEBGL_color_buffer_float I observed that the render target appears to only support RGBA32F_EXT, and RGB32F_EXT is deprecated, so I have adjusted accordingly)

Would it be possible for you to please confirm that's working for you too now? Thanks!

Mugen87 · 2018-03-27T12:20:20Z

Great 👍 . The example now works in Chrome and FF on my machine.

WestLangley · 2018-03-27T16:46:11Z

examples/js/loaders/EquiangularToCubeGenerator.js

+    			\n\
+					gl_FragColor = vec4( color, 1.0 );\n\
+				}",
+			blending: THREE.CustomBlending,


Can you please explain the reason for the custom blending?

Hey @WestLangley , I'm not specifically familiar with the THREE.ShaderMaterial so I just took that from an example. If its an option, this could probably just be left as the defaults.

WestLangley · 2018-03-27T16:51:18Z

examples/js/loaders/EquiangularToCubeGenerator.js

+
+	this.camera = new THREE.PerspectiveCamera( 90, 1, 0.1, 10 );
+	this.planeMesh = new THREE.Mesh( new THREE.BoxBufferGeometry( 1, 1, 1 ), this.getShader() );
+	this.planeMesh.material.side = THREE.DoubleSide;


Why is it planeMesh and why is it double-sided?

planeMesh should be renamed to boxMesh, it's double sided because the camera is inside the box, i.e. it needs to render and not be backface culled.

side: THREE.BackSide;

As you know, users copy code, so let's make it clear. :)

👍(btw, appreciate you taking the time to work through the code with me!)

WestLangley · 2018-03-27T16:52:53Z

examples/js/loaders/EquiangularToCubeGenerator.js

+	this.planeMesh.material.side = THREE.DoubleSide;
+	this.scene = new THREE.Scene();
+	this.scene.add( this.planeMesh );
+	this.scene.add( this.camera );


I don't think adding the camera to the scene is required.

WestLangley · 2018-03-27T16:54:09Z

examples/js/loaders/EquiangularToCubeGenerator.js

+
+			this.camera.position.set(0, 0, 0);
+			this.camera.up.set(v.u[0], v.u[1], v.u[2]);
+			this.camera.lookAt(new THREE.Vector3(v.t[0], v.t[1], v.t[2]));


this.camera.lookAt( v.t[ 0 ], v.t[ 1 ], v.t[ 2 ] );

WestLangley · 2018-03-27T17:05:06Z

How are you handing the encoding? Are you assuming .exr is in linear space? .png in sRGB?

richardmonette · 2018-03-27T17:11:35Z

EXR is in linear, yes.

WestLangley · 2018-03-27T17:37:18Z

Is there a reason why PNG seems so overexposed compared to EXR?

Likely the encoding.

Can you explicitly set the proper encodings in your example? THREE.sRGBEncoding for .png, THREE.LinearEncoding for .exr.

mrdoob · 2018-03-27T18:23:47Z

Unrelated to this PR, but I think there is something wrong with the pmrem code. By changing the roughness of the material we can see that the levels are not aligned (the lights in the reflection changes position).

/cc @bhouston

richardmonette · 2018-03-27T19:00:37Z

Unrelated to this PR, but I think there is something wrong with the pmrem code. By changing the roughness of the material we can see that the levels are not aligned (the lights in the reflection changes position).

https://github.com/mrdoob/three.js/blob/dev/examples/js/pmrem/PMREMGenerator.js#L103

That's something I am looking at in my next PR... 😄

mrdoob · 2018-03-27T20:26:44Z

That's something I am looking at in my next PR... 😄

Sweet!

I think this PR only needs @WestLangley last comment addressed and it's ready to go.

richardmonette · 2018-03-27T22:10:24Z

@WestLangley I've explicitly set the proper encodings as you suggested and things have come even further into line 👍

Left: Low dynamic range / PNG, Right: High dynamic range / EXR

You'll notice when adjusting exposure (especially where the EXR has more detail in the shadows) that the EXR / high dynamic range version works better, but there isn't the strange visual difference anymore 👌

WestLangley · 2018-03-27T22:39:39Z

Yes, the rendering is looking more reasonable now. :)

What I don't yet understand is why generateMipmaps is false for .exr and true for .png -- and what it should be. That can wait for another PR, though.

/ping @bhouston

bhouston · 2018-03-28T01:36:43Z

What I don't yet understand is why generateMipmaps is false for .exr and true for .png -- and what it should be. That can wait for another PR, though.

Actually it isn't because of EXR or PNG but rather RGBE can not be linearly interpolated as it isn't a monotonic encoding, where as RGBM is a relatively monotonic encoding.

bhouston · 2018-03-28T01:39:47Z

Unrelated to this PR, but I think there is something wrong with the pmrem code. By changing the roughness of the material we can see that the levels are not aligned (the lights in the reflection changes position).

This is caused by the sample pattern not being fully symmetric.

bhouston · 2018-03-28T01:41:22Z

If the PNG in this example is truly LDR and there is no encoding (e.g. RGBE, RGBM, RGBD, etc.) then it can be linear interp as well like the EXR HDR. There is no need to work around non-monotonic RGBA8 encodings for RGB HDR values.

richardmonette · 2018-03-28T01:47:01Z

This is caused by the sample pattern not being fully symmetric.

I've got some ideas about using a low discrepancy sampler, such as a Hammersley, to improve quality as well, for a follow up PR.

bhouston · 2018-03-28T02:11:24Z

I've got some ideas about using a low discrepancy sampler, such as a Hammersley, to improve quality as well, for a follow up PR.

Okay. I'm interested. I think that the issue is the current reliance on a GPU-based random number generator which is not uniform, nor is it consistent across different GPU families. I once had hope that there could be a reliable, high quality GPU-based "random" number generator -- test cases here https://threejs.org/examples/#webgl_postprocessing_procedural -- but I have since given up hope. I think a Hammersley distribution or a 2D Poisson disk would work great here.

You also hint that you can improve the theoretical foundation of this convolver, that would be really amazing. The problem with this one is that it uses the result of the previous level to create the next to get speed and reduce the number of samples required, thus it is a little tricky.

bhouston · 2018-03-28T02:14:38Z

I would rename this example to webgl_materials_envmaps_exr.html so that it is beside the existing weblg_materials_envmaps_hdr.html and weblg_materials_envmaps.html as they are all related.

…l_materials_envmaps_exr.html

mrdoob · 2018-03-29T01:52:54Z

Thanks!

bhouston · 2018-03-29T02:05:46Z

This was a great PR and work @richardmonette, very impressive.

add equiangular exr IBL example

dde7310

WestLangley reviewed Mar 26, 2018

View reviewed changes

adjust materials

8b57a8d

mrdoob added this to the r92 milestone Mar 27, 2018

Mugen87 requested changes Mar 27, 2018

View reviewed changes

switch to bufferGeo, use RGBAF (better supported)

f810577

Mugen87 approved these changes Mar 27, 2018

View reviewed changes

WestLangley reviewed Mar 27, 2018

View reviewed changes

richardmonette added 2 commits March 27, 2018 12:54

rename boxMesh, remove custom blending

5b003ff

dont need camera in scene, can avoid a new

ded66b2

be specific about using BackSide

fcc238f

add explicit encoding

65931b0

rename examples/webgl_materials_equiangular_ibl.html -> examples/webg…

05af930

…l_materials_envmaps_exr.html

mrdoob merged commit 1ec3ee2 into mrdoob:dev Mar 29, 2018

vorg mentioned this pull request Mar 29, 2018

[three] Use HDR Equirect Panorama vorg/pbr-compare#3

Open

bhouston mentioned this pull request Jul 13, 2018

Incorrectly weighted samples in PMREMGenerator #14469

Closed

13 tasks

WestLangley mentioned this pull request Jul 28, 2018

Added CubeTexture().fromEquirectangular() #9746

Closed

Uh oh!

add equiangular EXR IBL (Image based lighting) example #13693

add equiangular EXR IBL (Image based lighting) example #13693

Uh oh!

Conversation

richardmonette commented Mar 26, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Performance

Uh oh!

looeee commented Mar 26, 2018 • edited by WestLangley Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

richardmonette commented Mar 26, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

WestLangley commented Mar 26, 2018

Uh oh!

richardmonette commented Mar 26, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

WestLangley commented Mar 27, 2018

Uh oh!

mrdoob commented Mar 27, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Mugen87 commented Mar 27, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

richardmonette commented Mar 27, 2018

Uh oh!

Mugen87 commented Mar 27, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

WestLangley commented Mar 27, 2018

Uh oh!

richardmonette commented Mar 27, 2018

Uh oh!

WestLangley commented Mar 27, 2018

Uh oh!

mrdoob commented Mar 27, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

richardmonette commented Mar 27, 2018

Uh oh!

mrdoob commented Mar 27, 2018

Uh oh!

richardmonette commented Mar 27, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

WestLangley commented Mar 27, 2018

Uh oh!

bhouston commented Mar 28, 2018

Uh oh!

bhouston commented Mar 28, 2018

Uh oh!

bhouston commented Mar 28, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

richardmonette commented Mar 26, 2018 •

edited

Loading

looeee commented Mar 26, 2018 •

edited by WestLangley

Loading

richardmonette commented Mar 26, 2018 •

edited

Loading

Mugen87 commented Mar 27, 2018 •

edited

Loading

Mugen87 commented Mar 27, 2018 •

edited

Loading

mrdoob commented Mar 27, 2018 •

edited

Loading

richardmonette commented Mar 27, 2018 •

edited

Loading

bhouston commented Mar 28, 2018 •

edited

Loading

richardmonette commented Mar 28, 2018 •

edited

Loading