PMREMGenerator: FromScene behavior differs with renderer clear color vs scene.background

[gkjohnson]

Describe the bug

The resulting environment map from PMREMGenerator.fromScene differs when calling renderer.setClearColor( 0xffffff ) beforehand vs calling scene.background = new Color( 0xffffff );.

To Reproduce

You can see the difference in behavior in this fiddle:

https://jsfiddle.net/0saxwg6n/

Uncomment this line to see the behavior difference:

// NOTE: Uncomment this line to make it work
// scene.background = new THREE.Color( 0xffffff );

Code

Setting clear color:

renderer.setClearColor( 0xffffff );
const generator = new THREE.PMREMGenerator( renderer );
const rt = generator.fromScene( scene );
scene.environment = rt.texture;

Setting scene background:

scene.background = new THREE.Color( 0xffffff );
const generator = new THREE.PMREMGenerator( renderer );
const rt = generator.fromScene( scene );
scene.environment = rt.texture;

Expected behavior

The look of the resulting map is the same regardless of whether or not the background color is set via scene.background or renderer clear color.

Screenshots

renderer.setClearColor( 0xffffff );

scene.background = new Color( 0xffffff );

(an aside but is it expected that the background become so much darker when rendering to the environment map?)

Platform:

• Device: Desktop
• OS: Windows 10
• Browser: Chrome
• Three.js version: dev

[mrdoob]

(an aside but is it expected that the background become so much darker when rendering to the environment map?)

Hmm, should be fairly fixable actually... We should just take renderer.outputEncoding into account when rendering the background color?

[gkjohnson]

We should just take renderer.outputEncoding into account when rendering the background color?

Is it so simple? I'm unfamiliar with exactly how all of this should work when it comes to generating (HDR?) environment maps but 0xffffff in srgb is 0xffffff in linear and vice versa so I wouldn't expect color space conversion to affect the resulting color in the map.

It looks to me like this is probably the section of code responsible for darkening the color (and is not being applied to renderer clear color when scene.background is not defined):

three.js/src/extras/PMREMGenerator.js

Lines 269 to 276 in 9694de0

	background.convertSRGBToLinear();
	// Convert linear to RGBE
	const maxComponent = Math.max( background.r, background.g, background.b );
	const fExp = Math.min( Math.max( Math.ceil( Math.log2( maxComponent ) ), - 128.0 ), 127.0 );
	background = background.multiplyScalar( Math.pow( 2.0, - fExp ) );
	const alpha = ( fExp + 128.0 ) / 255.0;
	renderer.setClearColor( background, alpha );
	scene.background = null;

[WestLangley]

The renderer clear color and scene background color do not respond to tone mapping or encoding settings; textured backgrounds do. That is by design. Related: #15382.

[gkjohnson]

The renderer clear color and scene background color do not respond to tone mapping or encoding settings

I thought I'd recalled that. So do we know why a background of pure white results in such a medium gray color? Is that intended?

And to the original point of the issue -- is it intended that setting the clear color to white and setting the scene background to white results in a different color environment map? Or just an oversight that the clear color is not converted to RGBE the same way that scene.background is?

Perhaps @elalish knows?

[elalish]

So the trick with HDR is that it is on a floating-point scale, which goes vastly beyond the [0, 1] range of sRGB. Tone mapping is required to map back to [0, 1]. Think of 1 as a white piece of paper, and 10,000 as a light bulb. When you're looking at both of these, it would be odd to map them each to an sRGB of 1, right? In fact the paper will come out looking kind of a medium gray by comparison, depending on the light incident on it. Anything in the [0, 1] range will have very little effect on a PMREM, as the light is so dim. You really need to make use of the floating-point range if you want to define sources of light.

[gkjohnson]

@elalish I do understand that HDR represents lighting data far outside the the typical 0-1 color range but what I'm trying to get at is there's different behavior when rendering PMREM using renderer.clearColor vs scene.background. From the first post you can see the difference when setting the scene background to white vs just setting the clear color to white:

renderer.setClearColor( 0xffffff );

scene.background = new Color( 0xffffff );

As a user it's confusing to me that these are different and I believe it's an oversight. The difference stems from this section of the code that only converts the background color to an exponential representation but does not if it's using the renderer.clearColor. The behavior in these two cases should be the same, right? I'm happy to make the change I just want to confirm.

Behavioral difference aside I question whether or not it makes sense to map the background colors to an exponential representation considering they don't respond to lighting. It's odd to set the background to white and have it render as dull gray. If I want the background color to be "lightbulb" intensity should I set background color to values > 1.0? The docs state the range used for Color should be between [ 0.0, 1.0 ].

[elalish]

I agree it's a bit confusing, but I'm not sure what would be less confusing. Anything rendered into a PMREM should be HDR, which means colors restricted to the [0, 1] space are the confusing thing to me. To generate area lights, I use boxes with large emissive intensities to accomplish this. If you want something like a hemisphere light by setting a constant background color, I would tend to expect it to be >1, though I agree this sounds like a hack.

I'm actually more surprised that setting the clear color to white gives a white background, since PMREM is calculated in RGBE space. I don't know what #ffffff is in RGBE, but I don't think it's what you're expecting.

[gkjohnson]

I agree it's a bit confusing, but I'm not sure what would be less confusing.

What was getting me was that this section of code is specifically designed to be the inverse of the RGBEToLinear shader function which means that setting a white background should get converted to RGBE when writing to the cube map and get converted back to white when sampling but instead we're getting the tinted gray color.

After some testing this seems to be related to the premultipliedAlpha setting on the renderer and the fact that the alpha component is being overloaded to represent the exponent in this case. Setting it to false in the example above yields more expected results:

PremultipliedAlpa : True

PremultipliedAlpha : False

This StackOverflow answer mentions that there can be quirks when the premultiplied alpha color used to clear is invalid (white to RGBE is 1.0, 1.0, 1.0, 0.5...) and ultimately the result shouldn't be dependent on the premultiplied alpha setting. I think this can be fixed and made resilient to the setting by clearing the background with a flat color mesh with opacity rather than using the renderer clear color but I have to do more testing.

'm actually more surprised that setting the clear color to white gives a white background, since PMREM is calculated in RGBE space. I don't know what #ffffff is in RGBE, but I don't think it's what you're expecting.

Heh I think it's just an extremely blown out white, which is why we're not seeing the red tinting from the plane in the example image above.

[whatisor]

(an aside but is it expected that the background become so much darker when rendering to the environment map?)

Hmm, should be fairly fixable actually... We should just take renderer.outputEncoding into account when rendering the background color?

I wonder why we didn't follow this, but need to keep current? What is advantage(or difficulty) of it?