EXRLoader - non-image parts

[photopea]

I am not able to parse the following EXR file with EXRLoader. It has the bit 9 of flags set - “long names”, and the bit 10 - "nonimage parts". The compression is ZIP and is already supported by EXRLoader.

Could you implement the missing features?

https://drive.google.com/file/d/1fuQXEjaiLu9tUibvVdQKwlr0W289SzKO/view?usp=sharing

[sciecode]

Indeed the file used relies on deep data and tile-based internal representation, both of which are not currently supported. Those have been on my to-do list, however I haven't had enough time to tackle the problem so far. My plan is to work on the following features (in order):

• Tile-based data
• Deep data
• Multi-part file

It might take a while to actually implement these, so I apologize in advance. But I'll do my best to do it sooner rather than later.

[Mugen87]

I've recently wanted to load the following EXR file: test.exr.zip

But the loader reports this error: Something is wrong with PIZ_COMPRESSION BITMAP_SIZE.

Does the loader throw this message because one of the above features is missing? The file itself can be loaded into GIMP and macOS Preview.

[sciecode]

Hey Michael,

I was testing the file in question and it does not appear the issue is caused by the lack of features mentioned above. It seems to be related with some modifications made to the loader to handle a very weird case where firefox can only support RGBA half-float textures (#16778)

Now the real reason this happens is because we completely ignore number of channels on the input file and try to force the loader to result in RGBAFormat texture. In the case of the file you provided, the input file only contains a single channel, which completely breaks the entire logic. I'm surprised this wasn't noticed before to be honest.

Ideally we would need to check if the weird firefox behavior still exists, and if it doesn't, we simply handle the input data correctly. Othewise, I think we need to properly check the cases where only RGB formats need to be expanded to RGBA, and other cases can be decoded normally.

[Mugen87]

In general, it's recommended to avoid RGB textures since they might be emulated in software on certain platforms. So I think it's a good approach to always convert RGB to RGBA in the loaders.

I would treat all other cases without such a conversion. For example if an EXR file only defines a single color channel, the resulting texture could use THREE.RedFormat (resulting in gl.R8, gl.R16F or gl.R32F depending on the data type).

[sciecode]

My thoughts exactly, however, there is one problem.

For example if an EXR file only defines a single color channel, the resulting texture could use THREE.RedFormat (resulting in gl.R8, gl.R16F or gl.R32F depending on the data type).

These internalFormats are only available in WebGL2 contexts, for WebGL1 we are limited to using Luminance and Alpha internal formats, which only support UnsignedByteType. So we do have this other limitation to be mindful of, not entirely sure what the current policy for handling these sort of GL1/GL2 support issues.

[Mugen87]

Indeed, that's important to keep in mind! Considering the maturity of WebGL 2 I would say it's okay if certain loaders produce texture formats that can only be used with a WebGL 2 rendering context.

@mrdoob What do you think about this?

[sciecode]

In midst of working on a solution for the problem, I noticed there are more complications. Seems like simply using WebGL2 internal formats might not be enough. One of the side-effects of using THREE.RedFormat is that it actually only applies the input data to the red channel of the texture. so we don't actually get the original luminance behavior from the underlying texture. According to Khronos Image Format docs:

GL_INTENSITY is not available in any WebGL contexts and GL_LUMINANCE only accepts UnsignedByteType, with the extra incovenience of not being color-renderable ( although, this might not be a direct issue ).

All in all, the only real "solution" is to also treat single-channel images specially. We need to ignore the outputType specified by the user and always convert the image to LuminanceFormat - UInt8. Which is kind of a bummer, because it completely defeats the purpose of using an .EXR 16/32b luminance map in the first place.

Using the RedFormat texture is still valuable, because we make use of all the precision, but it pretty much requires a custom sampling shader to be used correctly.

[Mugen87]

Which is kind of a bummer, because it completely defeats the purpose of using an .EXR 16/32b luminance map in the first place.

That's the reason why I think we should use THREE.RedFormat. Besides, devs can patch the shader sampling via onBeforeCompile() if necessary. And the new node material opens up even more possibilities in that regard.

[sciecode]

That's the reason why I think we should use THREE.RedFormat. Besides, devs can patch the shader sampling via onBeforeCompile() if necessary. And the new node material opens up even more possibilities in that regard.

Sounds good to me. Working on a PR while we wait for other inputs.

[mrdoob]

I don't have an opinion on this one. Whatever you guys think is best 👍