Add a 32 bpc frame buffer option to Viewport

[Lauson1ex]

Describe the project you are working on

Third-person, high-fidelity visuals 3d game.

Describe the problem or limitation you are having in your project

Godot doesn't have, and won't have for a long time, if ever, MRTs (Multiple Render Targets). Therefore, enable us to use the second best thing: 32-bit per component frame buffers.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

For starters, it is more efficient to use a single, 32-bit per component frame buffer than using two MRTs. With clever use of custom shaders, it is possible to pack and store more information than just colors inside buffers. 32-bit per component frame buffers are actually 128-bit data buffers. With them, it is be possible to implement advanced features in Godot, such as Motion Vectors for exemple, and thus implement advanced post process effects like TAA and #2933 without having to completely overhaul the underlying rendering backend, and distribute those as user-created add-ons.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Currently, Godot 3.x only supports up to 16-bit per component frame buffers (GL_RGBA16F, when the option use_hdr in a Viewport is enabled). This proposal aims to add optional 32-bit per component frame buffers (VK_FORMAT_R32G32B32A32_SFLOAT for Vulkan, GL_RGBA32F for GLES3) in order to circumvent the lack of MRTs (since enabling this rendermode is far less troublesome than fully implementing MRTs). Due to the nature of what I just said, this is a feature intended for advanced user only, and therefore must be an option in the project settings that must be opt-ed in. Then, the option to use 32-bit per component frame buffers will be available to be enabled on Viewports, just like the use_hdr option.

Using reinterpret casts such as floatBitsToUint/uintBitsToFloat and bit manipulation in shaders, it is possible to pack information during the regular geometry rendering in a single pass. For example, Godot 4's DOF post process shader uses the alpha component to store the blur radius. With clever packing and bit manipulation, it is possible to store albedo, depth, normals and lights in a single 128-bit buffer, essentially creating a tiny deferred buffer in a forward renderer. Or, storing colors and motion vectors as the geometry gets rendered. The applications are quite endless.

For reference, I experimented packing colors and depth in a single buffer in Godot 3.x, by packing depth as a 24-bit value and 8-bit per component RGB (by transforming linear RGB to sRGB first), and then reading everything back with a single texture sample, unpacking, and finally enjoying depth in canvas_item shaders. It works well, but then I lose HDR. My project needs HDR, so this isn't usable. Higher bit-depth frame buffers must be available for that. 16-bit per component (48-bit in total) is enough for color with HDR. 16-bit per component (32-bit in total) are enough for motion vectors (since usual implementations use GL_RG16F). As you can see, you can pack a lot of data in these buffers. Then, it is the user's responsibility to implement whatever they may need, without having to change the underlying rendering backend. Literature on how to efficiently perform GPU data packing is plenty on the Internet.

A new shader render_mode will probably be needed for spatial shaders: disable_alpha, to be able to write to the alpha component of the buffer without making the geometry go to the transparent pipeline, and therefore enjoy the full 128-bit buffer, but this is a very low-priority requirement; having 96-bits to work with is a good start.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No.

Is there a reason why this should be core and not an add-on in the asset library?

This is core.

[Calinou]

Due to the nature of what I just said, this is a feature intended for advanced user only, and therefore must be an option in the project settings that must be opt-ed in. Then, the option to use 32-bit per component frame buffers will be available to be enabled on Viewports, just like the use_hdr option.

I wouldn't lock the ability to use a property behind a project setting. As long as it's not the default, I think it's fine. The only thing we need to be careful about is to emit a node configuration warning and print a warning message if an incompatible mode is used in GLES2.

[Lauson1ex]

I wouldn't lock the ability to use a property behind a project setting. As long as it's not the default, I think it's fine. The only thing we need to be careful about is to emit a node configuration warning and print a warning message if an incompatible mode is used in GLES2.

The justification for suggesting this feature to be locked behind a project settings is because I fear misuse. A user who is not careful may think: "32 is better than 16, right? So my game will automatically look better if I use this mode", when in reality they'll be using twice the video memory requirement for that Viewport. Plus the fact that frame buffers with 32-bit components use precision highp by default, which can impact performance compared to mediump.

Godot has settings that are locked behind project settings or outright hidden unless you read the API and enable them via scripting, so I thought it would only be appropriate, but I could be wrong.

[Calinou]

A user who is not careful may think: "32 is better than 16, right? So my game will automatically look better if I use this mode", when in reality they'll be using twice the video memory requirement for that Viewport. Plus the fact that frame buffers with 32-bit components use precision highp by default, which can impact performance compared to mediump.

In master, we can replace the hdr boolean property with a depth_per_component enum property with the following values: 8-bit (LDR), 16-bit (HDR), 32-bit (Advanced Shaders Only).

In 3.x, we have to keep the hdr boolean for compatibility with existing projects. I guess we have to add a second boolean property like depth_per_component_32. This property would be ignored if hdr is disabled (with a node configuration warning displayed).

[Lauson1ex]

In master, we can replace the hdr boolean property with a depth_per_component enum property with the following values: 8-bit (LDR), 16-bit (HDR), 32-bit (Advanced Shaders Only).

In 3.x, we have to keep the hdr boolean for compatibility with existing projects, so I guess we have to add a second boolean property like depth_per_component_32. This property would be ignored if hdr is disabled (with a node configuration warning displayed).

This is actually more usable than what I suggested. 🙂

[Calinou]

@Lauson1ex I've implemented this for both the master and 3.x branches, could you give it a try to confirm it works as intended in both versions? See the references above for the pull request links. Thanks in advance 🙂

[Lauson1ex]

@Lauson1ex I've implemented this for both the master and 3.x branches, could you give it a try to confirm it works as intended in both versions? See the references above for the pull request links. Thanks in advance 🙂

@Calinou Absolutely. Is there a precompiled version where I can test this on real quick? Or should I fork, merge and compile it myself?

[Calinou]

Absolutely. Is there a precompiled version where I can test this on real quick? Or should I fork, merge and compile it myself?

You may be able to find precompiled builds under the pull requests' respective Checks tab. See Testing pull requests in the documentation for more information.

[Lauson1ex]

You may be able to find precompiled builds under the pull requests' respective Checks tab. See Testing pull request in the documentation for more information.

Gotcha. Thank you.