Significant Performance Drop and High CPU Usage with BatchedMesh

[lanvada]

Description

Hello,

I exported a building model from Revit in glTF format and merged meshes with the same materials to manage their visibility in Three.js using the BatchedMesh class. However, I've encountered a significant performance issue when rendering these merged meshes with BatchedMesh compared to using Mesh.

Performance Comparison:

• BatchedMesh Rendering:
  • CPU Usage: ~40%
  • GPU Usage: ~30%
  • Frame Rate: ~20 FPS
• Mesh Rendering:
  • CPU Usage: ~15%
  • GPU Usage: ~90%
  • Frame Rate: ~60 FPS

This drastic difference in performance is concerning, especially the high CPU load and low frame rate when using BatchedMesh. I've already set .perObjectFrustumCulled and .sortObjects to false in BatchedMesh, which, if set to true, leads to an even more severe frame rate drop.

Additionally, I'm using three-csm and postprocessing frameworks alongside Three.js.

System Configuration:

• CPU: Intel i7-10700
• GPU: NVIDIA RTX 2080 Super

Could someone help me understand why BatchedMesh increases the CPU overhead so significantly and suggest any possible optimizations or solutions to improve the frame rate?

Thank you!

Reproduction steps

Using BatchedMesh to render more than 10 million triangles and vertices, there are about 100,000 different geometries.

Code

Code in the project
batched-mesh-performance-test

Live example

Code in the project
batched-mesh-performance-test

Screenshots

No response

Version

r165

Device

No response

Browser

No response

OS

No response

[lanvada]

In the past few days, I’ve tried to find a solution, but without success. I’ve uploaded the relevant code and models to GitHub. The models consist of 12 million triangles and 16 million vertices. Is such a high CPU performance cost necessary for BatchedMesh? I don’t think it should be. When rendering Batched3DModel in Cesium, I didn’t encounter such issues. I believe the mesh batching in Cesium and BatchedMesh should be quite similar, right?

Additionally, I’d like to mention that after the update to version 166, the performance consumption of BatchedMesh has worsened, and the frame rate has dropped further in the same scene.

Here is the link to the code and models: batched-mesh-performance-test

[gkjohnson]

For the sake of easily understanding the issue please provide a live example that doesn't require pulling and running a separate Github project. You can host a demo page with Github pages, for example. Recordings of the Chrome performance monitor would be helpful, as well.

[lanvada]

Here is the demo link: https://batched-mesh-performance-test.vercel.app

The model is compressed using Draco and is approximately 44MB in size, with a total of 7.6 million triangles and 9.6 million vertices. It takes about 10 seconds to load the model. Initially, the page does not use BatchedMesh, and the frame rate on my computer is 60 FPS. You can switch to BatchedMesh by clicking the button on the bottom left, after which the frame rate drops to about 17 FPS.

[lanvada]

I need to provide some additional details. When exporting the glTF model from Revit, I grouped meshes with the same materials. I added three extensions: EXT_instance_features, EXT_mesh_features, and EXT_mesh_gpu_instancing. I also assigned a _FEATURE_ID_0 attribute to each vertex to differentiate between different batches, and this attribute is parsed during loading. The related code can be found in two TypeScript files in the project I previously provided: MeshFeatures.ts and GltfToolkit.ts. If you need to load the model for debugging, you might need to use the relevant code to parse the different batches.
Since I have already used "_FEATURE_ID_0" to differentiate vertices of different batches, creating BatchedMesh could potentially be implemented by directly assigning values to internal properties (perhaps by renaming "_FEATURE_ID_0" to "_batchId"). However, I have not studied the BatchedMesh code in detail and have only used the BatchedMesh API to add geometries in a straightforward manner. This approach involves iterating over vertices and face indices and results in considerable additional memory allocation and copying overhead, making it inefficient.

[gkjohnson]

Thanks for producing a live link. I think this demo is too complicated to dig into, though. There are over 800 individual meshes and a mix of batched and instanced meshes as well as a lot of custom GLTF user code that make it difficult to understand what's going on. It think it would best if we had an example that used a single batched mesh compared to a merged mesh to show any performance differences. Ideally without any external geometry file dependencies.

[lanvada]

Thanks for producing a live link. I think this demo is too complicated to dig into, though. There are over 800 individual meshes and a mix of batched and instanced meshes as well as a lot of custom GLTF user code that make it difficult to understand what's going on. It think it would best if we had an example that used a single batched mesh compared to a merged mesh to show any performance differences. Ideally without any external geometry file dependencies.

Replicating this issue with a single BatchedMesh is actually quite "simple." You just need to increase the MAX_GEOMETRY_COUNT in the webgl_mesh_batch.html example to ten times its previous value. On my computer, when the geometryCount is 20,000, the CPU usage is around 20%. When the geometryCount is increased to 200,000, CPU usage rises to between 50% and 60%, yet GPU usage remains unchanged.

[lanvada]

Thanks for producing a live link. I think this demo is too complicated to dig into, though. There are over 800 individual meshes and a mix of batched and instanced meshes as well as a lot of custom GLTF user code that make it difficult to understand what's going on. It think it would best if we had an example that used a single batched mesh compared to a merged mesh to show any performance differences. Ideally without any external geometry file dependencies.

Replicating this issue with a single BatchedMesh is actually quite "simple." You just need to increase the MAX_GEOMETRY_COUNT in the webgl_mesh_batch.html example to ten times its previous value. On my computer, when the geometryCount is 20,000, the CPU usage is around 20%. When the geometryCount is increased to 200,000, CPU usage rises to between 50% and 60%, yet GPU usage remains unchanged.

Turning off the sortObjects, perObjectFrustumCulled, and useCustomSort options can reduce CPU usage by about 5%.

Additionally, I've noticed that enabling only the sortObjects option decreases the frame rate from 30 to 9. Is this a normal phenomenon?

[gkjohnson]

Replicating this issue with a single BatchedMesh is actually quite "simple."

I understand but I'm asking for a minimal reproduction case to be provided. I think it's a more than reasonable ask for a simple demonstration case separate from user code to made when reporting an issue and asking maintainers to spend time investigating. I can take a closer look once a this minimal repro is available.

Additionally, I've noticed that enabling only the sortObjects option decreases the frame rate from 30 to 9. Is this a normal phenomenon?

It depends on how many objects there are and where the bottleneck is. Frustum culling and sorting share a lot of the same logic, though, enabling one or the other will have a larger apparent impact then if one is already enabled and you enable the other. If you provide a simple reproduction case it will be easier to understand what you're describing.

[lanvada]

I can take a closer look once a this minimal repro is available.

It depends on how many objects there are and where the bottleneck is. Frustum culling and sorting share a lot of the same logic, though, enabling one or the other will have a larger apparent impact then if one is already enabled and you enable the other. If you provide a simple reproduction case it will be easier to understand what you're describing.

Ah, the case I mentioned above is actually based on the examples/webgl_mesh_batch.html. All I did was change the MAX_GEOMETRY_COUNT to 200,000 directly in the HTML, and then set it to this number in the browser. Give me a moment to fork this project and make the change, then I'll deploy it on Vercel. Alternatively, if it's convenient for you, you could just tweak the batch count limit in this example to replicate the issues I've mentioned.

[lanvada]

Sorry about this—I'm not very good at English, so I often rely on ChatGPT to help me write. If there are any impolite words or phrases, please forgive me...

[gkjohnson]

Ah, the case I mentioned above is actually based on the examples/webgl_mesh_batch.html. All I did was change the MAX_GEOMETRY_COUNT to 200,000 directly in the HTML, and then set it to this number in the browser. Give me a moment to fork this project and make the change, then I'll deploy it on Vercel.

If the sort behavior is separate from the original performance question then I'd prefer to focus on one thing at a time. You can ask at the forum if you'd like help understanding the performance implications of sorting objects.

Please provide a simple example in something like jsfiddle that shows the performance differences you're observing in #28776 (comment) without using any custom 3d model or complex feature processing logic.

[lanvada]

I've set up a page where you can switch between "BatchedMesh" and "MergedMesh". Here's the link: https://batched-mesh-performance-example.vercel.app/. Switching to "MergedMesh" might take about ten seconds or so.

What I've noticed is that when using "BatchedMesh", the CPU usage significantly increases—from 15% to 40% on my computer.

I did a quick debug with Spector.js and found that enabling the sortObjects option causes the texSubImage2D function to take up too much time, leading to a drop in frame rate. However, when I turn off sortObjects, only the multiDrawElementsWEBGL function remains. I'm wondering if the increase in CPU usage is a necessary cost of using multiDrawElementsWEBGL.

Also, another issue is when there are many materials in the scene (multiple BatchedMeshes or MergedMeshes), using the "MergedMesh" method allows the GPU to perform at its best, nearing 100% utilization. But with the "BatchedMesh" method, the GPU utilization seems to be about the same as when there's only a single material—around 30%.

I'm not sure if the above situations can be optimized, or is this just the nature of the WebGL API?

[Shakhriddin]

Replicating this issue with a single BatchedMesh is actually quite "simple." You just need to increase the MAX_GEOMETRY_COUNT in the webgl_mesh_batch.html example to ten times its previous value. On my computer, when the geometryCount is 20,000, the CPU usage is around 20%. When the geometryCount is increased to 200,000, CPU usage rises to between 50% and 60%, yet GPU usage remains unchanged.

If you set geometryCount to 200000, then the freezes are due to the update _indirectTexture. And of course, this is due to the fact that there are a lot of instances and do loop through them all takes a lot of time. You can see from this screenshot

@gkjohnson, @lanvada

[gkjohnson]

I've made a simpler example that just uses javascript and cubes to understand things a bit better. This demo allows for changes between a merged geometry, batched mesh, and instanced mesh by changing the "MODE" flag at the top. It also removes any extra texture sampling logic used in BatchedMesh to remove that as a possible performance bottleneck:

jsfiddle link

I'm seeing that between the three options, BatchedMesh is the only one that suffers from this performance degradation. Instances and merged geometry both work fine otherwise. InstancedMesh and the merged geometry run at 120 fps while the BatchedMesh runs at ~30 fps on my 2021 M1 Pro Macbook.

In terms of why this is happening - my only guess is that it's due to the buffers of draw "starts" and draw "counts" that must be uploaded to the GPU for drawing every frame, which will amount to ~1.6 MB of data for 200,000 items. It's hard to say for sure, though, because this isn't showing up on the profiler. It's possible that this GPU data upload is happening asynchronously and not reflected in the profiler unlike some of the texture upload function calls.

In the original example all of the problematic BatchedMesh sub geometry draws seem to be unique so unfortunately without something like indirect draw support (supported in WebGPU) I think this is just pushing the limits of what we can do with BatchedMesh too far.