Backend-agnostic GPGPU on Nodes #25273

LeviPesin · 2023-01-11T14:20:21Z

Description

A port of my ComputationRenderers from https://github.com/LeviPesin/THREE.js-PathTracing-Renderer/tree/webgpu/js/pathtracing/utils (originally made as a part of the unfinished port of erichlof's Three.js' pathtracing renderer).

API

See the example's code. Mainly just new functions WebGLRenderer.compute( ...computeNodes ), WebGLRenderer.deuploadBufferAttrribute( bufferAtrribute ), and WebGPURenderer.deuploadBufferAtrribute( bufferAtrribute, overwrite = false ) are introduced.

Original TODO

Remove explicit srcBuffer, move to ComputeNode and BufferAttribute, rework methods and arguments to constructor(renderer), createBuffer(...params), and compute(shaderNode, outBuffer)
Remove TypedBuffer by creating WebGLStorageBuffer in WebGLNodes and changing ArrayElementNode for it (also simplify BufferNode by inferring bufferType, also allow ComputeNode to use a buffer instead of count)
Remove ComputationRenderers by creating WebGLRenderer.compute() (in WebGLNodes)

API Modification

I don't really like the current API of ComputationRenderers -- I think @sunag's approach of WebGPURenderer.compute() and ComputeNodes is better...
ComputationRenderers mainly consist of setting up stuff needed for WebGL computations (this can be safely moved to a WebGLRenderer.compute() function and a thing like WebGLStorageBuffer to be able to use StorageNode), dealing with TypedBuffer.getBufferElement() and TypedBuffer.setBufferElement() (first can theoretically be moved to ArrayElementNode), and auto-transferring data from GPU back to CPU (this can also be moved to WebGLRenderer.compute() and WebGPURenderer.compute()).
Problems are in details -- for example, ComputeNode isn't really useful for WebGL computations -- they are not going for a specific number of iterations, they go for each pixel in the outBuffer. outBuffer is also the only buffer that we can write to in WebGL -- and even with it dealing with TypedBuffer.setBufferElement() is quite tricky...
API was changed.

Example (`misc_nodes_gpgpu`)

I made an example for testing WebGL and WebGPU computations, including some basic performance measuring.

Options

backend -- selects which backend will be used for doing computations.
example -- which example of a simple ShaderNode computation will be shown (Passthrough, AntiPassthrough, PassthroughColor -- outBuffer is 4-channeled instead of 1, ElementManipulation -- a srcBuffer is used, Multiplication, Random -- generates white noise based on PCG algorithm).
warmUps -- how many iterations of computations to perform; useful for performance measurements.
logBuffers and logPerformance -- whether to log data buffers (default false) and performance (default true).
refresh -- press to dispose renderer and clear caches.
clearConsole -- press to clear console.

Performance Problems

For some reason most examples are much slower with WebGPU. This is especially easy to see with Random example -- it takes about 100-150 ms to compute in WebGL and 200-300 ms in WebGPU on my machine.
For some reason, when changing backend from WebGL to WebGPU a short blink happens.

examples/jsm/gpgpu/WebGLTypedBuffer.js

examples/misc_nodes_gpgpu.html

LeviPesin · 2023-01-11T18:16:12Z

I think I've managed to create a prototype of WebGLBuffer, will commit it soon...

examples/misc_nodes_gpgpu.html

LeviPesin · 2023-01-12T13:55:33Z

WebGLBuffer works when used as a srcBuffer but fails for some reason when used as an outBuffer... Not sure why.

examples/misc_nodes_gpgpu.html

LeviPesin · 2023-01-12T13:57:17Z

examples/jsm/renderers/webgl/WebGLBuffer.js

+
+	}
+
+	dispose() {


I think we should also add a WebGPUBuffer.dispose() (and maybe BufferNode.dispose()) functions.

~~Also, when this function should be called?~~

LeviPesin · 2023-01-12T13:57:43Z

examples/jsm/nodes/accessors/TextureNode.js


 	generate( builder, output ) {

+		this.construct( builder ); // this is required for some reason?


Not really sure why...

examples/jsm/nodes/core/AssignNode.js

LeviPesin · 2023-01-14T14:38:22Z

I think I almost done fixing that error -- and also making a good API (nothing really user-facing except WebGLRenderer.compute()). Will commit it soon...

LeviPesin · 2023-01-14T16:13:11Z

For some reason in the shader appear mistakenly generated element( storage( attribute ), index ), which shouldn't be generated being a part of stack.assign( ..., ... )... Because of this the outBuffer is "used" as an srcBuffer and Chrome thinks a feedback loop is forming.
Here is the resulting shader:

// Three.js r149dev - NodeMaterial System

// <node_builder>

// uniforms
uniform sampler2D nodeUniform1; uniform vec2 nodeUniform2; 

// attributes


// varyings


// vars
vec4 nodeVar0; 

// codes


// </node_builder>

uniform vec3 diffuse;
uniform float opacity;
#ifndef FLAT_SHADED
	varying vec3 vNormal;
#endif
#include <common>
#include <dithering_pars_fragment>
#include <color_pars_fragment>
#include <uv_pars_fragment>
#include <uv2_pars_fragment>
#include <map_pars_fragment>
#include <alphamap_pars_fragment>
#include <alphatest_pars_fragment>
#include <aomap_pars_fragment>
#include <lightmap_pars_fragment>
#include <envmap_common_pars_fragment>
#include <envmap_pars_fragment>
#include <fog_pars_fragment>
#include <specularmap_pars_fragment>
#include <logdepthbuf_pars_fragment>
#include <clipping_planes_pars_fragment>
void main() {

	

	#include <clipping_planes_fragment>
	nodeVar0 = texture2D( nodeUniform1, vec2( ( ( 0.5 + float( ( ( ( uint( gl_FragCoord.y ) * uint( nodeUniform2.x ) ) + uint( gl_FragCoord.x ) ) % uint( 16.0 ) ) ) ) / 16.0 ), ( ( 0.5 + float( ( ( ( uint( gl_FragCoord.y ) * uint( nodeUniform2.x ) ) + uint( gl_FragCoord.x ) ) / uint( 16.0 ) ) ) ) / 16.0 ) ) );
	( vec4( vec3( ( ( ( ( uint( gl_FragCoord.y ) * uint( nodeUniform2.x ) ) + uint( gl_FragCoord.x ) ) * uint( 2.0 ) ) % uint( 256.0 ) ), ( ( ( ( uint( gl_FragCoord.y ) * uint( nodeUniform2.x ) ) + uint( gl_FragCoord.x ) ) * uint( 3.0 ) ) % uint( 256.0 ) ), ( ( ( ( uint( gl_FragCoord.y ) * uint( nodeUniform2.x ) ) + uint( gl_FragCoord.x ) ) * uint( 5.0 ) ) % uint( 256.0 ) ) ), 1.0 ) / vec4( vec3( 255.0 ), 1.0 ) );
	;
	
	vec4 diffuseColor = ( vec4( vec3( ( ( ( ( uint( gl_FragCoord.y ) * uint( nodeUniform2.x ) ) + uint( gl_FragCoord.x ) ) * uint( 2.0 ) ) % uint( 256.0 ) ), ( ( ( ( uint( gl_FragCoord.y ) * uint( nodeUniform2.x ) ) + uint( gl_FragCoord.x ) ) * uint( 3.0 ) ) % uint( 256.0 ) ), ( ( ( ( uint( gl_FragCoord.y ) * uint( nodeUniform2.x ) ) + uint( gl_FragCoord.x ) ) * uint( 5.0 ) ) % uint( 256.0 ) ) ), 1.0 ) / vec4( vec3( 255.0 ), 1.0 ) );
	#include <logdepthbuf_fragment>
	#include <map_fragment>
	#include <color_fragment>
	#include <alphamap_fragment>
	#include <alphatest_fragment>
diffuseColor.a = opacity;

	#include <specularmap_fragment>
	ReflectedLight reflectedLight = ReflectedLight( vec3( 0.0 ), vec3( 0.0 ), vec3( 0.0 ), vec3( 0.0 ) );
	#ifdef USE_LIGHTMAP
		vec4 lightMapTexel = texture2D( lightMap, vUv2 );
		reflectedLight.indirectDiffuse += lightMapTexel.rgb * lightMapIntensity * RECIPROCAL_PI;
	#else
		reflectedLight.indirectDiffuse += vec3( 1.0 );
	#endif
	#include <aomap_fragment>
	reflectedLight.indirectDiffuse *= diffuseColor.rgb;
	vec3 outgoingLight = reflectedLight.indirectDiffuse;
	#include <envmap_fragment>
	#include <output_fragment>
	#include <tonemapping_fragment>
	#include <encodings_fragment>
	#include <fog_fragment>
	#include <premultiplied_alpha_fragment>
	#include <dithering_fragment>
}

LeviPesin · 2023-01-14T16:18:21Z

examples/misc_nodes_gpgpu.html

+
+							const result = [ remainder( mul( index, 2 ), 256 ), remainder( mul( index, 3 ), 256 ), remainder( mul( index, 5 ), 256 ) ];
+							stack.assign( element( storage( outAttribute ), index ), params.backend === 'WebGL' ? color( ...result ) : uvec4( ...result, uint( 255 ) ) );
+							return stack;


These return stacks should not be needed ideally...

examples/jsm/renderers/webgl/nodes/WebGLNodeBuilder.js

LeviPesin · 2023-01-14T16:19:28Z

examples/jsm/renderers/webgl/nodes/WebGLNodeBuilder.js

+		this.setBuildStage( 'construct' );
+		node.build( this );
+		this.setBuildStage( currentBuildStage );
+		return node.build( this );


I feel like this is a little bit overcomplicated (but at least works)...

examples/jsm/nodes/display/ViewportNode.js

examples/misc_nodes_gpgpu.html

+			import WebGPU from 'three/addons/capabilities/WebGPU.js';
+			import WebGPURenderer from 'three/addons/renderers/webgpu/WebGPURenderer.js';
+
+			import { nodeFrame } from 'three/addons/renderers/webgl/nodes/WebGLNodes.js';


examples/misc_nodes_gpgpu.html

examples/webgpu_audio_processing.html

examples/jsm/renderers/webgl/nodes/WebGLNodes.js

examples/jsm/nodes/core/AssignNode.js

LeviPesin · 2023-01-15T11:35:30Z

examples/misc_nodes_gpgpu.html

+
+				const TypedArray = params.backend === 'WebGL' ? Uint8Array : Uint32Array; // WebGPU can only work with uint32
+
+				const index = instanceIndex;


Maybe make index an alias to instanceIndex in ShaderNodeBaseElements?

LeviPesin · 2023-01-16T13:33:48Z

I think I fixed that problem -- now Multiplication and Random examples render (although something is completely wrong with Multiplication colors -- maybe a problem somewhere in encodings or builder.getInstanceIndex). Investigating other problems...

LeviPesin · 2024-01-18T14:19:48Z

Closing because #27367 provides the same support.

LeviPesin added 2 commits January 10, 2023 00:51

Initial port

2b056f8

Clean up (includes TODO 1)

0a182f1

github-advanced-security bot found potential problems Jan 11, 2023

View reviewed changes

examples/jsm/gpgpu/WebGLTypedBuffer.js Fixed Show fixed Hide fixed

examples/jsm/gpgpu/WebGLTypedBuffer.js Fixed Show fixed Hide fixed

LeviPesin added 2 commits January 12, 2023 02:37

Fixes

fdd2250

Fix screenshot

c86b396

sunag reviewed Jan 11, 2023

View reviewed changes

examples/misc_nodes_gpgpu.html Outdated Show resolved Hide resolved

sunag reviewed Jan 11, 2023

View reviewed changes

examples/misc_nodes_gpgpu.html Outdated Show resolved Hide resolved

WebGLBuffer (TODO 2). Doesn't really work

6f0fbce

LeviPesin commented Jan 12, 2023

View reviewed changes

examples/misc_nodes_gpgpu.html Outdated Show resolved Hide resolved

LeviPesin commented Jan 12, 2023

View reviewed changes

Some nodes refactor

0a69697

github-advanced-security bot found potential problems Jan 14, 2023

View reviewed changes

examples/jsm/nodes/core/AssignNode.js Fixed Show fixed Hide fixed

Fix (still not working) (TODO 3)

cbf104b

Remove GH comments from code

bd312e7

LeviPesin commented Jan 14, 2023

View reviewed changes

examples/jsm/renderers/webgl/nodes/WebGLNodeBuilder.js Outdated Show resolved Hide resolved

LeviPesin commented Jan 14, 2023

View reviewed changes

github-advanced-security bot found potential problems Jan 14, 2023

View reviewed changes

sunag reviewed Jan 14, 2023

View reviewed changes

examples/webgpu_audio_processing.html Outdated Show resolved Hide resolved

sunag reviewed Jan 14, 2023

View reviewed changes

examples/jsm/renderers/webgl/nodes/WebGLNodes.js Outdated Show resolved Hide resolved

sunag reviewed Jan 14, 2023

View reviewed changes

examples/jsm/nodes/core/AssignNode.js Show resolved Hide resolved

Some cleanup

19d896d

LeviPesin commented Jan 15, 2023

View reviewed changes

Fix framebuffer loop problem

5871996

LeviPesin mentioned this pull request Mar 14, 2023

Allow webgl contexts to be GC'd sooner in Chrome. #25668

Closed

mrdoob modified the milestones: r151, r152 Mar 30, 2023

mrdoob modified the milestones: r152, r153 Apr 27, 2023

LeviPesin marked this pull request as draft May 3, 2023 14:36

Mugen87 modified the milestones: r153, r154 Jun 1, 2023

mrdoob modified the milestones: r154, r155 Jun 29, 2023

mrdoob modified the milestones: r155, r156 Jul 27, 2023

sunag mentioned this pull request Aug 28, 2023

Add HashNode #26663

Merged

mrdoob modified the milestones: r156, r157 Aug 31, 2023

mrdoob modified the milestones: r157, r158 Sep 28, 2023

mrdoob modified the milestones: r158, r159 Oct 27, 2023

mrdoob modified the milestones: r159, r160 Nov 30, 2023

mrdoob modified the milestones: r160, r161 Dec 22, 2023

LeviPesin mentioned this pull request Jan 18, 2024

WebGPURenderer: partial compute() shader support for WebGL backend #27367

Merged

LeviPesin closed this Jan 18, 2024

LeviPesin deleted the computation-renderer-port branch January 18, 2024 14:19

sunag removed this from the r161 milestone Jan 18, 2024

LeviPesin mentioned this pull request Feb 7, 2024

WebGPURenderer: Fix vec2 and vec3 for storageObject in StorageBufferNode #27697

Merged


		generate( builder, output ) {

		this.construct( builder ); // this is required for some reason?


		const TypedArray = params.backend === 'WebGL' ? Uint8Array : Uint32Array; // WebGPU can only work with uint32

		const index = instanceIndex;

Uh oh!

Backend-agnostic GPGPU on Nodes #25273

Backend-agnostic GPGPU on Nodes #25273

Uh oh!

Conversation

LeviPesin commented Jan 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

API

Original TODO

API Modification

Example (misc_nodes_gpgpu)

Options

Performance Problems

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LeviPesin commented Jan 11, 2023

Uh oh!

Uh oh!

LeviPesin commented Jan 12, 2023

Uh oh!

Uh oh!

LeviPesin Jan 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LeviPesin Jan 12, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

LeviPesin commented Jan 14, 2023

Uh oh!

LeviPesin commented Jan 14, 2023

Uh oh!

LeviPesin Jan 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

LeviPesin Jan 14, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Check notice

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LeviPesin Jan 15, 2023

Choose a reason for hiding this comment

Uh oh!

LeviPesin commented Jan 16, 2023

Uh oh!

LeviPesin commented Jan 18, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

LeviPesin commented Jan 11, 2023 •

edited

Loading

Example (`misc_nodes_gpgpu`)

LeviPesin Jan 12, 2023 •

edited

Loading

LeviPesin Jan 14, 2023 •

edited

Loading