Soft Shadows via Light Source Sampling

[bhouston]

The idea here is that we use the fastest shadow algorithm possible, and then accumulate it across frames. To get accurate accumulated soft shadows, we should jitter the light origin within the volume of the light. For a lights of any type we jitter in their volume in a Poisson-like fashion. This will create physically correct penumbra if you accumulate over time.

We can either accumulate in the main frame buffer or we could accumulate in a shadow buffer.

Sketchfab is currently jittering their light sources in order to get soft shadows when the scene is static. We have also recently implemented a basic light jitter approach in Clara.io and it gives good result. We should take it to the next level and integrate it with core Three.JS.

[bhouston]

More details here: http://dee.cz/fcss/

[mrdoob]

Similar to this? http://madebyevan.com/shaders/lightmap/

[bhouston]

Similar to this? http://madebyevan.com/shaders/lightmap/

Fairly similar, but it should look better faster and less messy.

[looeee]

it should look better faster

When we are talking about accumulating across frames, how many frames will it take before it looks good? Will there be ugly jank at the start?

[Usnul]

@looeee

When we are talking about accumulating across frames, how many frames will it take before it looks good?

• that depends on sampling approach. Going by standard PCF implementation with 16 taps - you would need about that to get similar quality.

Will there be ugly jank at the start?

• Yes, it will look like ugly jank at the start :)
• You can hold the first few samples to avoid this "jank" from being seen.

[bhouston]

The result should look like what we have now when the scene is dynamic, but soon as it stops moving, we can apply the shadow jitting within the light volumes and it should refine to soft shadows. It is like this TAA example I Wrote a few years ago but instead of sub-pixel jittering the camera to anti-alias, it jitters the lights in their source volumes to get physically correct penumbra:

https://threejs.org/examples/webgl_postprocessing_taa.html

[gkjohnson]

Here's a quick proof of concept for this sort of thing that I put together awhile ago and basically does what's being described here:

https://raw.githack.com/gkjohnson/threejs-sandbox/0f2c762/volume-lights/

A pointlight is moved along the surface of an emission volume while the frame is merged with the previous buffer.

@bhouston

We should take it to the next level and integrate it with core Three.JS.

How are you imagining this being added to core? Would it be a separate renderer? Or an option in the current one to automatically jitter and accumulate lights over time if the camera didn't change position?

We can either accumulate in the main frame buffer or we could accumulate in a shadow buffer.

Can you elaborate on what you mean by "accumulate in a shadow buffer"?

[gkjohnson]

If this is still interesting I updated my volume-lights demo when migrating to modules with a few added features. It now includes random surface sampling of the light shape (thanks to @donmccurdy's MeshSurfaceSampler!), an option to fade the result in after it's had a moment to resolve to avoid the janky look at the start, and an option to use more point lights per frame:

https://raw.githack.com/gkjohnson/threejs-sandbox/bb05ead/volume-lights/

Using random sampling and more lights results in the image resolving much more quickly (~1 to 3 or 4 seconds). For the random sampling case I've overriden the Math.random function with one that can be seeded so the same initial shadow results can be reproduced when moving the camera.

[bhouston]

@gkjohnson that is beautiful stuff. My favorite settings are:

• light count: 3
• fade, true
• fade delay, 1 second
• fade transition, 1 second

I think one issue with it is that the light/shadow used when moving around isn't very representative of the final iterative average shadow. I wonder if there is a way to fix that so that the instanteous is closer to the average.

[gkjohnson]

@bhouston thanks! I think I prefer the shorter fades myself.

I think one issue with it is that the light/shadow used when moving around isn't very representative of the final iterative average shadow. I wonder if there is a way to fix that so that the instanteous is closer to the average.

Using more lights helps with this a bit because the lights are spread among the object and the shadows are more representative of where they'll wind up. Deliberately seeding lights at the extremes for the first render -- points on the shapes surface closest to the 8 corners of a bounding box, for example -- could help give a more complete impression of the file shadow extents but the discrete shadows won't look great. VSM or PCSS shadows might be better just for the initial render, as well. Temporal reprojection AA would help with the initial few renders when moving the camera, too.