@react-three/xr does not play well with @react-three/postprocessing

[Bossieh]

First of all thanks a lot for this awesome library! The future is now! :D

But it seems that when I use <EffectComposer> from @react-three/postprocessing in my scene the hit testing goes wrong. There are still hit results but the position/rotation is completely wrong.

If I remove the <EffectComposer> everything works completely fine.

Versions i'm using:

• @react-three/xr: 5.0.4
• @react-three/postprocessing: 2.4.6
• @react-three/fiber: 8.0.19

<XRButton
    mode="AR"
    sessionInit={{
        requiredFeatures: ['hit-test'],
        optionalFeatures: ['dom-overlay', 'dom-overlay-for-handheld-ar'],
        domOverlay: { root: document.body }
    }}
>

<Canvas
    gl={{
        preserveDrawingBuffer: true
    }}
    shadows
    frameloop="demand"
    onCreated={(rootState) => {
        rootState.camera.position.x = 12
        rootState.camera.position.y = 5
        rootState.camera.position.z = 13
    }}
>
    <XR referenceSpace="local">
        // my 3D scene
    </XR>
    <EffectComposer autoClear={false}> // this guy messes up hit testing
        <Outline
            blur
            edgeStrength={4}
            pulseSpeed={0.5}
            visibleEdgeColor={0xffffff}
            hiddenEdgeColor={0xffffff}
        />
        <SMAA />
    </EffectComposer>
</Canvas>

[CodyJasonBennett]

I'm assuming you're testing on a mobile device? https://github.com/pmndrs/postprocessing doesn't formally support WebXR rendering, not yet -- I'll be implementing this once they support viewport regions.

I don't believe EffectComposer is using camera transforms from WebXR pose, which would be the only way the hit test would not be aligned. You may want to add a lambda pass and call gl.xr.updateCamera(state.camera) to keep the scene camera in sync with WebXR.

You may also be interested in taking rendering into your own hands in the meantime, see https://twitter.com/Cody_J_Bennett/status/1482585611781480448, https://codesandbox.io/s/7x7p3q.

[Bossieh]

Thank you for your quick reply, yes I am testing this on a mobile device.

So if I understand correctly @react-three/postprocessing has to support XR first? If so I will open an issue about this at @react-three/postprocessing

[CodyJasonBennett]

Indeed, it will need work from both @react-three/postprocessing and postprocessing.

[Bossieh]

I have opened an issue at @react-three/postprocessing: pmndrs/postprocessing#452

[hecodeit]

You may also be interested in taking rendering into your own hands in the meantime, see https://twitter.com/Cody_J_Bennett/status/1482585611781480448, https://codesandbox.io/s/7x7p3q.

@CodyJasonBennett
Codesandbox demo don't working now, only showing black screen.

And,
useDepthBuffer of @react-three/drei get the same problem?
Try get it work, but don't look correct.

#1427

[bbohlender]

Postprocessing with threejs unfortunately does not work in WebXR. This is not related to this repository though, just a limitation of threejs.