PerspectiveCamera - New helper method `.getMarginAt(distance)`

[Bug-Reaper]

Description

Looking to PR a new PerspectiveCamera helper method .getMarginAt(distance) which returns the height, width and Vector3s for the corners of the camera's viewable area at the input arg distance.

A PerspectiveCamera's viewable margins at an arbitrary distance is a foundational piece of info you can build a lot of clean experiences from. It enables consistent position and scale for 3D designs and is practical for devices or UX with a variable aspect ratio that must be accounted for.

Some common use-cases include:

Guarantee your rendered content fits on the screen.

Scale 3D content to take up an exact percentage of available screen height/width.

Situations where you want to have objects cleanly enter and exit the visible area.

Reset a user controlled camera to an ideal-default.

Place a backdrop plane that exactly covers the visible area.

Solution

Took a function I've been copying around for years and modified it to be a helper method like so.

Alternatives

Wanted some takes on how to best return the result, currently returns the info in a JSON obj like this:

{
  topLeft:<Vector3>,
  topRight:<Vector3>,
  bottomRight:<Vector3>,
  bottomLeft:<Vector3>,
  height:<float>,
  width:<float>
}

I can't think of any methods that return JSON like this but perhaps it's okay? I also couldn't find any native THREE class that contains all of the above in an easily accessible way. If we don't want to just return JSON obj like this perhaps there's an opportunity to extend an existing class or create a new one for this use-case.

[WestLangley]

Related: https://stackoverflow.com/questions/13350875/three-js-width-of-view/13351534#13351534

[Mugen87]

@WestLangley How about we turn your answer into a separate helper method first. Considering the upvotes for your answer, it seems a lot of devs are searching for this topic.

Maybe something like PerspectiveCamera.getDimensionAtDistance( target: Vector2 )?

Sidenote: Since the dimension at a distance between [near,far] is equal for orthographic cameras, we don't need an abstract method on Camera level.

[gkjohnson]

WestLangley How about we turn your answer into a separate helper method first. Considering the upvotes for your answer, it seems a lot of devs are searching for this topic.

Just a note that the answer provided in the link is only relevant for the symmetrical projection matrices set via the PerspectiveCamera fields. It will not provide a correct answer for manually assigned skewed or other custom projection matrices like those WebXRManager assigns.

[WestLangley]

1. Since the camera can be tilted, I'd suggest "frustum height" instead of "visible height".
2. But we have to take into consideration camera.zoom, so maybe "visible height" is better after all. 🤔
3. In addition to what @gkjohnson said, "view offset" is not accommodated.

Anyway, I would prefer an API that is something like the following:

frustumHeight( distance ) { // or visibleHeight()

	return 2 * Math.tan( MathUtils.DEG2RAD * 0.5 * this.fov ) * distance / this.zoom;

}

frustumWidth( distance ) { // or visibleWidth()

	return this.frustumHeight( distance ) * this.aspect;

}

[Bug-Reaper]

I like the API suggestion proposed 👍

Took that and added a couple of things:

• Made frustumWidth() & frustumHeight() account for the .scale of the camera & its ancestors.
• Added a frustumCorners() function to return Vector3s for the four frustum corners at a given distance.

Plan to commit in the docs bump soon and PR.

[Mugen87]

Do devs typically want both the frustum height and width at the same time? Or are there use cases where you only want one of them?

If you need both most of the time, I find it a waste that frustumHeight() is effectively called twice. On the other hand if frustumWidth() and frustumHeight() are usually not called in the animation loop, I guess this tiny issue can be ignored.

Made frustumWidth() & frustumHeight() account for the .scale of the camera & its ancestors.

As long as we didn't solve #26659, maybe it's better to ignore the scale topic at the moment.

[gkjohnson]

Imo this should operate on the projection matrix itself rather than using the settings on the camera. And I think it's okay to make an assumption that the projection matrix does not include other rotation or translation transforms in it.

Again because these matrices can be off-axis it's not enough to return a width and height. If a user is using this with a WebXR frustum to place something the result will not be what's expected. Providing a 2d min bounds and 2d max bounds point should be enough to represent the view bounds at a certain distance in the frustum. These would be bounds provided in the camera frame.

const distance = 1;
const min = new Vector2();
const max = new Vector2();
camera.getViewBounds( distance, min, max )

Do devs typically want both the frustum height and width at the same time?

I think I would expect both of them to be provided simultaneously as all other size getter functions do.

[Bug-Reaper]

Do devs typically want both the frustum height and width at the same time? Or are there use cases where you only want one of them?

Of my own common use-cases, most of them need both. Most of the dev inquiries about this in discord/forum/SO are for cases where both are needed.

If you need both most of the time, I find it a waste that frustumHeight() is effectively called twice.

I noticed this too, but did like the API layout of separate height/width methods 🤔. For performance sensitive implementation you could always run frustumHeightand calc width on your own but you'd have to be aware of the caveat w/frustumWidth.

As long as we didn't solve #26659, maybe it's better to ignore the scale topic at the moment.

Agree with this.

---

Here's an alternate version where we have:

• frustumDimensions( distance ) => returns: { height:<float>, width:<float> }

• frustumCorners( distance ) => returns: { topLeft:<Vec3>, topRight:<Vec3>, bottomRight:<Vec3>, bottomLeft:<Vec3> }

I'd like to improve and account for WebXR frustum use-cases also. Believe either of the past two implementations adds a lot of value for default PerspectiveCamera based development already.

[Bug-Reaper]

Imo this should operate on the projection matrix itself rather than using the settings on the camera. And I think it's okay to make an assumption that the projection matrix does not include other rotation or translation transforms in it.

So I see that we can extract some props from the PerspectiveMatrix:

  const aspectRatio = projectionMatrix.elements[5] / projectionMatrix.elements[0];
  const fovRadians = 2 * Math.atan(1 / projectionMatrix.elements[5]);
  // const zoomFactor = ???? Couldn't figure this one? Possibly irrelevant b/c of update to matrix[0]/matrix[5]?

I think fov calc is marginally faster and aspect ratio is marginally slower. I guess the benefits are more to do with custom perspectiveMatrix situations.

Also I'm a little unclear how two Vector2s provide a more accurate height/width? Think I need to better understand what type of manipulation the custom WebXR perspective matrix apply to the camera frustum?

[gkjohnson]

I think fov calc is marginally faster and aspect ratio is marginally slower. I guess the benefits are more to do with custom perspectiveMatrix situations.

I'd start with a correct answer before worrying about marginal performance differences.

Think I need to better understand what type of manipulation the custom WebXR perspective matrix apply to the camera frustum?

As an example here's a visualization of a more extreme off axis frustum from this presentation. As you can see providing just the width and height of the frustum at a certain distance is not enough to capture the box that describes the view.

Also I'm a little unclear how two Vector2s provide a more accurate height/width?

It's the same principle as how Box3 can describe an axis aligned bounding box with two 3d Vector3s. Each point describes the minimum and maximum extent along the x and y axis

You can find the bottom left and top right points of this 2d bounds by projecting a ray out onto the distance you want the bounds at, similar to raycaster.setFromCamera. Note that this code is notional and should be validated:

function getBounds( dist, minTarget, maxTarget ) {

  temp.set( - 1, - 1, 0.5 ).unproject( camera );
  temp.multiplyScalar( dist / Math.abs( temp.z ) );
  minTarget.x = temp.x;
  minTarget.y = temp.y;

  temp.set( 1, 1, 0.5 ).unproject( camera );
  temp.multiplyScalar( dist / Math.abs( temp.z ) );
  maxTarget.x = temp.x;
  maxTarget.y = temp.y;

}

[Mugen87]

So if developers want the width and height of the visible rectangular region at a certain distance, they would do this:

camera.getBounds( dist, minTarget, maxTarget );
const width = maxTarget.x - minTarget.x;
const height = maxTarget.y - minTarget.y;

How about we simplify the width/height computation and provide an additional method like Box3.getSize(). This method could internally use getBounds() and does the above computation for the user. Based on @Bug-Reaper's suggestion in #27556 (comment):

getFrustumDimensions( dist, target ) {

    camera.getBounds( dist, minTarget, maxTarget );
    target.x =  maxTarget.x - minTarget.x;
    target.y = maxTarget.y - minTarget.y;

    return target;

}

frustumCorners() would need no further updates.

Everyone happy with this? 😇

[gkjohnson]

How about we simplify the width/height computation and provide an additional method like Box3.getSize()

Are there use cases that only need the dimensions of the view port but not the bounds? The ones all listed in OP require placing or animating something relative to the view rectangle. I suppose if a user could just be assuming that the frustum is symmetrical when placing something?

Either way I'm fine with the proposal. I'm just curious if the width and height are actually what's needed for the provided use cases.

[Mugen87]

I guess it does not hurt if PerspectiveCamera provides both getters for bounds and dimensions (as a convenient method). Devs can then pick the best method for their code.

@Bug-Reaper Are you in for a PR^^?

[Bug-Reaper]

Either way I'm fine with the proposal. I'm just curious if the width and height are actually what's needed for the provided use cases.

Some of the common scenarios lend themselves to a centerpiece object that needs to be scaled to fit perfectly within height/width. In these cases, the specific bounds are not needed since object is placed in the already-known center of view.

Example codepen (not mine) where some 3D stuff is sized based on frustum height/width and not bounds.

---

Proposal sounds good to me 🤝

Tested getBounds() code from @gkjohnson's comment #27556 (comment), just added a slight bump so the method remains accurate when the camera.position/camera.rotation weren't (0,0,0)/(0,0,0,1).

// ...
  temp.set( - 1, - 1, 0.5 ).unproject( camera ).applyMatrix4(cam.matrixWorldInverse);
// ...
  temp.set( 1, 1, 0.5 ).unproject( camera ).applyMatrix4(cam.matrixWorldInverse);
// ...
// EDIT: Looks like we can temp.set( x, y, z ).applyMatrix4( camera.projectionMatrixInverse ) to accomplish same as above 3x faster. Plan to amend in final proposed code

Love this approach!

---

All together here's 3 methods: getBounds(), frustumDimensions(), frustumCorners()

 getBounds( distance : float, minTarget : Vector2, maxTarget : Vector2 ) : undefined
 frustumDimensions( distance : float ) : Vector2
 frustumCorners( distance : float ) : Object

All methods should now account for asymmetric perspective matrix via the improved getBounds() foundation. Plan to add a bit more polish, doc updates, and PR.