Create a specification for rigid body physics

[jratcliff63367]

A number of people and companies have expressed an interest in having a GLTF extension which can describe rigid body physics.

A number of popular physics engines are widely used, each with a differing API and data serialization format. While many concepts between them are the same, there also tends to be significant differences between them as well.

Past efforts to create a standardized physics data definition have ended up with something so watered down that it was of questionable value, or so highly specific to one physics engine, that it could not be easily used by others.

I see the goal of this effort to create a data spec which represents a 'union' of the features of a number of popular physics engines; such that those things which are in common between them use a shared DOM (data object model) and those properties which are unique to each can still be specified and, conceivably, ignored by other physics engines which do not need them.

Even just the concept of a 'physics engine' can be challenging, as the definition can be wide ranging in scope. Some physics engines support soft bodies, particle systems, fluid dynamics, complex machines, and more.

While it would be desirable to support all features for all physics engines, that is probably unrealistic as an initial goal.

I would like to set the phase one goal to simply defining the basic components shared by any rigid body dynamics system; which is largely collision geometries, physical material properties, simulation settings, rigid body settings, collision filtering, and joints.

We should probably identify features that go beyond this, but leave them for future extensions efforts. Future work should also address the specific needs of robots which include cameras, sensors, grippers, transmissions, and a lot complex machine data.

Typically a DOM for a physics engine is similar to the scene graph you would see for rendering data; there are meshes and geometries which can be instanced at different locations and scale and they can be linked together with joints. There may need to be a way to specify a named binding between a physics representation and the visual representation it is associated with. Likely a parallel scene graph which simply describes those bindings will be needed.

The basic components of any rigid body system can be described in the following outline:

• Simulation properties
  • Each physics engine defines certain global properties which control how that system performs. Many of these properties tend to be unique for each different physics engine supported. Includes things like gravity, solver iteration count, and other global settings.
• Data Resources (objects with large amounts of binary data)
  • Triangle meshes
  • Convex Hulls
  • Heightfields
• Physics Materials
  • Defines physical properties of a material surface, friction, etc.
• Geometries (physical geometric representations for rigid bodies or parts of a compound rigid body)
  • Plane
  • Box
  • Sphere
  • Capsule
  • Cylinder
  • Triangle Mesh
  • Convex Hull
  • Heightfield
• Rigid Body (can be static, dynamic, or kinematic)
  • Each rigid body has one, or more, geometries and their associated unique properties; such as what material to use, what scale, etc.
  • A large array of physical properties are associated with each rigid body (mass, inertia tensor etc.)
• Aggregates
  • A collection of rigid bodies which can be treated as a single group.
• Joints (a joint is a type of constraint which joins two rigid bodies together)
  • Fixed
  • Spherical
  • Revolute
  • Prismatic
  • Distance
  • D6
• Body Pair Filters
  • Defines a sequence of object pairs which should or should not interact in the physics simulation.
• Collision filter logic
  • Describes the code/logic for how collision filtering between two rigid bodies should be interpreted.

The initial phase of this project will simply be requirements gathering. We have some graduated students who will be evaluating a number of well known physics engines to understand the data object model requirements for each. Once those are gathered we can try to create a 'union' of these specifications as well as identify engine specific properties. The last phase will be to actually create the schema itself and submit it for review. This process could take weeks to months to complete, and is only starting today.

Physics engines under consideration are:

• PhysX SDK : from NVIDIA
• FLEX SDK : from NVIDIA
• Bullet SDK : (open source by Erwin Coumans)
• Newton physics engine : (open source by Julio Jerez)
• Havok : from Microsoft
• Mujoco : Commercial physics engine, popular within the robotics community

[Moguri]

A rigid body physics extension has already been started as #1104. Maybe we can build from there?

[pjcozzi]

@jratcliff63367 thanks for kicking off this effort, it is much appreciated! Adding @akhabbaz and @rms13 to the conversation.

[pjcozzi]

@Moguri it would be great to see if there are collaboration opportunities and perhaps end up with one extension.

[jratcliff63367]

Yeah, I'm fine with merging the efforts. That one seemed specific to Blender only? I think the larger goal is to be more general to a wide range of physics engines.

[Kupoman]

@jratcliff63367 That extension is not meant to be Blender specific, and tries to generalize physics settings when possible. The reason it has the BLENDER prefix is because blendergltf is currently the only exporter using it. When multiple exporters and importers start supporting it, it can be promoted to an EXT or KHR extension.

[jratcliff63367]

Ok, this seems great then. We are looking for engineering resources to continue this effort so that it can encompass a number of popular physics engines as well. I'm trying to rustle up some college interns now.

[jratcliff63367]

Here's an example of a skeletal mesh and rigid body system I would like to be able to fully describe in GLTF.

https://www.youtube.com/watch?v=hw1TpxfglrE&feature=youtu.be

[donmccurdy]

I see the goal of this effort to create a data spec which represents a 'union' of the features of a number of popular physics engines; such that those things which are in common between them use a shared DOM (data object model) and those properties which are unique to each can still be specified and, conceivably, ignored by other physics engines which do not need them.

While it would be desirable to support all features for all physics engines, that is probably unrealistic as an initial goal.

I am a bit nervous about this broad extension scope. glTF has so far had a lot of success with focusing on common features that can be implemented consistently across many engines. Is there a (still useful) approach to physics that would allow us to do the same?

If a particular physics engine wants to jump in and propose their own extension (building on this one), e.g. BULLET_physics_extras, they can certainly do that. But if everything is defined in a single superset physics extension, it becomes very hard to ensure consistent implementations, as glTF's extensionsRequired and extensionsOptional are no longer useful indicators of support.

[jratcliff63367]

I was envisioning something where you specify the common properties and anything engine specific can be added on as additional key/value pair tags. I'm not really familiar with JSON schema specs yet, but that's fairly common in XML specs.

[meshula]

Sometimes the motivation to incorporate physics in a graphics specification comes from an observation that it's nice to represent arbitrary polytopes with vertices edges and faces, and hey, graphics representations do that too, but the end goals are very different, and I think that ultimately the equivalence is superficial. I say this because I did a couple of physics engines at LucasArts where initially I went in with the goal of sharing graphics data with the physics engine, but it never turned out that the data could be shared in a practical sense. Physics representations by necessity are simpler and tend to bound an object rather loosely, and hugely benefit from implicit formulations. I know I'm belaboring the obvious, but I wanted to be clear about where my objection is coming from.

That said, I'm not meaning to rain on the parade. To the contrary, I am enthusiastically in support of the creation of a physics schema that can be independent of a particular engine. I've always been hopeful and then disappointed when things like PAL appear and then don't really gain traction. I just don't feel that gltf shouldn't take on more than representations of graphics state and things to render. I'm wondering if it might be a good idea to form a Physics Interchange oriented group to come up with a specification, independent of gltf, for broader use.

Long story short, all the goals you've laid out are awesome, I think they deserve their own, independent of gltf, effort.

[vpenades]

Please, take BepuPhysics for consideration to the list of physics engines. It's one of the few 100% C# free physics engines around

[Moguri]

@jratcliff63367 What's the current state of this proposal? I would like to move #1104 along so we at least have rigid body physics, but I would still prefer to collaborate on physics in general.

Maybe it makes sense to break this general physics extension into multiple physics extensions (e.g., rigid bodies, soft bodies, constraints)? This may make it easier on consumers if physics isn't an all or nothing deal (e.g., start with rigid bodies, then start implementing something like constraints). A consumer could ignore most of the extension, but it is odd to list support for an extension if support for large chunks are missing.