SkinnedMesh acceleratedRaycast

[ashconnell]

In Three v0.152.0 they introduced a raycast method on SkinnedMesh (see: mrdoob/three.js#25791)

This causes it to no longer inherit the three-mesh-bvh accelerated raycast added to Mesh.

I changed my setup from:

// Add the extension functions
THREE.BufferGeometry.prototype.computeBoundsTree = computeBoundsTree;
THREE.BufferGeometry.prototype.disposeBoundsTree = disposeBoundsTree;
THREE.Mesh.prototype.raycast = acceleratedRaycast;

To this:

// Add the extension functions
THREE.BufferGeometry.prototype.computeBoundsTree = computeBoundsTree;
THREE.BufferGeometry.prototype.disposeBoundsTree = disposeBoundsTree;
THREE.Mesh.prototype.raycast = acceleratedRaycast;
THREE.SkinnedMesh.prototype.raycast = acceleratedRaycast;

And it seems to have fixed the slow raycasting issues I was having.

Does it make sense for this to be the new setup in the README?
If so, I will make a PR.

[gkjohnson]

Hello! Can you explain your use case? The BVH generated by computeBoundsTree is static with no bone transforms applied so it's never been intended to be used with SkinnedMesh.

[ashconnell]

In our virtual world platform people can drop GLTF models into the world in realtime, including avatars.

Before placing them in the world, every mesh calls .computeBoundsTree() so that our gaze and camera bumper raycasts are ultra fast.

This also included SkinnedMeshes since they extend Mesh.
We only generate a BVH once for these though, which means its only approximate but is "good enough".

Now that ThreeJS has added a raycast method to SkinnedMesh, our code continues to work but whenever you look at a SkinnedMesh the performance tanks hard because it no longer inherits the accelerated raycast from three-mesh-bvh.

[makc]

@ashconnell you could as well do

THREE.SkinnedMesh.prototype.raycast = function(){};

since what you did will not produce correct result any way

[gkjohnson]

I'd be hesitant to imply this is supported at all since there are common cases where the result will not at all be correct such as when the bone skeleton is rotated 90 degrees from the original model pose or significantly offset. I'm happy to leave this open in case others have thoughts but I think for the moment this has to be an unsupported expert use case / workaround.

[asyncore]

We ran into this exact issue and yes, since the bone transforms were not applied the raycaster would not produce the correct result. @gkjohnson , when you say this has to be an unsupported use case, is there no official way to accelerate raycasting for skinned meshes?

[gkjohnson]

when you say this has to be an unsupported use case, is there no official way to accelerate raycasting for skinned meshes?

There's currently no official way to seamlessly accelerate raycasting for skinned meshes, no.

Generally to accelerate skinned geometry raycasting you need to update the BVH based on the transformed vertices from the skinned mesh bones, which takes time. Whether that time to update the BVH is worth would depend on the use case and the complexity of the geometry. Most game engines will use a simplified mesh or simplified collider shapes like spheres or capsules to handle intersections with skinned geometry instead but this of course means you'll get a less precise result.

If you really need pixel-perfect accuracy with the mesh then a workaround solution right now would be to generate a static geometry based on the bones of the SkinnedMesh, generate and raycast against a BVH for that geometry, and then refit the BVH when needed. You should be able to use the StaticGeometryGenerator to generate the static sibling mesh, but it's otherwise not so difficult to manually generate, as well.

If this is for a professional project and you'd like to fund some ergonomic improvements to the project for skinned meshes (simplified functions, removing need for a copied geometry, etc) I'd be happy to discuss what that would look like if you want to send me an email.