Tydra work items

[syoyo]

dev is a current active working branch for Tydra https://github.com/syoyo/tinyusdz/tree/dev/src/tydra , especially OpenGL/Vulkan friendly data structure conversion.

Tydra provdes a handy logic to convert some USD scene with material/texturing to OpenGL/Vulkan-like renderer.
Limited support of keyframe animation, skinned mesh and blendshapes.

Phase1(mostly done)

Node hierarcy(scene graph), static mesh, material/textures

• Scene graph construction

  • Xform(at a specific time)
  • Mesh, GeomSubset

• Load asset from USDZ

  • AssetResolution and FileSystem handler support for USDZ Is there a way to perform asset resolution? #140 Embedded texture buffer empty #134

• Material/Texturing

  • BindMaterial resolution(consider material bind strength).
  • Basic UsdUVTexture/PrimvarReader support. UV primvar name resolution.
  • Basic Texture image load(jpg, png, exr, ...)
  • Custom texture image loader? [TODO] [Tydra] Support custom texture loader callback #143
  • Linear-sRGB colorspace conversion of texture.

• Mesh conversion

  • Convert mesh at a static time

• Mesh Triangulation

  • Triangulate common vertex attributes(normals, texcoords, vertex_colors, vertex_opacities)
  • Triangulate GeomSubset indices
  • Validate triangulation is correct

• Vertex index buffer build

  • To render a mesh with single index buffer(OpenGL/Vulkan)
  • Convert 'facevarying' primvars to 'vertex' primvars
  • Validate index buffer build is correct

Phase2(mostly done)

• Vertex Skinning

  • Read vertex weights
  • Build skel hierarchy(bone hierarchy)
  • Skinning transform matrix build(?)

• Animation

  • Node xform animation(timeSamples)
  • SkelAnimation(rotation, translation, scale) https://github.com/syoyo/tinyusdz/tree/tydra-skelanimation

• BlendShapes

  • Basic BlendShape Prim support

• Muli-value in Attribute support Support having multiple values in Attribute #154

• Test with some USD files

=> merged into dev

Advanced(future task)

• AR object [Tydra] [RenderScene] Support some AR specific Mesh/Material(Collider Mesh, Physics Material, ...) #163
• BlendShape
  • In-between blendshapes
• Optimize vertex buffer and vertex index build.
• Wide color texture support
  • Linear-displayP3 colorspace conversion of texture
  • ACEScg(Linear) colorspace conversion of texture.
• Subdivision surface
  • Subdivide surface using osd(OpenSubdiv)
  • Interpolate primvars
• Material collection
• MaterialX support
• Vertex animation of a mesh
• UDIM texture support
• Support some ACES color profile as a built-in
• Multiple Animation(SkelAnimation) support

[syoyo]

With a1f1a06 , at least single textured quad USC scene is now able to convert to RenderMesh/RenderScene.

tydra_to_renderscene models/uv-quad-textured.usda --dumpobj

Then open .obj with MeshLab

[syoyo]

Note on testing using tydra_to_renderscene example.

tydra_to_renderscene input.usd [--dumpobj] [--notri] [--noidxbuild]

--dumpobj Dump mesh as wavefront .obj
--notri Do not triangulate mesh
--noidxbuild Do not rebuild vertex indices(for raytraced renderer or mesh data conversion)

[syoyo]

DamegedHelmet.usdz can now be converted through

tydra_to_renderscene DamagedHelmet.usdz --dumpobj

then open .obj in MeshLab(or Blender)

[syoyo]

In Blender, set up-axis -Z to load .obj(exported from tydra_to_renderscene)

[syoyo]

Object capture example(No texture colorspace conversion is done yet though)

[syoyo]

To support animation correctly, we'll need to implement #154 since some USDZ file(e.g. robot_walk_idle.usdz has the following attribute

def SkelAnimation "Animation " {
  quatf[] rotations = ... # default value(value at `Default` timecode)
  quatf[] rotations.timeSamples = ...
}

as a workaround, rendermesh-refactor can support having default value and timeSamples(but uses timeSamples only when both are declared) in this commit: bdedcc0

so at least SkelAnimation in some USDZ files can now be handled correctly(but no default time value though).

[syoyo]

tydra_to_rendermesh robot_walk_idle.usdz --dumpobj is now at least able to export meshes and skin weights . Next step is to convert Skeleton(skel hierarchy) and SkelAnimation(animation samples)

To support Skinning, I am working on RenderScene to USD exporter(read USD with Blender to test RenderScene's Skinning data is correctly converted)

[syoyo]

Initial success of supporting SkelAnimation convert in renderscene-refactor branch.
(now in 'dev' branch)

tydra-usd-skintest.mp4

Read USD -> Convert to RenderScene through Tydra -> Export USD again(tydra_to_renderscene --dumpusd) -> Read it in Blender 4.1

[syoyo]

Initial support of BlendShape read/export.

https://github.com/syoyo/blender-usdz-test-scenes/tree/main/scenes/blendshapes

blendhsape-animation.mp4

tydra_to_renderscene input.usd --dumpusd, then open USD file in Blender 4.1

[syoyo]

Extended Vulkan-glTF-PBR with TinyUSDZ Tydra to load USD asset.

Current status: Static mesh + diffuseColor texture works.

https://github.com/syoyo/Vulkan-glTF-USDZ-PBR

[syoyo]

Now rendermesh-refactor is merged into dev. b41e1dd Further Tytra-related development will be done in dev branch.

[syoyo]

Now PBR shading starting to work!

https://github.com/syoyo/Vulkan-glTF-USDZ-PBR