v3d-web

Single camera motion capture/humanoid rigging in browser.

Objectives

v3d-web is a project aiming to bridge the gap between our reality and the virtual world. With the power of latest XR and web technologies, you can render and rig your favorite 3D humanoid avatars right inside your browser. Try it out now!

Features

• Contains a complete solution from image capturing to model rendering, all inside browsers.
• Thanks to the latest machine learning technologies, we can achieve a fairly accurate facial and pose estimation from a single camera video stream.
• It also comes with a complete WebGL rendering engine, in which VRM models can be present with highly complicated backgrounds. VRM model rendering is done with v3d-core.
• There is a demo site showing how this project can be seamlessly embedded into a modern UI framework like React.js.

Usage

Install from NPM

npm install v3d-web

In browser

Only latest FireFox, Chrome/Chromium, Edge and Safari are supported. WebGL and WebAssembly are necessary for this project to work correctly.

In the simplest case, all you need is:

const vrmFile = 'testfile.vrm';
try {
    this.v3DWeb = new V3DWeb(vrmFile);
} catch (e: any) {
    console.error(e);
}

You will need HTML elements with certain ids. See index.html.

A more complicated example can be found at the repo for our demo site.

Contributing

See CONTRIBUTING.md.

Build

1. Clone this repo and submodules:

   git clone https://github.com/phantom-software-AZ/v3d-web.git

2. Build v3d-web

   npm install
   npm run build && tsc

Debugging

Go to root folder of this repository and run:

$ npm run debug

The debug page can be opened in any browser with http://localhost:8080/.

Demo Site

See this demo site for a live example.

Credits

• MediaPipe
• babylon.js
• babylon-vrm-loader
• babylon-mtoon-material
• VRM Consortium
• Demo model used: Kaori/ Kaori by ClaValLuis From VRoid Hub. Used according to VRM PUBLIC LICENSE 1.0.

Acknowledgement

An adorable and perseverant individual who keeps on pursuing dreams.

Licenses

see LICENSE.