Acknowledgements

Unity toolkit for Sign Language Recognition Integration.

Designed and Maintained by

• Ananay Gupta ([email protected])
• Nicholas Unger ([email protected])
• And the Unity subteam members:
• Marie Andry
• Charlie Lin

Setup

Getting started with the toolkit is relatively simple, but there are a few things you need to do to make sure you're ready to go.

Step 1: Download Unity Hub.

• Assuming you do not already have a version of Unity to work with, you'll need to install Unity and the version of Unity the toolkit is built on. Navigate to Unity and download Unity Hub for your platform of choice.

• Navgiate to the Unity download archive.

• Search for Version 2022.3.45f1. Select "Hub installation".

• Within Unity Hub, install Version 2022.3.45f1. Select the platforms you anticipate building your game to in the future. (The screenshots are not comprehensive; review all the platforms you can build to in the installation wizard and select the ones you need.)

Step 2: Clone the repo and add the project.

• This is pretty simple. Clone this repo using either GitHub Desktop or the Terminal or Command Line.
• In Unity Hub, select Add -> Add Project from Disk and navigate to where you cloned the SLRCore-Unity repo to. You can now use the toolkit locally.
• Note: If you try to open the project before adding the necessary packages, you may receive a warning asking you if you want to enter Safe Mode. Click ignore for now. These compilation errors will go away once you download the necessary packages.

Step 3: Download the necessary packages and add them to the project.

Step 3a: Download the MediaPipe for Unity plugin

• Navigate to Homuler's MediaPipe plugin for Unity and then find the releases page.
• From the most recent release, download the file with the file format .unitypackage. In the attached screenshot, it is named "MediaPipe.0.15.0.unitypackage".

• Now you can add the package to the toolkit. Open the SLRCore-Unity project in Unity Hub, and from the menu bar, select Assets -> Import Package -> Custom Package...

• In your OS's file explorer, navigate to where you downloaded the MediaPipe for Unity package and select it.
• In the import Unity Package Wizard, you can select what to import and what to not import. If you're unsure, import everything.

• You've successfully added the MediaPipe plugin.

Step 3b: Install TensorFlow Lite for Unity.

• From the Unity project, select Edit -> Project Settings in the menu bar.
• A pop-up window should open. Navigate to the Package Manager tab.

• Click the plus button in the bottom left of the left column of the "Scoped Registries" tab to add a new scoped registry.

• You now need to add two scoped registries. Fill in the Name, URL, and Scope(s) exactly as shown below.

• Close the Project Settings window. In the sidebar, click the packages folder, then, in the menu bar, select Assets -> View in Package Manager.

• In the Package Manager pop up window, select the drop-down and select "My Registries".

• Under the dropdown labeled "Koki Ibukuro," find "TensorFlow Lite" and click the Install button.

• Assuming you've also added the MediaPipe plugin package, the toolkit is now fully operational and you can use the wealth of hand recognition features to create sign language recognition games for multiple platforms.

Step 4: Build your game.

• Once you're done developing your game, you can build to your platform of choice. From the menu bar, select File -> Build Settings if you need to change aspects of your build.

• The Build Settings window should pop up. From here, you can select which scenes you would like to build as well as which platform you would like to build to. Keep in mind that, if you are changing build platforms (say, from Windows to Android,) you must select the "Switch Platform" button and allow Unity some time to prepare to build for that platform. Each platform you build for has its own platform specific settings in addition to the build specific settings. For more information, refer to the Unity documentation on build settings.

Step 5: Happy developing!

• That's all you need to get started with the Unity Sign Language Recognition toolkit. The toolkit gives you access to robust tools to help quickly develop games that take advantage of sign language recognition capability and create experiences unlike anything seen before.

• If you have questions, please feel free to reach out to Ananay Gupta or Nicholas Unger, whose emails are listed above.