lala-companion

3D personified desktop assistants, tuned for you, powered by AI vision and voice.

Early access in-development.

Features

• 3D VRM react-three-fiber avatars.
• Always on microphone, voice to voice AI conversations.
• Screen vision with GPT4-V. Self operating computer functions.
• Install Lala Companion on Linux, Windows or Mac.
• Resizable transparent overlay frame, always on-top AI interface for desktops.

Join Lalaland

Discord | X

Examples:

demo.mp4

Dev Local Setup

npm i --force

npm run start

Dev Notes

This Electron React app consists of 2 main renderers or "views" that talk to eachother through IPC. The reason being, is its hard to build nice GUIs in an also fully transparent, click through, focus-less threejs 3D scene.

As a rule, no buttons or 2D layers go in the 3D overlay. Only threejs 3D. Vanilla UIs go in "main". It ruins the vibe to mix.

Use Typescript.

AI Streaming is done with ai package from Vercel. useChat hook is good.

We are hosting public AI APIs connected in source code from Lalaland company as a donation out of good faith, these will be subject to logins, rate limits and API keys soon.

A local backend mode would also be nice, coming soon.