What Is Happening?
Meta built a new system that lets you talk out loud to Muse and interrupt it mid-sentence. You do not have to wait for the AI to finish speaking before you reply. The app offers hundreds of voice styles, along with animated avatars that change their facial expressions and hand gestures to match the mood of your talk. Behind the scenes, two technologies—Muse Realtime Voice and Muse Realtime Avatar—run as one continuous streaming system. The voice model creates a single stream of audio tokens containing both the words and how they are said. This shared stream drives the voice and the face at the exact same time, keeping speech, lip movements, and expressions perfectly synchronized without needing a separate lip-syncing pass.
How Does It Work?
Starting a call opens a live talking session where you speak naturally, like talking on the phone with a friend. You can choose between voice-only mode, side-by-side view, or full-screen video mode. The avatar is driven by an AI model called a Diffusion Transformer, which reads the audio stream and recent video context to generate portrait video chunks at 25 frames per second. Using a process called model distillation, Meta cut the AI's step-by-step workload 60-fold—shrinking 120 complex teacher math checks down to just two student evaluation steps per chunk. This lowers the wait time to under one second (about 870 milliseconds) and allows one Nvidia GB200 server chip to run 12 video chats at once. Meanwhile, Muse can perform background tasks like web searches while your conversation keeps going.
Why Is It Important?
Moving from typed text to live spoken conversations makes a huge difference in how people use AI. By pairing a flexible voice with realistic facial expressions, Meta is turning a simple software tool into something that feels like a real digital companion. Small AI startups struggle to gain traction, but Meta reaches billions of users across WhatsApp, Instagram, and Facebook. This massive distribution network means Meta could bring animated, voice-driven AI companions into everyday life faster than any niche app.
What Are the Challenges?
Giving AI a voice and a face raises serious safety and privacy questions. To stop people from making fake videos, Meta adds an invisible watermark called "Video Seal" to all video chats and restricts the app to users 18 years of age and older. However, security experts still worry about how Meta handles personal voice and biometric data. People also worry about users forming strong emotional attachments to digital friends, as well as ongoing debates around targeted ads and copyright protections.
What Happens Next?
Meta plans to bring these voice features to hardware like Meta AI Glasses and a small wearable device called the Muse Charm. For now, the rollout is limited to the United States. The app is not yet available on international storefronts like the Canadian App Store, meaning creators in global markets like India will have to wait for regional access.
