Microsoft patents AI system capable of real-time altering of live speech and lip sync

Microsoft recently filed a patent application, US 2026/0341309 A1, detailing a system that uses artificial intelligence to modify spoken content in live broadcasts and AR presentations almost instantaneously. The technology, titled Near Real-Time Artificial Intelligence (AI) Enhancement of Video or Augmented Reality (AR) Presentations of Live Events, employs an AI agent that listens to audio to identify words for modification or deletion based on preset rules or viewer profiles, such as age, language, and location. Generative AI then replaces these words in the audio while simultaneously generating new video frames to ensure the speaker's lip movements perfectly match the altered speech. Potential modifications include filler words, stutters, mispronunciations, and offensive language, alongside colloquialisms like 'y'all' and 'bite the bullet.' The processed video is expected to have a latency of approximately 1 to 10 seconds. Furthermore, the patent covers real-time translation, allowing the system to substitute original speech with a translated version and synchronize the lip movements accordingly. While the application was submitted on April 29, 2025, and the patent office rejected certain claims on July 29, 2026, Microsoft has not confirmed plans for commercial deployment, noting that the filing is not an authorization of the technology.