Thanks to Full-Duplex Architecture: ChatGPT Voice listens while it speaks
OpenAI renews ChatGPT Voice with GPT-Live: New generation of language models uses a full-duplex architecture, the model now listens and speaks simultaneously.
(Image: PeopleImages / Shutterstock.com)
Previously, ChatGPT Voice could either listen or speak, but not both simultaneously. The new GPT-Live model aims to fix this. Instead of waiting for speech pauses, the model processes the audio stream continuously using the full-duplex method and decides on the fly whether to speak or listen. Short feedback like “Mhmm” should be possible, as well as actual interruptions.
The original ChatGPT Voice function from 2023 worked as a triple cascade: Speech-to-Text (STT) transcribed the input, a language model generated a response from it, and Text-to-Speech (TTS) finally spoke it out. In 2024, the Advanced Voice Mode replaced this triple chain with a single audio model, noticeably improving latency and naturalness. However, this did not change the turn-based principle.
(Image: OpenAI / Bearbeitung heise medien)
Delegate Instead of Waiting
In addition to the full-duplex method GPT-Live brings another innovation that is intended to make conversations smoother. It delegates computationally intensive tasks in the background to GPT-5.5. Those who ask for something during a conversation that requires a web search or deeper reasoning should not have to wait. The conversation continues until the result is available. According to OpenAI, users can choose between three levels: Instant for quick answers, Medium, and High for more complex questions (Reasoning).
In OpenAI's own measurements, testers preferred GPT-Live-1 in 75.7 percent of cases compared to Advanced Voice Mode, and GPT-Live-1 mini in 69.2 percent. According to OpenAI, the hit rate on the scientific reasoning benchmark GPQA increases from 45.3 to 84.2 percent. On the search benchmark BrowseComp, it jumps from 0.7 to up to 75.2 percent, which the company explains almost exclusively by background delegation.
Videos by heise
Security During Ongoing Conversations
Because downstream moderation does not work in real-time conversations as it does in text chats, GPT-Live can intervene during the conversation by redirecting its output, offering resources, or ending a conversation in high-risk situations. For teenagers, age-appropriate behavior has reportedly been trained directly into the model, and parents can control the function via parental control settings. In cases of self-harm, an adapted support process, including a referral to a crisis hotline, is triggered. Voice imitation is not intended: GPT-Live exclusively uses predefined voices.
GPT-Live should not be confused with OpenAI's Realtime API. The latter is a configurable developer interface for custom applications, such as call centers, voice assistants, or translation services. It continues to work on a turn-by-turn basis, even though latency has significantly decreased compared to the old cascade. The GPT-Realtime-2 presented in May introduced an extended context window of 128,000 tokens, parallel tool calls, and adjustable reasoning levels.
At launch, video, and screen sharing during voice calls, which were previously a feature of Advanced Voice Mode, are missing. OpenAI notes that for less common languages, the model may speak with a foreign accent or experience gaps.
The first model, GPT-Live-1, will be standard for Go, Plus, and Pro users, while the smaller variant, GPT-Live-1 mini, will be available for free access. It is available on iOS, Android, and ChatGPT.com.
(vza)
