Abstract:
OpenAI recently announced that GPT-Live-1, currently used for ChatGPT voice function, will be officially opened to developers. Developers can now integrate this voice model into their own applications and business processes through APIs to build full-duplex voice assistants that can listen to users in real time and respond at the same time. The voice front-end price of this model is US$0.05 per minute, and the model used in the background for complex reasoning needs to be billed separately according to the corresponding model price.
GPT-Live-1 is a new generation of real-time speech model launched by OpenAI this year. Its biggest feature is its full-duplex architecture. Traditional voice AI usually needs to complete the three steps of speech recognition, language model inference and speech synthesis in sequence. Only after the user finishes speaking can the system start processing and generate answers. GPT-Live-1, on the other hand, can generate speech output while listening to the user speak, so it can handle interruptions, pauses, echoes, and rapid back-and-forth conversations more naturally.
This means that users do not have to wait for the AI to finish speaking completely before continuing to speak. If the user changes his mind, adds information, or directly interrupts, GPT-Live-1 can adjust its behavior in time according to the ongoing conversation. The model can also tell when to keep talking, when to pause, when to keep listening, and when to call on tools.
OpenAI stated that this architecture can significantly reduce the delay in voice interaction and solve the connection problems that are prone to occur in actual conversations in the traditional "speech recognition + large language model + speech synthesis" pipeline. Since GPT-Live-1 itself handles both input and output audio, developers no longer need to coordinate complex switching between multiple independent models.
In the test published by OpenAI, the performance of GPT-Live-1 in the Full Duplex Bench test was 30 percentage points higher than that of GPT-Realtime-2.1, especially in terms of turn-taking delay and interactive behavior. When paired with GPT-6 Astra at medium inference intensity, GPT-Live-1 also achieved first place in the Tau3 test. Tau3 is mainly used to evaluate the ability of voice agents to complete end-to-end tasks.
Another important feature of GPT-Live-1 is that it is not responsible for all complex reasoning work. OpenAI separates the speech model responsible for real-time listening, speaking and interaction from the background model responsible for deep reasoning. When users ask questions that require search, complex analysis, or long tasks, GPT-Live-1 can hand over these tasks to the background model while continuing to maintain natural voice communication with the user.
Developers are therefore free to choose backend models based on specific business needs. For example, for a large number of simple tasks such as reservations and order updates, faster and lower-cost models can be used; for complex customer problems, they can be handled by stronger inference models. This architecture allows developers to find a balance between responsiveness, reasoning capabilities and cost of use.
OpenAI stated that this decoupled design also allows GPT-Live-1 to continue to communicate with users while performing tasks in the background, instead of letting users face a long period of silent waiting. The model can continue a natural conversation while completing more complex work in the background, then bring the results back to the current conversation when the task is completed.
Practical applications are already starting to emerge. Early examples demonstrated by OpenAI include voice services such as Yelp Host and Hatch, and language learning platform Speak. Speak found in early evaluations that compared with previous turn-based speech systems, GPT-Live-1 can give language learners more time to think, reducing the number of system proactive interruptions to users by nearly 80%.
For enterprises, this capability is especially suitable for customer service, telephone customer service, reservations, and other scenarios that require continuous voice interaction. GPT-Live-1 natively supports phone scenarios, so developers can use it to build full-duplex voice agents that can answer and handle phone calls, such as restaurant reservations, customer support, and other businesses that require real-time communication.
GPT-Live-1 also provides native automatic speech recognition transcribed text and model response text, and enhances the understanding of alphanumeric combinations, while supporting keyword bias functions. Although it is not a turn-based model in the traditional sense, it still supports turn detection natively. Therefore, if developers need to clearly control when the user finishes speaking and when the system starts answering, they can still design their applications around clear dialogue turns.
GPT-Live-1 has also been optimized for scenes with noisy background environments. The model can better distinguish between user speech, background noise and silence. It will not frequently interrupt conversations because of sounds in the surrounding environment, nor will it constantly remind users when they are just thinking briefly. This capability is especially important for phone calls, customer service, and mobile voice assistants in real-world environments.
OpenAI has also expanded the sound selection available for GPT-Live-1. Compared to the relatively limited number of real-time voices previously available, the new version offers more diverse voices and covers a richer range of accents, dialects and languages. OpenAI says it will continue to add available voices and languages in the coming months.
In terms of model deployment, developers do not need to hand over all tasks to GPT-Live-1. OpenAI hopes to use it as a front-end responsible for real-time voice interaction, taking on more complex work through background models and agent frameworks. This approach also allows developers to combine different models according to the actual needs of different tasks.
For example, developers can let GPT-Live-1 be responsible for receiving user voice requests, maintaining natural conversations, and handling interruptions, and then hand over questions that require complex reasoning to the background model; after the background model completes the work, GPT-Live-1 converts the results into natural voice answers. This mechanism also applies to applications that need to call external tools.
OpenAI also demonstrated how to combine GPT-Live-1 with tools such as Codex. Developers can let the speech model receive the tasks proposed by the user, then transfer the relevant context to background tools such as Codex for processing, and then return the processing results to GPT-Live-1, which will continue to communicate with the user via voice.
In terms of price, GPT-Live-1 is now officially available through the OpenAI API, and the voice front-end charges for $0.05 per minute. It’s important to note that this is only the cost of the real-time voice tier. If the developer equips GPT-Live-1 with a background inference model, the cost of calling the background model needs to be calculated separately based on the corresponding model pricing.
This means that for business applications that require a large number of voice calls, the actual cost is not just $0.05 per minute, but is composed of the voice connection time and the inference consumption of the background model. However, developers can control the overall cost according to the complexity of the task by choosing different back-end models.
The opening of GPT-Live-1 also means that OpenAI is further extending the core technology behind the ChatGPT voice mode from consumer products to the developer ecosystem. Previously, GPT-Live-1 mainly existed as the voice interaction capability of ChatGPT. Now developers can directly use similar technologies to build their own voice applications and agents.
From a technical perspective, OpenAI hopes that GPT-Live-1 will not only solve "enabling AI to speak", but enable AI to truly participate in continuous, real-time, and interruptible natural conversations. By splitting listening, speaking, real-time interaction and deep reasoning, OpenAI attempts to achieve lower voice latency, a more natural conversation experience and stronger background task processing capabilities at the same time.
As voice agents gradually shift from simple voice question and answer to complex tasks such as telephone customer service, reservations, language education, business processing, and the ability to autonomously call tools to complete tasks, the importance of real-time speech models is also increasing. The open API of GPT-Live-1 means that developers can now directly embed the current generation of real-time voice technology used by ChatGPT into their own products, and the voice layer price of US$0.05 per minute also allows this capability to officially enter the commercial application stage.
Comments