In-Depth Review of Gemini 3.8 Live – Google's Native Real-Time Speech Dialogue Model

1. What is Gemini 3.8 Live

Gemini 3.8 Live is a series of native real-time speech dialogue models launched by Google, which includes two variants: Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking. This series employs an end-to-end speech-to-speech architecture, eliminating the traditional cascaded pipeline of "speech recognition → text-based large model → speech synthesis" and directly processing input and output at the audio level. This significantly reduces interaction latency while preserving acoustic details such as tone and speaking rate. The model supports automatic detection and seamless switching across 97 languages, and features near-real-time visual understanding and asynchronous tool calling capabilities. It can process camera video and multi-step API tasks simultaneously without interrupting the conversation. According to the official data, the Extended Thinking variant achieved a score of 82.6 on the Artificial Analysis speech quality index, ranking first, and scored 97.7% on the Big Bench Audio benchmark. It is now being rolled out across multiple channels to developers, enterprise users, and general consumers.

Image source: Official article
Image source: official article

Technical positioning and domain: This model belongs to the field of speech interaction agents, focusing on end-to-end speech dialogue, real-time inference, and multimodal understanding. Its positioning goes beyond that of a simple voice assistant, aiming to serve as a speech agent capable of executing complex tasks. It can process visual input, tool calling, and deep reasoning in parallel, supporting multiple vertical scenarios such as customer service, office collaboration, and travel booking.

Development background: The model was developed by Google's DeepMind team and represents the latest iteration of the Gemini series in the direction of speech interaction. Previously, Google had launched Gemini 3.1 Flash Live to validate the feasibility of end-to-end speech architecture. This release introduces mechanisms such as asynchronous tool calling and configurable thinking budgets, optimizing the cost structure for large-scale deployment and continuing Google's technical roadmap in the field of multimodal large models.

Core value: This series addresses the issues of high latency, loss of acoustic details, and the inability of traditional cascaded speech solutions to execute multi-step complex tasks. By integrating visual understanding, tool calling, and speech output within a single model, developers can build speech agents that are "able to listen and see, and speak while acting" without needing to stitch together multiple models. Additionally, the per-minute billing pricing model lowers the threshold for large-scale deployment.

2. Key Features

• End-to-end voice conversation: Directly processes audio input and output using an end-to-end native audio modeling architecture, without the need for text conversion, achieving near real-time response. This approach preserves acoustic details such as tone, speaking rate, and pauses, making the conversation experience closer to human natural communication, and ranks second in user preference on the Speech Agent Arena.

• Asynchronous tool calling: The model can perform API calls and function operations in parallel in the background while continuously delivering voice responses to the user. During multi-step tasks, the conversation remains uninterrupted, allowing users to continue interacting without waiting for the tool execution to complete, making it ideal for scenarios requiring integration with multiple external systems such as ticket booking, inquiries, and business processing.

• Real-time visual understanding: Supports near real-time processing of camera video, up to approximately 1 frame per second, providing dynamic visual context for the conversation. Users can operate physical objects while asking questions, and the model can provide targeted answers based on the visual content, suitable for scenarios such as troubleshooting, onboarding guidance, and on-site operations where visual input is required to answer.

• Automatic language switching for 97 languages: The model can automatically detect and seamlessly switch between languages during the conversation, without requiring manual settings from the user. This capability covers the world's major languages, enabling a single model to serve multilingual user groups and reducing the complexity of deploying cross-border services.

• High-precision alphanumeric parsing: Specifically optimized for technical string data such as confirmation codes, claim numbers, and order IDs, the model can accurately identify and repeat continuous combinations of numbers and letters, meeting the strict requirements for information accuracy in industries such as banking, insurance, and logistics.

• Incremental content updates: Seamlessly integrates real-time audio with structured data, dynamically returning context-aware responses during the conversation. The model can adjust its answers in real-time based on the latest visual information or results returned from tools, rather than outputting fixed pre-set scripts.

• Configurable thinking budget: The Extended Thinking variant allows developers to adjust the intensity of background reasoning as needed. It can allocate more computational resources for multi-step thinking on complex tasks, while reducing latency and cost in simple conversations, achieving a flexible balance between reasoning resources and response speed.

• End-to-end content watermarking: All generated audio includes an embedded SynthID invisible watermark, directly woven into the audio signal itself. The watermark remains detectable even after editing or compression, used to prevent voice forgery and the spread of false information.

3. How to Use

1. Developer Integration Methods: Developers can use Gemini 3.8 Live and the Extended Thinking variant through the Gemini API and Google AI Studio. First, register for a Google AI developer account and obtain an API key, then configure the voice input and output parameters according to the official documentation.

2. Enterprise Access: Gemini 3.8 Live is available in the private preview of Gemini Enterprise. Enterprises can apply to join the preview program. The Extended Thinking variant is set to launch on Gemini Enterprise for Customer Experience and Google Workspace enterprise customers. Enterprise users can consult their account manager for specific integration procedures.

3. General User Experience: All users can experience the voice conversation capabilities of Gemini 3.8 Live in Search Live. The Extended Thinking variant is available in Gemini Live, and Google AI Pro and Ultra subscribers can use it in Workspace Docs. All Google AI subscribers can experience it in Gmail and Keep.

4. Model Configuration and Invocation: Developers can choose between the standard version or the Extended Thinking variant based on their business scenarios, and configure the thinking intensity level via API parameters. Audio input is priced at $0.005 per minute, and output at $0.018 per minute. The Extended Thinking variant has the same pricing, allowing developers to flexibly adjust their model selection based on their budget.

5. Multilingual and Visual Capabilities: The model supports automatic detection of 97 languages by default, requiring no additional configuration. To enable visual understanding, developers must pass the camera video data stream in API requests and set the frame rate parameter (up to approximately 1 frame per second). Once configured, the model can integrate visual context into the conversation.

6. Notes and Best Practices: Developers are advised to combine the SynthID watermark detection mechanism with generated audio for source verification in sensitive scenarios such as finance and healthcare. For tasks requiring multiple-step tool calling, it is recommended to design asynchronous calling logic appropriately to avoid excessive use of background inference resources, which could affect response speed.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

From a technical perspective, both Gemini 3.8 Live and OpenAI gpt-realtime-2.1 adopt end-to-end audio modeling architectures, representing the cutting-edge direction in speech interaction models. However, they differ significantly in pricing strategies, visual capabilities, and audio watermarking. Gemini 3.8 Live's per-minute billing model is more favorable for large-scale real-time conversations, offering stronger cost control. Its asynchronous tool calling and "think while speaking" mechanism provide a smoother user experience when handling multi-step tasks. In contrast, OpenAI gpt-realtime-2.1 offers a higher single-inference limit under its token-based billing model, making it more suitable for high-value, short-duration professional consultation scenarios.

For applications that are cost-sensitive and require long-duration continuous conversations, such as customer service and assistant tools, Gemini 3.8 Live's cost advantages and asynchronous tool calling capabilities are clear positive factors. For expert systems requiring deep, complex reasoning and where the value of a single task is high, a trade-off can be made based on the evaluation of gpt-realtime-2.1's inference capabilities versus Gemini 3.8 Live's cost efficiency. Traditional cascaded solutions are mainly suitable for existing systems that already have mature ASR/TTS components and do not wish to migrate their current technology stack.

6. Editor's Summary

Gemini 3.8 Live provides a clear technical answer in the architectural evolution of speech interaction models: end-to-end speech-to-speech processing is no longer an experimental exploration, but a mature solution capable of supporting large-scale commercial deployment. According to benchmark data, the Artificial Analysis speech quality index of 82.6 points and a 97.7% score on the Big Bench Audio test demonstrate that it has reached the top level of the current industry in core metrics such as speech naturalness and understanding accuracy. The introduction of asynchronous tool calling upgrades the speech model from a "dialogue engine" to a "task execution engine," a breakthrough in capability that has a substantial impact on the application boundaries of speech agents.

In terms of practical value, the per-minute pricing strategy brings the cost of real-time speech conversations down to the range of $0.005–$0.018 per session. Compared to OpenAI's pricing of approximately $0.06–$0.11 per minute, this provides a viable economic path for large-scale, long-duration speech interaction scenarios. The automatic language switching among 97 languages and visual understanding capabilities further expand the range of applicable scenarios, allowing a single model to serve global multilingual users and handle practical tasks that require visual context. The SynthID watermark mechanism adds content traceability to generated audio, which holds practical significance in the context of rising risks of speech forgery.

In terms of target users, this model offers significant value to three categories: first, developers and product teams with large-scale speech interaction needs, who can quickly build speech agents using low-cost APIs; second, enterprises in industries such as finance, customer service, and transportation that require multi-step tool calling, as the asynchronous tool calling capability can be directly integrated into business processes; third, Workspace users who already have a foundation in the Google ecosystem, who can directly experience the model in scenarios such as Docs, Gmail, and Keep. When selecting this model, it is necessary to consider the business scenario comprehensively: if the goal is to achieve cost-controlled large-scale real-time conversations and multi-task parallel processing, Gemini 3.8 Live is a solution worth prioritizing under current technological conditions; if deep complex reasoning is required and tasks are infrequent but high-value per instance, a technical selection decision can be made after comparing the reasoning limits of other models.

7. Application Scenarios

• Onboarding Guidance for Employees: New employees can ask AI voice questions in real-time through the camera feed while operating equipment or systems. The model instantly answers questions about operational procedures and system access points by combining visual context, eliminating the need to interrupt work to consult documentation and significantly shortening the onboarding period. This is applicable to scenarios such as manufacturing equipment operation and warehouse system training.

• Intelligent Customer Service and Banking Operations: Users can voice their business needs during a voice conversation. The model asynchronously calls functions in the background to retrieve account information and perform multi-step operations, while simultaneously broadcasting the processing progress via voice. Users do not need to wait for silent processing, achieving an interactive experience close to that of human customer service agents, which corresponds to the typical scenarios in the τ-Voice-banking benchmark test.

• Travel and Booking Assistant: Users can voice travel requests in natural language that include multiple conditions such as date, destination, and budget. The model asynchronously calls APIs for ticket booking and hotel reservations in the background, maintaining natural conversation during the query process and promptly providing available options. The model handles the complexity of multi-condition queries, eliminating the need for users to input information step-by-step.

• Programming and Design Collaboration: Developers can verbally describe functional requirements while looking at sketches or interface prototypes. The Extended Thinking variant performs multi-step reasoning in the background, generating usable React component code while simultaneously explaining the implementation logic to the user. The model's visual understanding capability allows it to interpret sketch content, while voice output explains the code logic, forming a complete collaborative workflow of "description → generation → explanation."

• Real-time Fault Diagnosis Guidance: Users can verbally describe hardware or software issues in Search Live. The model provides step-by-step diagnostic instructions by combining the camera feed. After users perform operations guided by voice instructions, the model adjusts the diagnostic direction based on the new visual feedback. This mode is suitable for scenarios requiring real-time interactive guidance, such as IT operations and home appliance repair.

8. FAQ

Q: What is the difference between Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking?
A: The standard version is designed for large-scale deployment, focusing on dialogue fluency, visual understanding, and cost-effectiveness, making it suitable for most real-time voice interaction scenarios. The Extended Thinking version enhances multi-step reasoning capabilities, allowing the model to perform deep thinking in the background while providing voice updates on progress, making it ideal for tasks requiring complex logical analysis. However, the depth of reasoning can be configured, and developers can adjust it as needed.

Q: How can developers obtain access to Gemini 3.8 Live?
A: Developers can use Gemini 3.8 Live directly via the Gemini API and Google AI Studio. Enterprise users must apply for access through the Gemini Enterprise private preview. Regular users can experience the standard version in Search Live and the Extended Thinking version in Gemini Live. Subscribers to Google AI Pro and Ultra can also use Gemini 3.8 Live in Workspace, Gmail, and Keep.

Q: What languages does Gemini 3.8 Live support?
A: It supports automatic detection and seamless switching among 97 languages. No manual language setting is required during conversations. The model automatically identifies the language currently used by the user and responds accordingly. It also allows for real-time language switching during the conversation, making it suitable for international scenarios involving multilingual interactions.

Q: Can Gemini 3.8 Live process visual information?
A: Yes. The model supports near-real-time processing of camera video, with a maximum of approximately 1 frame per second. It can combine visual context to answer user questions. Developers can enable this capability by passing the camera video data stream and setting the frame rate parameter in the API request. This is applicable to scenarios such as troubleshooting and on-site guidance.

Q: Does Gemini 3.8 Live's audio output include a watermark?
A: Yes. All generated audio includes an embedded SynthID invisible watermark, which is directly woven into the audio signal itself. Even after editing or compression, the watermark can still be detected. This mechanism is used to prevent voice forgery and the spread of false information. Enterprises can use detection tools in sensitive scenarios to trace and verify the authenticity of the audio.

Q: How is Gemini 3.8 Live billed compared to OpenAI gpt-realtime-2.1?
A: It is billed on a per-minute basis, with audio input costing $0.005 per minute and audio output costing $0.018 per minute. The Extended Thinking variant uses the same pricing model, and developers do not need to pay extra for deep reasoning capabilities. This pricing model is particularly favorable for long-duration continuous conversation scenarios.

9. Project Links

• Gemini Official Website: https://gemini.google/
• Gemini Live Experience Entry: https://gemini.google/overview/gemini-live/
• Google Official Blog (Model Release Notes): https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-8-live-gemini-3-8-live-extended-thinking/
• Google DeepMind Gemini Page: https://deepmind.google/models/gemini/
• Gemini API Documentation: https://ai.google.dev/gemini-api/docs
• Official GitHub Organization (google-gemini): https://github.com/google-gemini