Hy ASR 3.0 Preview – The New Generation Speech Recognition Model from Tencent Hunyuan

1. What is Hy ASR 3.0 Preview

Hy ASR 3.0 Preview is a new generation speech recognition model launched by Tencent Hunyuan, built upon the Hy3 large language model and employing a Mixture of Experts (MoE) architecture. It integrates high-precision speech recognition with deep semantic understanding. The model supports Mandarin Chinese, English, Cantonese, and dialect recognition across ten major regional dialects. It features intelligent contextual error correction, hotword injection, and stable transcription capabilities in noisy environments. On open-source benchmark datasets, its word error rate (WER) is controlled around 3%, and it outperforms industry competitors comprehensively in Tencent's custom multi-dimensional evaluation sets.

Image source: Official article
Image source: official article

Technical positioning and domain: Hy ASR 3.0 Preview belongs to the field of end-to-end speech recognition based on large language models. Its unique feature lies in elevating the speech recognition system from the traditional "word-by-word transcription" to a semantic-level recognition that "understands context and delivers results with one click." By integrating the powerful language understanding capabilities of the Hy3 base model, this model not only performs well in standard recognition tasks, but also demonstrates significant advantages in complex scenarios requiring contextual semantic understanding, establishing a new paradigm for LLM-driven ASR.

Development background: This model was developed by the Tencent Hunyuan large model team, leveraging Tencent's years of technical accumulation in the AI field and its vast speech data resources spanning tens of millions of hours. Based on the Hy3 base model, the team independently developed an unsupervised speech encoder and addressed pain points in traditional ASR systems such as dialect recognition, noise interference, and long audio processing through multi-stage joint training and post-training enhancement. The goal is to provide more intelligent and robust speech recognition capabilities for internal products like Tencent Cloud and Tencent元宝, as well as external enterprise clients.

Core value: Hy ASR 3.0 Preview overcomes the limitations of traditional speech recognition models in semantic understanding, dialect coverage, and complex acoustic environments. Its core value lies in equipping the ASR system with contextual awareness through the use of a large language model, enabling it to automatically resolve homonym ambiguities and understand the user's true intent. Additionally, the hotword injection mechanism reduces the customization costs for recognizing industry-specific terminology, offering high-accuracy, low-latency speech transcription services for scenarios such as intelligent customer service, content creation, and office collaboration.

Technical features: The model employs a MoE architecture, combining an independently developed unsupervised speech encoder with the Hy3 base model. It is pre-trained on tens of millions of hours of multi-source speech data and undergoes post-training with Supervised Fine-Tuning (SFT) covering complex audio and contextual scenarios, as well as multi-stage reinforcement learning. This results in a comprehensive improvement in dialect comprehension, contextual modeling, and robustness in complex scenarios. The joint training strategy between the encoder and the LLM ensures deep alignment between acoustic and semantic representations, further enhancing recognition accuracy.

2. Key Features

• General Accurate Recognition: Supports Mandarin Chinese, English, Cantonese, and mixed Chinese-English speaking scenarios. Based on the high-precision acoustic representation of the Hy3 base model, it effectively reduces character errors, omissions, and error accumulation in long audio. It achieves approximately 3% WER on open-source benchmark datasets, reaching industry-leading performance in general scenarios.

• Contextual Semantic Understanding: Combines context information to accurately capture user intent, leveraging the semantic reasoning capabilities of large language models to intelligently correct homonyms and eliminate semantic ambiguity. It supports long-range dependency modeling, making transcription results more aligned with the speaker's true intent in scenarios such as meeting transcripts and multi-turn conversations.

• Hotword Injection Adaptation: Supports enhanced recognition of hotwords such as brand names, personal names, and industry-specific terminology. Users can inject business-specific vocabulary via API configuration without the need for retraining the model. This mechanism significantly reduces integration and maintenance costs in specialized domains (e.g., healthcare, finance, legal), improving the accuracy of key term recognition.

• Robust in Complex Environments: Specifically optimized for acoustic scenarios such as high noise, whispering, and murmuring, through robust noise training and model architecture improvements. It maintains stable recognition performance even under extremely low signal-to-noise ratios. This capability enables the model to be practically useful in environments such as in-vehicle systems, outdoor settings, and open office spaces.

• Broad Dialect Coverage: Supports 10 major dialect regions and over 20 sub-regions, including Northeastern Mandarin, Cantonese, Wu, Hokkien, Xiang, Gan, Hakka, and more. Through multi-stage dialect data injection training, the model can accurately recognize the phonetic characteristics of different dialects, significantly expanding the user base for speech recognition.

• Stable Processing of Long Audio: Designed for long-duration audio such as meeting recordings, podcasts, and interviews, the model employs context window management and progressive decoding strategies to effectively suppress error accumulation and maintain consistent transcription quality throughout. It also supports streaming output, reducing end-to-end latency and meeting real-time transcription requirements.

3. How to Use

1. Environment Requirements and Prerequisites: To use the Hy ASR 3.0 preview, you need a Tencent Cloud account and must have enabled the speech recognition service. It is recommended to use Linux or Windows operating systems, and the network environment must be able to access the Tencent Cloud API endpoint (cloud.tencent.com). If you are using Tencent元宝 (Tencent Yuanbao) for experience, you only need to install the latest version of the Tencent Yuanbao App (available for both iOS and Android), with no additional configuration required.

2. Accessing via Tencent Cloud API: Log in to the Tencent Cloud console, navigate to the speech recognition product page, create an application, and obtain the SecretId and SecretKey. In the API management interface, select the "Hy ASR 3.0 preview" engine and configure recognition parameters (such as language model, hotword list, etc.). Call the real-time speech recognition WebSocket interface or the recorded audio file recognition interface, and input audio data for recognition. Specific interface addresses and parameter descriptions can be found in the official Tencent Cloud documentation.

3. Free Experience in Tencent Yuanbao: Open the Tencent Yuanbao App, click the voice input button (press and speak), and you can directly use the Hy ASR 3.0 preview model for speech transcription. Tencent Yuanbao has deeply integrated this model, supporting dialect recognition, context correction, and transcription in complex environments. Users can experience the latest capabilities without any configuration.

4. Key Configuration Notes: When calling the API, you can specify the recognition language via parameters (e.g., Chinese, English, Cantonese, or automatic detection), and set a hotword list to improve the accuracy of specific vocabulary. For long audio tasks, it is recommended to use the recorded audio file recognition interface and set appropriate segmentation parameters (such as segment duration and maximum wait time) to balance latency and accuracy.

5. Notes and Best Practices: To ensure the best recognition results, it is recommended to record audio in a quiet environment with a sampling rate of no less than 16kHz and an encoding format of PCM or OPUS. For dialect recognition, you must explicitly specify the dialect type in the request (e.g., "cantonese"). When injecting hotwords, each hotword can be assigned a weight (0-100), and it is recommended to prioritize high-frequency and easily confused proper nouns. In production environments with high concurrency, it is advised to use the SDK provided by Tencent Cloud for integration and enable the automatic reconnection mechanism to ensure service stability.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

Selection Recommendations: If your business scenario requires extremely high accuracy in recognizing Chinese, Cantonese, and dialects, and also demands contextual semantic correction and hotword injection capabilities, Hy ASR 3.0 preview is currently the best option, especially suitable for applications such as intelligent customer service and meeting transcription that require deep contextual understanding. However, its cloud dependency and non-open-source nature should be considered in terms of cost. For teams requiring local deployment, strict data privacy, or full control over the model, Whisper large-v3 is more attractive due to its MIT open-source license and mature community ecosystem. However, its WER in Chinese and dialect scenarios is significantly higher than that of Hy ASR, and it lacks semantic understanding capabilities. If your enterprise is deeply integrated with the Alibaba Cloud ecosystem, Qwen-Audio 3.0 ASR offers convenient API integration and broad multilingual coverage, but its support for dialects and contextual understanding is currently inferior to Hy ASR. Doubao-Seed-ASR 2.0 performs well in general recognition, but its capabilities in dialects and context are relatively weak, making it more suitable for users with primary general transcription needs. Overall, Hy ASR 3.0 preview has established a significant technical advantage in the field of Chinese semantic-level recognition, but its lack of open-sourcing and local deployment options remain its shortcomings.

6. Editor's Summary

The Hy ASR 3.0 preview marks a significant step forward in the evolution of speech recognition technology toward "semantic understanding." Its technological innovations are primarily reflected in three areas: first, it employs the Hy3 large language model based on a MoE architecture, enabling the speech recognition system to leverage the deep semantic reasoning capabilities of LLMs and fundamentally altering the limitations of traditional ASR systems that rely solely on acoustic models; second, it features a self-developed unsupervised speech Encoder pre-trained on tens of millions of hours of multi-source data, achieving a balance between high-quality and high-efficiency acoustic representation extraction; third, a multi-stage post-training strategy (SFT + reinforcement learning) has been specifically optimized for long-tail scenarios such as context, dialects, and complex acoustics, enhancing the model's robustness in practical applications.

In terms of practical value, this model significantly improves the accuracy and readability of transcribed text in scenarios such as intelligent customer service, content creation, and office collaboration, reducing the cost of manual proofreading. It is particularly appealing to enterprise users who require dialect support and custom hotwords. The target audience mainly includes developers requiring high-precision speech transcription, content creators, customer service system integrators, and product teams aiming to lower user input barriers.

As a preview version, Hy ASR 3.0 already demonstrates technical capabilities that surpass current competitors. If the model continues to improve in areas such as open-sourcing, offline deployment solutions, and more detailed documentation in the future, it has the potential to become the mainstream choice in the field of Chinese speech recognition and drive the entire industry toward an LLM-driven ASR paradigm.

7. Application Scenarios

• Smart Customer Service: In both phone and online customer service scenarios, Hy ASR 3.0 preview enables precise recognition of brand names and business terminology through hotword injection. Combined with real-time contextual semantic understanding, it transcribes user speech into structured text that the customer service system can directly access, significantly improving service response efficiency and problem resolution speed.

• Content Creation: For long-form audio content production such as podcasts, meetings, and interviews, the model leverages its contextual semantic understanding to intelligently resolve homonym ambiguities. With one click, it outputs coherent, high-quality text, eliminating the need for manual proofreading and greatly reducing post-production editing costs while accelerating the content publishing process.

• Voice Search: Supports users in performing voice searches using dialects and accurately recognizes voice commands in noisy environments (e.g., malls, streets). This capability effectively lowers the input barrier for users with multiple languages and accents, enhancing the user experience and reach of search products.

• Office Collaboration: In real-time transcription scenarios for meeting minutes, the model maintains stable recognition even with low-volume inputs such as whispers or murmurs. Combined with contextual error correction, it generates highly accurate and logically coherent meeting records, helping teams achieve efficient communication and information retention.

• Smart Terminals: In IoT device interactions such as in-car navigation and smart home systems, the model is specifically optimized for high-noise acoustic environments, ensuring robust speech recognition in complex scenarios. Users can control devices via voice commands, eliminating the need for touch interaction, and enhancing driving safety and home convenience.

8. FAQ

Q: Is the Hy ASR 3.0 preview free?
A: Currently, the experience through the Tencent元宝 App is completely free, and users can directly use the voice input feature. For API calls via Tencent Cloud, it is billed based on usage. Please refer to the Tencent Cloud official website's speech recognition product pricing page for specific prices.

Q: Which dialects are supported?
A: It supports ten major dialect regions, including Northeastern Mandarin, Cantonese, Wu, Minnan, Xiang, Gan, Hakka, and more, as well as over 20 secondary sub-regions, covering China's main dialect areas. When making API calls, you need to specify the dialect type to achieve the best results.

Q: How to access the API?
A: Log in to the Tencent Cloud console, enable the speech recognition service, and obtain your SecretId and SecretKey. Call the real-time speech recognition WebSocket interface or the audio file recognition interface, selecting "Hy ASR 3.0 preview" as the engine parameter. Detailed API documentation can be found on the Tencent Cloud official website's speech recognition product page.

Q: How is context-aware intelligent correction implemented?
A: Based on the semantic understanding capabilities of the Hy3 large language model, the model combines contextual information during the recognition process and uses attention mechanisms to infer and correct homonyms and polysemous words, outputting the most contextually appropriate transcription results. This capability is particularly effective in scenarios such as meeting transcription and multi-turn dialogues.

Q: What are the advantages compared to open-source models like Whisper?
A: Hy ASR 3.0 preview achieves significantly lower WER in Chinese, Cantonese, and dialect recognition, and also features context correction and hotword injection capabilities. However, it requires cloud-based API usage and is less flexible than Whisper, which can be deployed locally. Whisper's advantage lies in being open-source and supporting offline operation, but its semantic understanding capabilities are relatively weaker.

Q: Is the model open-source?
A: Currently, the model weights of Hy ASR 3.0 preview are not open-source; it is only available through Tencent Cloud API and the Tencent元宝 App. Its base model, Hy3, has been open-sourced on Hugging Face, but the ASR-specific model is not yet available. The Tencent Hunyuan team has not yet announced an open-source plan.

9. Project Links

• Tencent Cloud Speech Recognition Product Page: https://cloud.tencent.com/product/asr (Hy ASR 3.0 preview API service entry)
• Tencent Yuanbao Official Website: https://yuanbao.tencent.com (Hy ASR 3.0 preview is integrated within the app and can be freely experienced)
• Base Model Hy3 (Hugging Face): https://huggingface.co/tencent/Hy3
• Official Tencent Hunyuan Organization (Hugging Face): https://huggingface.co/tencent