Hy-MT2-1.8B Review: The Ultimate Quantization Practice of Tencent HunYuan's On-Device Translation Large Model

1. What is Hy-MT2-1.8B

Hy-MT2-1.8B is an on-device translation large model launched by Tencent HunYuan, supporting bidirectional translation across 33 languages with only 1.8B parameters. Its translation quality surpasses commercial APIs such as Microsoft and DouPao on benchmarks like FLORES-200. Through two extreme quantization solutions—2-bit (574MB) and 1.25-bit (440MB)—the model can run smoothly locally on mobile phones and PCs. It has already been deployed in Bilibili's live streaming danmaku real-time translation scenario, achieving high-quality on-device translation capabilities with low cost and high privacy.

Technical Positioning and Domain: Hy-MT2-1.8B belongs to the machine translation direction in the field of natural language processing. However, unlike traditional cloud-based translation services, its core positioning is on-device translation. This model fully deploys translation capabilities onto terminal devices using ultra-low-bit quantization technology, enabling high-quality cross-language translation services even in offline environments, thus filling the gap in high-performance offline translation on mobile devices.

Development Background: This model was developed by the Tencent HunYuan team, driven by the massive multilingual scenario demands within Tencent's social and content ecosystems. The team has previously accumulated rich experience in model compression and inference optimization within the HunYuan large model series. Hy-MT2-1.8B represents a significant technical achievement in the on-device AI direction. In collaboration with Intel's team, the model also completed operator adaptation for the x86 platform, demonstrating a clear technical path of industry-academia-research synergy.

Core Value: This model addresses the core contradiction of deploying high-quality translation models on on-device platforms under limited computational and storage resources. By leveraging its self-developed Sherry sparse quantization scheme, the model reduces its size to 440MB while maintaining minimal degradation in translation quality, enabling standard mobile phones and PCs to run the translation model continuously. This completely eliminates the latency, privacy, and cost issues associated with cloud-based translation.

Technical Features: The model employs two self-developed technologies: Stretch Elastic Quantization (SEQ) and 1.25-bit fine-grained sparse quantization (Sherry). The former quantizes parameters into four discrete values, while the latter achieves ultra-low-bit representation through a sparse strategy of "retaining 3 out of every 4 parameters as ±1 and setting 1 to zero." Combined with the STQ kernel specifically designed for CPUs and optimizations using Intel's VNNI instruction set, the model achieves a throughput improvement of 2.7 to 5 times on mainstream Core processors.

2. Key Features

• Bidirectional Translation Across 33 Languages: Supports major international languages such as Chinese, English, Japanese, and Korean, as well as minority languages and dialects. Direct translation between any languages is possible without the need for intermediate languages, reducing information loss caused by secondary translation and meeting the multilingual communication needs in global scenarios.

• On-Device Local Inference: The model is compressed to 440~574MB using 2-bit and 1.25-bit quantization techniques, enabling it to run locally on ARM architecture smartphones and x86 architecture PCs. No internet connection is required to complete translation tasks. It can run smoothly on regular mid-range smartphones and remain resident in the background to respond at any time.

• Low-latency Real-time Translation: Designed for high-real-time scenarios such as live stream chat, the response time for a single translation is controlled within 500~800ms, keeping up with the pace of live broadcasts. The model demonstrates strong understanding of non-standard texts such as internet slang and meme culture, which has been validated in real-world applications.

• Privacy and Security Protection: The entire translation process is completed locally on the device, with no need to upload text data to the cloud, eliminating the risk of data leakage at the physical level. This feature is of significant value for translation tasks involving commercial secrets or personal privacy.

• Cross-platform Architecture Compatibility: Supports both ARM mobile devices and x86 PCs. The Intel team has optimized vectorization, weight reordering, and VNNI instruction fusion for low-bit formats, increasing token throughput on Core processors by approximately 2.7 times (Q2_0C) and 5 times (STQ), extending the ultra-low bit solution from mobile devices to PCs and edge devices.

• Open Source and Open Ecosystem: The model weights and inference framework code are fully open-sourced, allowing developers to freely download, perform secondary development, and deploy for commercial use. Combined with the AngelSlim inference framework, it lowers the integration threshold for on-device translation capabilities, facilitating continuous enrichment of the community ecosystem.

3. How to Use

1. Environment Requirements: A computer or smartphone equipped with a mainstream CPU is required, with no restrictions on the operating system (supports Windows, Linux, Android, iOS, etc.). After quantization, the model is as small as 440MB, which can be accommodated with standard storage space; inference is performed on the CPU and does not require a dedicated GPU. For x86 devices, it is recommended to use Intel Core processors that support the VNNI instruction set to achieve optimal performance.

2. Download Model Weights: Visit the Hugging Face repository (huggingface.co/collections/AngelSlim/hy-low-bit-model) and select the 2-bit (574MB) or 1.25-bit (440MB) quantized version based on your needs. The former offers slightly better translation quality, while the latter has a smaller size and faster inference speed. Choose between the two based on your device's performance and accuracy requirements.

3. Get the Inference Framework: Clone the AngelSlim repository from GitHub (github.com/tencent/AngelSlim), which includes the inference kernel and loading logic for extremely low-bit quantization. After cloning, follow the instructions in the README to compile the framework. The framework supports CPU inference and does not require additional installation of GPU dependencies such as CUDA.

4. Load and Inference: Load the downloaded model weights using the AngelSlim framework and perform translation inference on the local CPU. For x86 devices, enable the Intel-optimized VNNI instruction set kernel to achieve the best performance; for ARM devices, use the STQ kernel optimized for mobile devices. Developers can refer to the example code in the repository to quickly integrate it into their applications.

5. Best Practices Recommendations: For ultra-low-latency scenarios such as live streaming chat messages, it is recommended to use the 1.25-bit version and enable batch processing mode. For quality-priority scenarios such as document translation, the 2-bit version is recommended. Before deployment, run benchmark tests on the target device and adjust the model version and thread configuration based on the actual throughput performance.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

To evaluate the industry positioning of Hy-MT2-1.8B, we selected three representative solutions for comparison: Google Translate Gemma-4B, Microsoft Translator API, and DouPao Translation.

Selection Recommendations: For privacy-sensitive scenarios (such as translating medical records, legal documents, or business secrets) and situations requiring offline operation (such as cross-border communication in environments without internet access, or translation on embedded devices), Hy-MT2-1.8B's on-device architecture and open-source nature make it a suitable choice. Especially in scenarios requiring low-latency real-time translation, such as live streaming and gaming, its response speed of 500–800ms and local inference capabilities offer clear advantages.

For users requiring support for a vast number of languages or high-quality translations in specialized fields, cloud-based services like Microsoft Translator API remain more reliable options, albeit at the cost of data being uploaded to the cloud and pay-per-use billing. Google Translate Gemma-4B is suitable for developers with sufficient device performance and a need for larger model capacity, but it requires handling quantization and compression issues independently, resulting in higher deployment costs. Overall, Hy-MT2-1.8B has established a good balance between size, quality, and speed in the on-device translation domain.

6. Editor's Summary

Hy-MT2-1.8B demonstrates notable technological innovation in the field of on-device translation models. Its core value lies in the combined application of two proprietary quantization techniques: Stretch Elastic Quantization (SEQ) compresses parameters into four discrete values, while Sherry sparse quantization further reduces the average bit count to 1.25-bit through a "fine-grained sparsity" strategy, enabling the 1.8B parameter model to be compressed to 440MB while maintaining acceptable translation quality. This technical approach has been accepted for oral presentation at ACL 2026, indicating academic recognition of its innovativeness. Particularly noteworthy is that the model has not stopped at the theoretical level of the research paper, but has been deployed on x86 operators through collaboration with Intel, extending extremely low-bit inference from mobile devices to PCs and edge devices, forming a complete technical implementation path.

In terms of practical value, the model has already been deployed and validated in real-world scenarios on Bilibili's live streaming danmu system, achieving a response speed of 500 to 800ms, which meets the demand for high-frequency real-time translation. The privacy protection features brought by its on-device inference architecture are particularly meaningful in today's context of increasing emphasis on data security. For social application developers, companies expanding overseas, game localization teams, and users concerned about privacy, Hy-MT2-1.8B provides a low-cost, highly controllable translation solution.

In terms of target users, the model is primarily aimed at three categories: first, developers who need to integrate on-device translation, who can quickly incorporate it using open-source weights and the AngelSlim framework; second, enterprise users who prioritize data privacy, as local deployment can help avoid compliance risks associated with data transmission across borders; third, scenario operators with strict real-time requirements, such as live streaming platforms and game developers. The model's limitations include a narrower language coverage compared to cloud-based services and its inference performance depending on the hardware capabilities of the terminal device.

In terms of development potential, the on-device AI approach represented by Hy-MT2-1.8B aligns with industry trends. As terminal computing power continues to improve and quantization techniques evolve, performance in terms of quality and speed for on-device models will continue to improve. The quantization methodology of this model can also be applied to other NLP tasks, and its technical value is expected to extend beyond the translation scenario itself, providing a reference model for the large-scale deployment of on-device AI.

7. Application Scenarios

• Live Stream Chat Real-time Translation: Already deployed in Bilibili's live streaming environment, the model performs real-time translation of chat messages locally, with a single response latency of 500~800ms, keeping up with the live stream pace. It demonstrates strong understanding of non-standard expressions such as internet slang and meme culture, enabling cross-language audience interaction and enhancing the international atmosphere in live streams.

• Offline Instant Messaging: Integrated into social apps, it enables real-time mutual translation for cross-border chats even in offline or low-network conditions. Translation is performed locally, ensuring that conversation content remains on the device and safeguarding communication privacy. This scenario provides clear value for users such as business travelers and international students who often face unstable network environments.

• Mobile Web and Document Translation: Embedded as a browser plugin or system-level translation service on mobile devices, it translates foreign language web pages and PDF documents locally. No need to upload documents to the cloud, making it ideal for handling sensitive content such as commercial contracts and academic papers, while also avoiding the waiting time associated with cloud-based translation services.

• Real-time In-game Localization: When embedded in mobile or PC game clients, it can translate chat messages, task text, and UI interfaces from overseas servers in real-time. Running on the device side, it does not add a burden to the server and is unaffected by network fluctuations, providing players with a seamless cross-server gaming experience.

• Cross-border E-commerce Content Translation: On the merchant side, it enables batch translation of product titles, detail pages, and customer service scripts locally, supporting multi-language store operations. Translation data remains on the local device, avoiding the need for content to be processed through third-party platforms, while also reducing per-use API costs for cloud translation, making it suitable for high-frequency, large-volume translation needs.

8. FAQ

Q: How can a model with 1.8B parameters achieve a size of only 440MB?
A: The model employs a two-tier compression strategy. First, it uses Stretchable Elastic Quantization (SEQ) to quantize parameters into four discrete values: {-1.5, -0.5, 0.5, 1.5}, achieving 2-bit representation. Then, it applies Sherry sparse quantization, retaining 3 parameters as ±1 and setting 1 to 0 out of every 4 parameters, requiring only 1.25 bits on average per parameter. Combined with Quantization-Aware Distillation (QAD) technology, the model maintains nearly lossless translation quality while significantly reducing its size.

Q: How to choose between the 2-bit (574MB) and 1.25-bit (440MB) versions?
A: The core difference between the two versions lies in the trade-off between size and quality. The 2-bit version retains more information and offers slightly higher translation quality, making it suitable for scenarios where quality is prioritized, such as document translation. The 1.25-bit version has a smaller size and faster inference speed, ideal for latency-sensitive applications like live streaming subtitles. It is recommended to test both versions on the target device to determine which performs best for your specific use case.

Q: What languages does the model support? Does it include minority languages?
A: The model natively supports bidirectional translation between 33 languages, covering major international languages such as Chinese, English, Japanese, and Korean, as well as five minority languages/dialects. Direct translation is possible between any of the supported languages without the need for intermediate languages like English, reducing information loss from secondary translations.

Q: Can it run smoothly on low-end phones?
A: After quantization, the model's minimum size is only 440MB, and inference is performed on the CPU without relying on a dedicated GPU. It can run on mid-range phones, but inference speed is affected by device computational power. For low-end devices, it is recommended to use the 1.25-bit version and close other high-load background applications. High-end devices can achieve near real-time translation performance.

Q: What are the advantages and disadvantages of on-device translation compared to cloud-based translation APIs?
A: The advantages of on-device translation include keeping data local, ensuring privacy and security, eliminating network latency, and avoiding pay-per-use costs. The disadvantages are that it has a smaller language coverage compared to cloud services (e.g., Microsoft supports over 100 languages), and translation quality is constrained by terminal computational power, with less capability for handling complex texts than cloud-based large models. Both approaches are suitable for different scenarios and can complement each other.

9. Project Links

1. GitHub Repository: https://github.com/Tencent-Hunyuan/Hy-MT2
2. Inference Framework AngelSlim: https://github.com/tencent/AngelSlim
3. Hugging Face Model Library: https://huggingface.co/collections/AngelSlim/hy-low-bit-model