WeLM – WeChat's Self-Developed Large Language Model

1. What is WeLM

WeLM (WeChat Language Model) is a large language model self-developed by the WeChat team, now iterated to version V4. The model employs a highly sparse MoE (Mixture of Experts) architecture, with a parameter scale of approximately 80B to 130B. It is specifically designed for the WeChat ecosystem, emphasizing low cost and high efficiency. WeLM supports a 128K ultra-long context window, enabling it to natively access data from group chats, Moments, and other ecosystem components. It also utilizes Hidden Decoding technology to obscure the reasoning process and achieve rapid response. As the intelligent foundation for WeChat's AI Agent "Xiaowei," WeLM is not open to the public and is deeply integrated into intelligent interaction scenarios within WeChat.

Image source: Official article
Image source: official article

Technical positioning and domain: WeLM belongs to the domain of large language models in natural language processing, and is positioned as a closed-source AI engine dedicated to the WeChat ecosystem. Unlike open-source large models that focus on general capabilities, WeLM specializes in intelligent services for social scenarios. By deeply integrating with WeChat's ecosystem data, it achieves precise intent recognition and personalized responses, offering unique advantages in tasks such as group chat management, Moments interaction, and content-based Q&A.

Development background: WeLM was developed by the WeChat team based on its deep technical expertise in the social domain. It aims to provide low-cost, high-efficiency AI capabilities to WeChat's over 1 billion daily active users. With extensive experience in natural language processing and large-scale system optimization, the WeChat team launched WeLM to address challenges such as high data integration costs, large response latency, and difficulty in managing privacy risks when using general-purpose large models in WeChat scenarios, thereby promoting the practical deployment of the WeChat AI Agent "Xiaowei."

Core value: The core value of WeLM lies in its deep, native integration of AI capabilities with the WeChat ecosystem. It can provide intelligent summarization, response suggestions, and content-based Q&A services across scenarios such as group chats, Moments, and official accounts, without requiring users to manually upload data or switch applications. This significantly enhances users' information processing efficiency and social experience. Meanwhile, its extreme cost control allows WeChat to support massive user interactions at a low cost on a daily basis.

Technical features: WeLM uses a highly sparse MoE architecture, combined with optimization technologies such as GQA (Grouped Query Attention), KV-Mirror, and Multi-Token Prediction, to significantly reduce inference costs while maintaining model performance. Hidden Decoding technology enables low-latency responses by concealing the reasoning process, making it suitable for real-time communication scenarios. Its 128K ultra-long context window allows it to handle long text data such as group chat histories and Moments updates, achieving deep contextual understanding.

2. Key Features

• Native Ecosystem Data Integration: WeLM can natively access WeChat ecosystem data such as group chats, Moments, official accounts, video accounts, and favorites, without requiring users to manually upload or authorize external tools. This enables precise context awareness and personalized services. This capability significantly enhances WeLM's intent recognition accuracy in WeChat scenarios compared to general models that require manual data uploads.

• Smart Group Chat Summary: Automatically organize the massive volume of historical messages in group chats, accurately extract key information, core decisions, and action items, helping users quickly grasp the latest developments in the group and save reading time. This feature is especially useful in active communities, preventing users from missing important information.

• Moments Interaction Assistance: Based on the content of friends' Moments posts, combined with the intimacy level of the user's relationship with their friend and their historical interaction style, WeLM intelligently generates appropriate and personalized comments or reply suggestions, improving the quality and efficiency of social interactions.

• Script Generation and Response Suggestions: By integrating the current chat context, friend relationships, and the user's personal language style, WeLM recommends natural and appropriate response phrasing suitable for business communication, daily chats, and other scenarios, helping users express themselves more efficiently.

• In-depth Q&A on Ecosystem Content: WeLM provides in-depth interpretation of official account articles and video account content. Users can ask questions about specific details of an article or the theme of a video, and WeLM will instantly provide accurate answers, serving as a knowledge assistant within WeChat.

• Low-Cost High-Efficiency Inference: Utilizing a highly sparse MoE architecture, combined with technologies such as GQA, KV-Mirror, and Multi-Token Prediction, significantly reduces inference costs, allowing WeChat to support high-frequency calls for over 1 billion daily active users at a low cost, achieving widespread accessibility of AI services.

• Fast Response (Hidden Decoding): By employing Hidden Decoding technology to obscure the inference process, WeLM achieves low-latency responses while maintaining output quality, avoiding long waiting times for users and perfectly aligning with the real-time requirements of WeChat's instant messaging scenarios.

3. How to Use

1. Environment Requirements: Ensure that WeChat is updated to the latest version (iOS or Android), and you have a normally functioning WeChat account. Some features may be in internal testing and require waiting for official full release by WeChat.

2. Find the Entry Point: Search for "Xiaowei" on the WeChat chat list page, the top search bar, or the Discover page, and click to enter the AI Agent interface. You can also access it directly via the internal testing link pushed by WeChat.

3. Data Authorization: During the first use, authorize WeLM to access ecosystem data such as group chat history, Moments, and Favorites, as prompted. After authorization, WeLM can provide personalized responses and contextual understanding based on your personal data.

4. Initiate a Conversation: After clicking on "Xiaowei," directly input text or voice to present a task request, such as "Summarize recent group chat messages" or "Recommend a comment for Moments." The system will automatically process and return the results.

5. Multi-round Optimization: Based on Xiaowei's feedback, continuously refine your instructions through multiple rounds of conversation, such as requesting a more detailed summary or adjusting the response style, to improve the quality of the results.

6. Notes: WeLM is not open to the public and is only available for internal use within WeChat; data authorization can be managed at any time in the settings; some advanced features may require specific permissions or internal testing eligibility.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

For users who deeply utilize the WeChat ecosystem, WeLM is the only AI assistant that natively integrates data from group chats, Moments, and other features, offering an irreplaceable advantage in intelligent summarization and interactive assistance within social scenarios. If users require a general-purpose AI assistant that supports cross-platform and multi-scenario applications, ByteDance Doupai, Baidu Wenxin, and Alibaba Qwen provide broader application ecosystems and open interfaces, making them suitable for diverse needs such as office work, learning, and content creation. For users who prioritize multimodal capabilities, Doupai and Qwen offer more comprehensive support across modalities like image and audio. If enterprise developers need private deployment, they should opt for the API services of Wenxin or Qwen.

6. Editor's Summary

As a large language model developed internally by WeChat, WeLM demonstrates a unique approach in terms of technological innovation and practical application scenarios. Its highly sparse MoE architecture, combined with optimization techniques such as GQA, KV-Mirror, and Multi-Token Prediction, achieves exceptional cost control while maintaining model capabilities, providing a viable economic model for large-scale AI services. The Hidden Decoding technology enables low-latency responses by concealing the inference process, precisely meeting the stringent real-time requirements of instant messaging scenarios. More notably, WeLM's deep native integration with the WeChat ecosystem allows it to directly access contextual data from group chats, Moments, and other features without requiring users to manually upload information, achieving a level of accuracy in intent recognition and personalized responses that is difficult for general-purpose models to match. This design fully illustrates the advantages of "specialized small models" in vertical scenarios, offering a model for the integration of AI with super apps.

In terms of practical value, WeLM brings real efficiency improvements to WeChat users through its "micro-agent" functionality, such as group chat summaries, Moments interaction assistance, and content-based Q&A. Particularly in community environments where information overload is common, its summarization capabilities significantly reduce the user's information processing burden. However, WeLM's closed nature also limits the spread of its technological influence, as external developers and enterprises cannot directly access its capabilities. In the future, as the WeChat ecosystem continues to expand, WeLM is expected to play a role in more scenarios, and may even offer customized services to partners through limited API access or model distillation methods.

In terms of target users, WeLM is primarily aimed at regular WeChat users, especially those who need to manage a large number of group chats, frequently interact on Moments, or seek to enhance their social efficiency through AI. For technical researchers and developers, the WeLM technical blog provides practical experience in areas such as sparse MoE and long context extension, which holds reference value.

7. Application Scenarios

• Group Chat Intelligent Summarization: WeLM can automatically organize the massive historical messages in WeChat group chats, accurately extracting key information, core decisions, and action items, helping users quickly grasp the dynamics of the group chat and save reading time. Suitable for active communities such as work groups and interest groups.

• WeChat Moments Interaction Assistance: WeLM can generate appropriate, personalized comments or reply suggestions based on a friend's WeChat Moments content and the user's relationship with that friend, enhancing the quality and efficiency of social interactions.

• Ecosystem Content Q&A: WeLM can deeply interpret content from WeChat official accounts, video accounts, and other ecosystem sources, instantly answering user questions about article details, video topics, and more, becoming a knowledge assistant within WeChat.

• Personal Memory Retrieval: WeLM supports cross-chat records, bookmarks, likes history, and other personal data, helping users quickly and accurately locate and recall important past information, such as files and links from previous chats.

• Intelligent Script Generation: WeLM can combine the current chat context, friend relationships, and personal style to recommend appropriate and natural reply wording and communication scripts for users, suitable for scenarios such as business communication and daily chatting.

8. FAQ

Q: Is WeLM free to use?
A: Currently, WeLM is provided through the WeChat AI Agent "Xiaowei." It is free to use during both the functional beta testing and full release phases. However, there has been no official announcement regarding potential future charges.

Q: How can I get access to WeLM?
A: Update your WeChat to the latest version and search for "Xiaowei" within WeChat to find the entry point. Some features may be in beta testing and will be gradually made available by WeChat's official team.

Q: What languages does WeLM support?
A: WeLM is primarily optimized for Chinese scenarios and can handle content such as Chinese group chats and Moments. It also supports mixed Chinese and English input, although its English capabilities have not been officially detailed.

Q: Does using WeLM require authorizing personal data, and how is privacy and security ensured?
A: WeLM processes data within the closed-loop WeChat ecosystem. WeChat promises to comply with privacy protection policies, and users can manage their authorization scope in the settings. However, users should still be cautious when authorizing access to sensitive chat records.

Q: How does WeLM differ from general-purpose large models (such as GPT-4)?
A: WeLM is a specialized model for the WeChat ecosystem, deeply integrated with WeChat data, offering stronger advantages in intent recognition and personalized responses within social contexts. General-purpose models like GPT-4 excel in knowledge breadth, multimodal capabilities, and open ecosystems, but cannot natively access WeChat data.

Q: Does WeLM support multimodal input?
A: At present, WeLM mainly processes text, and there is no publicly available information indicating support for multimodal inputs such as images or voice. However, WeChat's "Xiaowei" does support voice input, which may be converted to text via speech recognition before being processed by WeLM.

9. Project Links

1. Official Documentation/Blog: https://welm.weixin.qq.com/ (WeLM technical blog, containing detailed information on architecture, training, etc.).