Qwen3.8-Flash – A Multimodal MoE Model Launched by Tongyi Qianwen

1. What is Qwen3.8-Flash

Qwen3.8-Flash is a multimodal Mixture-of-Experts (MoE) model launched by Tongyi Qianwen from Alibaba Cloud, with a total parameter count of 125B, activating only 6B parameters per token, and natively supporting a context length of up to 262K. The model incorporates four major architectural innovations: GDN+QSA hybrid attention, gated residual networks, N-gram embeddings, and the Muon optimizer. Its training cost is only 1/9 of that of Qwen3.7-Plus, while offering enhanced capabilities in coding and office tasks. This model is designed for scenarios such as long-context processing, code intelligence, office automation, and multimodal understanding, and provides two usage options: open-source weights and low-cost API, balancing performance and engineering efficiency.

Image source: Official article
Image source: official article

Technical positioning and domain: Belongs to the field of multimodal large language models, utilizing a Mixture-of-Experts (MoE) sparse activation architecture to significantly reduce computational costs per inference while maintaining flagship-level performance. Positioned between general-purpose foundational models and vertical task models, it focuses on production-grade applications such as code generation, software engineering, long-range office tasks, and multimodal agent interactions, representing an important strategic move by Tongyi Qianwen at the intersection of efficient inference and complex task execution.

Development background: Developed by the Tongyi Qianwen team at Alibaba Cloud, leveraging the long-term accumulation of the Tongyi series of models in Chinese and multilingual scenarios. The team has deep technical expertise in MoE architecture, long-context modeling, and multimodal alignment. The Qwen3.8-Flash model features multiple forward-looking architectural designs and is officially positioned as an early validation version of the Qwen4 series architecture, reflecting the team's exploration of the next-generation model technology roadmap.

Core value: Addresses the balance challenge between long-sequence processing, agent task execution, and deployment costs in large models. Through a sparse design that activates only 6B parameters per token, inference costs are kept at a low level. The native 262K context length, combined with the ability to scale up to 1M, enables end-to-end analysis of entire technical manuals or large codebases. The API pricing is 1 RMB per million input tokens and 3 RMB per million output tokens, significantly lowering the threshold for enterprise-level applications.

Technical features: Four architectural innovations form the basis of its differentiated capabilities — the GDN+QSA hybrid attention mechanism achieves an 8.6x increase in Prefill throughput under 1M context; gated residual networks use four parallel branches and dynamic gating to suppress activation anomalies and support FP8 storage; the N-gram embedding layer with 51B parameters can be offloaded to Host Memory with almost no increase in token-level computational load; the Muon optimizer re-fits the Scaling Law, supporting larger learning rates and batch sizes, and eliminating the ineffective Batch Size Warmup strategy.

2. Key Features

• Ultra-long Context Processing: Natively supports a context window of up to 262K tokens, which can be extended to 1M tokens. It enables in-depth understanding and Q&A on entire technical manuals, legal contracts, or large codebases in one go. Under a 1M token context, the prefill throughput reaches 8.6 times that of previous generations, ensuring both speed and accuracy in long-sequence processing, and allowing global analysis tasks to be completed without segmentation.

• Code Agent (Agentic Coding): Supports end-to-end software engineering tasks, including requirement analysis, code generation, bug fixing, unit test writing, and maintenance of multi-language projects. Achieves a score of 58.7 on the DeepSWE 1.1 benchmark and 62.5 on the SWE-bench Pro, demonstrating the ability to handle complex, production-level development tasks.

• Office Automation: Capable of executing long-range office workflows and professional tasks, with integrated tool calling functionality to achieve cross-platform task closure. Scores 73.9 on the CoWorkBench long-range office benchmark and 55.7 on the JobBench professional work benchmark, enabling automatic meeting note organization, batch report processing, and the writing and formatting of business documents.

• Multimodal Understanding: Supports long video comprehension, scientific chart analysis, visual math problem solving, and real-world scene interpretation. Achieves a score of 76.6 on the LVBench long video understanding benchmark, 95.7 on the MathVision visual math benchmark, and 90.6 on the CharXiv scientific chart analysis benchmark, meeting the deep content understanding needs in STEM fields.

• Device and Interface Control: Controls web pages, Android devices, and desktop operating systems through visual perception and reasoning capabilities. Scores 84.5 on the AndroidWorld mobile control benchmark, 52.3 on the OSWorld 2.0 computer usage benchmark, and 64.0 on the Vision2Web visual web development benchmark, supporting cross-device tasks such as automatic App testing and file management.

• Reasoning and Q&A: Covers scientific reasoning, competitive programming, instruction-following, and multidisciplinary integrated reasoning. Achieves a score of 91.7 on the GPQA Diamond scientific reasoning benchmark, 91.9 on the LiveCodeBench competitive programming benchmark, and 81.3 on the IFBench instruction-following benchmark, showcasing comprehensive reasoning capabilities across multidisciplinary complex problems.

3. How to Use

1. Environment Requirements and Prerequisites: The deployment environment must be equipped with a GPU that supports FP8 computation. Due to the model's total parameter count of 125B, including a 51B parameter N-gram embedding layer, it is recommended to prioritize multi-GPU clusters or servers with ample host memory. The N-gram embedding layer supports offloading to host memory with asynchronous prefetching and overlapping computation, effectively reducing GPU memory usage. Specific memory requirements depend on context length and concurrency scale; it is advised to refer to the deployment documentation in the official repository for configuration.

2. Open-Source Weight Deployment: Download the Qwen3.8-Flash-Next weights from Hugging Face and deploy them for inference in a local or cloud environment. The GR (Gated Residual) mechanism supports FP8 storage, significantly reducing memory access overhead; during deployment, adjust parallel strategies and batch sizes according to actual hardware configurations to achieve optimal inference throughput and latency performance.

3. API Service Invocation: Call the official API through the Qwen AI platform, with input pricing at 1 RMB per million tokens and output pricing at 3 RMB per million tokens. The API method eliminates the need for managing underlying infrastructure manually, making it ideal for rapid integration into existing business systems. Developers can invoke the API using standard RESTful interfaces and flexibly adjust context length and multimodal input based on business requirements.

4. Qwen Office Usage: Experience the model's capabilities directly within the "Qwen Office" product, suitable for long document processing, code generation, and office automation tasks. This entry point is designed for non-technical users, allowing them to utilize core features such as ultra-long context analysis and office process automation without writing any code, making it ideal for quick validation and daily use in enterprise office scenarios.

5. Review Technical Reports and Deep Customization: Access the GitHub repository to obtain technical reports and learn about the architecture details and training methods of the hybrid attention mechanism GDN+QSA, gated residual mechanisms, N-gram embeddings, and the Muon optimizer. For teams requiring deep customization, you can perform domain-specific fine-tuning or architectural improvements based on the Scaling Law analysis and training strategies outlined in the technical reports, starting from the open-source weights.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

In the context of code intelligence agents and software engineering scenarios, Qwen3.8-Flash-Next outperforms both DeepSeek-V4-Flash and Claude-Opus-4.6 on core benchmarks such as DeepSWE 1.1 and SWE-bench Pro. It also demonstrates a clear advantage in long-range office tasks (73.9 on CoWorkBench), making it a strong candidate as the primary model for enterprise-level development and automation tasks. For requirements involving multimodal control and long video understanding, Qwen3.8-Flash-Next also leads with its performance on AndroidWorld and LVBench. However, Claude-Opus-4.6 performs closely in scientific reasoning but lags behind in office and control tasks.

In terms of general reasoning capabilities, the three models are relatively close—GPQA Diamond scores hover around 91 for all, and LiveCodeBench also falls within the same tier. When choosing, it's important to consider the specific use case: if cost control and long context handling are priorities, Qwen3.8-Flash-Next's 6B active parameters and input pricing of 1 yuan per million tokens offer a significant advantage. If there is an existing dependency on the Claude ecosystem or a need for its specific security features, Claude-Opus-4.6 remains a viable option. DeepSeek-V4-Flash has a higher total parameter count and active parameters than Qwen3.8-Flash-Next, but it trails behind in multiple task benchmarks. Its suitability for specific applications requires further evaluation.

6. Editor's Summary

Qwen3.8-Flash demonstrates clear technical foresight in terms of architectural innovation. The GDN+QSA hybrid attention mechanism combines the efficient memory capabilities of Gated DeltaNet with the precise lookup capabilities of Qwen Sparse Attention. By arranging three layers of GDN and one layer of QSA in every four layers, this design effectively balances computational efficiency and information recall quality in long-sequence modeling. The gated residual mechanism extends the traditional single residual flow into four parallel branches, dynamically controlling the ratio of information read and write operations. This not only preserves long-range information pathways but also suppresses activation outliers, while supporting FP8 storage—a design that directly addresses the memory bottleneck issues in large model training and inference. The N-gram embedding layer captures phrase collocation patterns with 51B parameters, yet achieves near-zero computational overhead through Host Memory offloading and asynchronous prefetching, showcasing a refined engineering approach.

In terms of practical value, the training cost being only 1/9 of that of Qwen3.7-Plus holds significant industrial implications. It means that the cost of acquiring flagship-level model capabilities has been significantly reduced. With an API pricing model of 1 yuan per 1 million tokens, small and medium teams now have the opportunity to integrate high-performance multimodal models into their actual business applications. Its leading performance on benchmarks such as DeepSWE 1.1, SWE-bench Pro, and CoWorkBench indicates that its task completion capabilities in real-world production scenarios have been systematically validated.

This model is suitable for three types of users: first, R&D teams that need to process ultra-long documents and large codebases; second, enterprise developers looking to build office automation and agent applications at a low cost; and third, researchers interested in the evolution of next-generation large model architectures. As an early validation version of the Qwen4 series architecture, Qwen3.8-Flash explores innovations in hybrid attention mechanisms, gated residuals, and optimizer design, providing a reference technical path for future model development. The simultaneous release of its open-source weights and technical reports also creates favorable conditions for the community to deeply understand and conduct further development.

7. Application Scenarios

• Intelligent Analysis of Long Documents and Knowledge Bases: Leverage the 1M tokens ultra-long context capability to perform in-depth understanding and Q&A on entire technical manuals, legal contracts, or large codebases in one go. It can extract global information, perform correlation analysis, and generate conclusions without the need for segmented processing, making it ideal for scientific literature reviews, legal document reviews, and enterprise knowledge base construction.

• End-to-End Software Development and Code Maintenance: Function as a code agent to complete requirement analysis, code generation, bug fixing, unit test writing, and maintenance of multi-language projects. With a performance score of 62.5 on SWE-bench Pro, it can cover the full development process from feature implementation to code review, significantly improving team delivery efficiency.

• Enterprise Office Process Automation: Execute cross-platform long-range office tasks, including automatically organizing meeting minutes, batch processing Excel reports, and drafting and formatting business documents. With integrated tool calling capabilities, it achieves end-to-end closed-loop automation. With a benchmark score of 73.9 on CoWorkBench, it can greatly reduce repetitive manual operations.

• Multimodal Content Understanding and Educational Tutoring: Parse long video courses, scientific paper charts, and visual math problems, providing step-by-step explanations and answer verification. With scores of 95.7 on MathVision and 90.6 on CharXiv, it is capable of performing in-depth teaching assistance and academic research support in STEM fields.

• Intelligent Agent Control Across Devices: Through visual perception and reasoning capabilities, automatically control Android phones to perform app testing, execute file management on desktop operating systems, or generate deployable web frontends based on design drafts. With a score of 84.5 on AndroidWorld and 64.0 on Vision2Web, it is suitable for mobile automation testing and frontend development efficiency improvement scenarios.

8. FAQ

Q: What are the key differences between Qwen3.8-Flash and Qwen3.7-Plus?
A: The two models differ significantly in architecture design and application positioning. Qwen3.8-Flash incorporates four major architectural innovations: GDN+QSA hybrid attention, gated residual networks, N-gram embeddings, and the Muon optimizer. Its training cost is only 1/9 that of Qwen3.7-Plus. It also demonstrates stronger capabilities in coding and office tasks, and natively supports a context length of 262K, which can be extended up to 1M.

Q: Why can the training cost be reduced to 1/9 of Qwen3.7-Plus?
A: This is mainly due to systematic optimizations at the architectural level. The GDN+QSA hybrid attention reduces computational demands in long-sequence scenarios; gated residual networks support FP8 storage, reducing memory access overhead; the N-gram embedding layer can be offloaded to Host Memory with almost no increase in token-level computational load; and the Muon optimizer, after re-fitting the Scaling Law, supports larger learning rates and batch sizes, eliminating the inefficient Batch Size Warmup strategy, thereby comprehensively improving training efficiency.

Q: What hardware configuration is required for local deployment?
A: The total number of model parameters is 125B, with only 6B activated per token. The GR mechanism supports FP8 storage, and the N-gram embedding layer can be offloaded to Host Memory, making the memory usage relatively manageable. Specific configurations depend on the context length and concurrency scale. It is recommended to use GPUs that support FP8 computation and have sufficient Host Memory. For detailed requirements, please refer to the deployment documentation in the official repository.

Q: What is the role of the N-gram embedding layer? Does it affect inference speed?
A: The N-gram embedding layer contains 51B parameters and captures phrase collocation patterns by combining local context lookups with conventional word embeddings. This layer can be offloaded to Host Memory and utilizes asynchronous prefetching and overlapping computation, resulting in almost no increase in token-level computational load. Therefore, it has minimal impact on inference speed.

Q: How is the API charged? What usage scale is it suitable for?
A: The API pricing is 1 yuan per million input tokens and 3 yuan per million output tokens. This pricing is relatively low among flagship-level multimodal models and is suitable for a wide range of scenarios, from prototype validation to large-scale production use, especially for businesses with high-frequency demands for long-context processing.

Q: Is the model fully open source? Can it be used for commercial purposes?
A: The Flash-Next weights have been open-sourced on Hugging Face, and the technical report is available in the GitHub repository. The open-sourced weights support deployment on local or cloud environments. For specific commercial usage terms, please refer to the official license documentation. It is recommended to review the licensing files in the official repository before commercial use.

9. Project Links

1. Project Website: https://qwen.ai/blog?id=qwen3.8-flash-next
2. GitHub Repository: https://github.com/QwenLM/Qwen3.8-Flash-Next
3. Technical Paper: https://github.com/QwenLM/Qwen3.8-Flash-Next/blob/main/tech_report.pdf
4. Hugging Face Model Library: https://huggingface.co/Qwen/Qwen3.8-Flash-Next