Spark-Audio-1.0-Preview – iFLYTEK's Domestic-Developed Speech Foundation Model

1. What is Spark-Audio-1.0-Preview

Spark-Audio-1.0-Preview is a fully domestically developed speech foundation model launched by iFLYTEK. It employs a 0.65B audio encoder and a 30B-A3B MoE language model architecture, trained on 13 million hours of audio and a large volume of text data using purely domestic computing clusters. This model can directly parse semantic, emotional, and contextual information from speech, supporting the identification of 99 languages and 202 dialects, as well as multiple tasks such as speech transcription, translation, speaker recognition, sentiment analysis, and audio question-answering. It demonstrates leading performance in complex scenarios such as high noise and low volume, and can be widely applied in business areas like intelligent customer service, automotive cockpits, and medical electronic records.

Image source: Official article
Image source: official article

Technical Positioning and Domain: Spark-Audio-1.0-Preview belongs to the domain of speech understanding and generation, targeting multi-task applications such as speech recognition, speech translation, speaker recognition, and sentiment analysis. As a speech foundation model, it is positioned between general speech understanding models and specialized speech recognition systems. iFLYTEK aims to replace the traditional "speech-to-text - text understanding" cascaded approach with a unified end-to-end architecture, maintaining high accuracy in complex acoustic environments while preserving prosody, emotion, and contextual information in speech, thereby providing standardized foundation capabilities for upper-layer speech applications.

Development Background: This model was developed by the iFLYTEK Research Institute, leveraging the company's nearly two decades of technical accumulation and access to massive real-world speech data resources in the field of speech technology. The development motivation stems from two aspects: first, traditional cascaded speech understanding pipelines suffer from information loss and error accumulation, limiting their effectiveness in complex scenarios; second, the maturation of domestic computing power ecosystems has made it feasible to train large models on purely domestic chip clusters. iFLYTEK aims to standardize its accumulated speech understanding capabilities through the construction of a speech foundation model, supporting various speech-related services within its StarFire ecosystem.

Core Value: This model addresses two key issues. The first is the end-to-end speech understanding challenge, achieved by allowing the language model to directly receive audio input, thereby avoiding the semantic loss, emotional information loss, and increased latency that occur in traditional approaches where transcription precedes understanding. The second is the verification of domestic computing power feasibility. As the first speech foundation model in the industry trained on a purely domestic computing cluster, it demonstrates that domestic computing power is internationally competitive in training large-scale speech models, providing empirical reference for the self-reliance and controllability of the industry chain.

Technical Features: The model architecture combines a 0.65B Dense audio encoder with a 30B-A3B MoE language model, achieving near-par understanding capabilities with lower inference costs compared to larger models. The technical approach adheres strictly to end-to-end training, feeding both speech and text as dual-modal inputs into the same model. Through pre-training of the audio encoder and staged post-training, the model gradually enhances its generalization in complex scenarios, ultimately enabling synchronized understanding of tone, emotion, and ambient sounds, forming a complete perception pipeline distinct from cascaded solutions.

2. Key Features

• End-to-end speech understanding: This is an architectural-level feature that distinguishes the model from traditional speech recognition systems. After the audio signal is processed by the encoder, it is directly input into the large language model, without first being transcribed into intermediate text, thereby avoiding information loss and error propagation in cascaded pipelines. The model can simultaneously understand multidimensional information in speech, including semantic content, speaker emotion, and environmental background noise, achieving a leap from "hearing clearly" to "understanding deeply."

• Speech transcription: Supports speech-to-text transcription in 99 languages and 202 dialects. Compared to models of similar size, this model demonstrates superior performance in multilingual and multidialect recognition, achieving state-of-the-art (SOTA) results on the Fleurs Chinese test set. In high-noise and low-volume scenarios, the transcription accuracy is significantly better than traditional end-to-end approaches, making it suitable for applications such as meeting minutes, subtitle generation, and voice search.

• Multilingual translation: Enables cross-language translation from speech to speech and speech to text, preserving the original speech's rhythm, tone, and intonation. Compared to pure text-based machine translation, this model can use the prosodic information of the source language to aid in semantic understanding, delivering more natural translation results in scenarios such as multilingual conference interpretation, cross-language customer service, and localization of audiovisual content.

• Multidialect recognition: Natively supports the recognition of 202 dialects, including commonly used ones such as Cantonese, Wenzhou dialect, Hokkien, and Sichuan dialect. Unlike approaches that train dialects as independent languages, this model learns cross-dialect common features through unified audio representation, maintaining high recognition accuracy even on dialects with sparse data, thus significantly reducing the data collection costs for dialect speech applications.

• Environmental sound recognition: The model can simultaneously parse human speech and environmental background sounds, understanding the context in which the sounds occur. For example, it can distinguish between different acoustic environments such as train stations, restaurants, and high-speed driving, and use the scene information as auxiliary context for semantic understanding. This capability has practical value in scenarios such as security monitoring, intelligent vehicle cabins, and smart home systems.

• Speaker identification: Automatically distinguishes different speakers within the same audio segment, providing structured information for meeting transcription, interview records, and customer service quality checks. This feature can achieve basic speaker differentiation without pre-registered voiceprints, and when combined with the transcription engine, enables content archiving at the speaker level, significantly reducing the cost of manual sorting and annotation.

• Emotion analysis: Captures prosodic features such as tone, rhythm, and pitch in speech to infer the speaker's emotional state, such as calm, angry, happy, or anxious. In intelligent customer service scenarios, emotion recognition can be used to adjust service strategies in real time and issue warnings for high-risk conversations; in medical scenarios, it can assist in assessing the patient's psychological state.

• Audio question answering: Users can ask natural language questions about complex audio content, and the model can directly answer based on its deep understanding of the audio. This capability goes beyond simple transcription and retrieval, requiring the model to truly comprehend the event progression, character relationships, and logical information within the audio. It is applicable to scenarios such as long-form audio content review, training recording analysis, and podcast content retrieval.

3. How to Use

Spark-Audio-1.0-Preview is currently available for online experience, and no local deployment is required. Here is the specific usage process:

1. Environmental Requirements: No software installation or runtime environment configuration is needed. Simply use modern browsers such as Chrome, Edge, or Safari to access the official experience page. The experience process does not require high network bandwidth and can be smoothly used in a regular internet environment. If local private deployment is required, it is recommended to configure a domestic AI acceleration card cluster with strong computational power. Specific hardware specifications and deployment solutions will be announced by the official channel.

2. Access the Official Experience Page: Open the experience URL https://iflytekresearch.iflytek.com/experience/spark-audio in your browser. After the page loads, the model environment will be automatically initialized. The first load may take several seconds to over ten seconds.

3. Select Input Method: The experience page supports two input methods: text input and audio input. Audio input can be recorded in real-time via a microphone or uploaded from a local audio file. Text input is used to test the model's text comprehension capabilities, making it easier to compare model performance across audio and text modalities.

4. Experience Specific Features: Choose the function type based on your testing needs. For audio transcription, you can directly upload an audio file to obtain text results; for translation, you can set the source and target languages for cross-language conversion; for dialect identification, you can select a specific dialect for targeted testing; for audio question-answering, you need to ask specific questions about the uploaded audio and view the model's response.

5. Observe Output Results: The model's output includes multi-layered information such as transcribed text, translation results, emotional tags, and speaker identification. It is recommended to use test audio with different noise backgrounds and varying volumes to compare the model's performance in complex scenarios, thereby comprehensively evaluating its practical capabilities.

6. Precautions: The experience page is currently open in a preview format, primarily aimed at functional verification and technical evaluation, and is not suitable for production environment calls. The current version's feature scope, call frequency limits, and commercial licensing methods still need to be referenced from the official documentation and technical specifications to be released later. If you need to integrate the model's capabilities into your business system, it is recommended to contact iFlytek officially to obtain a dedicated solution based on fine-tuning of this base model.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

The following table compares Spark-Audio-1.0-Preview with mainstream large audio models from the perspective of core technologies:

In terms of selection recommendations, for enterprise users pursuing self-reliance and control, especially state-owned enterprises and government agencies with domestic compute deployment needs, Spark-Audio-1.0-Preview is currently the only domestically developed large-scale trained speech foundation model verified on domestic compute power. It is recommended to prioritize this model. It has differentiated advantages in multi-dialect coverage and noise resistance, making it suitable for complex acoustic scenarios in Chinese environments such as call centers and in-vehicle voice systems. However, attention should be paid to the still-to-be-disclosed information regarding context length and API openness.

For business scenarios requiring processing of ultra-long audio (e.g., meeting recordings over 10 hours) or high-concurrency real-time speech interaction, Qwen3.5-Omni-Flash has more clearly defined technical parameters in terms of long context and low latency. For general AI applications requiring comprehensive multimodal processing capabilities and simultaneous understanding of both image and speech content, the mature ecosystem and 1,000,000-token context window of Gemini-3.1 Pro are more appealing. For domestic developers, it is recommended to comprehensively evaluate based on their own business's compute resources, audio data duration, and real-time requirements, and to prioritize models with fully disclosed technical parameters for benchmark testing.

6. Editor's Summary

Spark-Audio-1.0-Preview has made a valuable exploration in the technical roadmap of large speech foundation models. Its core innovation lies in completely replacing the traditional cascaded approach with an end-to-end architecture. By combining a 0.65B audio encoder with a 30B-A3B MoE language model, it achieves synchronized understanding of speech semantics, emotion, and scene information, structurally addressing the issue of information loss in cascaded pipelines. The model was trained on 13 million hours of data and completed training on a purely domestic computing power cluster, providing empirical evidence for the performance and stability of domestic AI chips in large-scale model training. This practice holds significant importance for the autonomous controllability of the industrial chain.

In terms of practical value, the model's performance in multilingual, multi-dialect coverage and complex acoustic scenarios is already comparable to, or even exceeds, closed-source models with a parameter size one order of magnitude larger. Its SOTA results on the Fleurs Chinese subset and benchmark evaluations against Gemini-3.1 Pro support its claimed capabilities. Meanwhile, the model shows no significant decline in text-based tasks, giving it a clear cost advantage as a "multi-purpose model." In application scenarios with strong demand for Chinese and dialects, such as intelligent customer service, in-vehicle interaction, and medical transcription, the model has a clear path to real-world deployment.

In terms of target users, the model primarily caters to three categories: enterprise technical decision-makers with requirements for autonomous controllability, engineers developing speech applications, and researchers in the field of speech technology. The first two groups are advised to pay attention to its foundation model fine-tuning capabilities and the pace of subsequent commercialization and open access. Researchers, on the other hand, can delve into two key areas: the design of the end-to-end architecture and the methodology for training on domestic computing power.

Overall, Spark-Audio-1.0-Preview reflects iFLYTEK's strategic intent to transform over two decades of accumulated speech technology expertise into foundation model capabilities. While the current preview version still has room for improvement in terms of functional completeness and ecosystem maturity, its technical foundation has already shown clear advantages. Whether it can form a complete closed-loop in open-source toolchains, API services, and industry-specific solutions will determine its long-term standing in the competition for large speech models.

7. Application Scenarios

• Intelligent Customer Service and Call Centers: In customer service call scenarios, the model transcribes customer speech in real time and combines sentiment analysis to determine the customer's emotional state. When negative emotions such as anger or anxiety are detected, the system can prompt the agent to adjust their communication strategy or prioritize transferring the call to a senior agent, thereby improving problem resolution rates and customer satisfaction. The model's dialect recognition capability enables the customer service system to serve more users who speak dialects, expanding the service coverage.

• In-Car Cabin Interaction: In high-noise environments such as high-speed driving, open windows, or music playback, the model uses its noise-resistant capabilities to accurately recognize the driver's voice commands. It supports dialect interaction, allowing drivers to control navigation, air conditioning, windows, and other functions using their native language. Environmental sound recognition can detect acoustic scenarios inside and outside the vehicle, automatically adjusting interaction strategies and the volume of voice feedback to create a more natural in-car voice interaction experience.

• Real-Time Translation and Multilingual Meetings: In scenarios such as cross-border video conferences, international business negotiations, and academic presentations, the model can perform speech-to-speech simultaneous interpretation and multilingual translation, preserving the speaker's tone, rhythm, and other prosodic information to make the translation more expressive and natural. Compared to pure text translation, translation with prosodic information is more effective in conveying the speaker's attitude and emotions, thereby improving the efficiency of cross-language communication.

• Meeting Transcription and Minutes Generation: For internal company meetings, interviews, court proceedings, and other scenarios, the model automatically completes speech transcription, speaker identification, and structured minutes generation. The system can label the speech content of different speakers, extract key decision points and action items, and automatically generate a draft of the meeting minutes, helping staff reduce repetitive organization tasks and improve the efficiency of meeting documentation.

• Medical Electronic Health Records and Content Moderation: In medical scenarios, doctors can complete medical record entry through voice narration, and the model transcribes speech directly into structured electronic health record text, supporting the recognition of professional terminology to reduce the burden of manual entry. In content moderation scenarios, the model automatically transcribes audio and video content, performs sentiment analysis, and identifies speakers, helping moderators quickly locate segments of non-compliant content and improving the efficiency of content safety governance.

8. FAQ

Q: What is the fundamental difference between Spark-Audio-1.0-Preview and traditional speech recognition systems?
A: Traditional speech recognition systems typically use a cascaded architecture, first transcribing speech into text and then feeding the text into a language model for semantic understanding. This approach results in information loss at each step, especially for non-textual information such as tone, emotion, and ambient sounds, and errors can accumulate along the chain. Spark-Audio-1.0-Preview employs an end-to-end single-model architecture, directly inputting audio signals into a large language model to simultaneously understand semantics, emotion, and context. This avoids information loss in the cascaded pipeline and is the core reason for its superior performance in complex scenarios.

Q: Does the model's support for 99 languages and 202 dialects mean it achieves the same recognition level for all of them?
A: Not necessarily. The model achieves high recognition accuracy for mainstream languages and commonly used dialects. For less common languages and dialects with limited data, it still has recognition capabilities, but the accuracy may decrease. iFLYTEK officially stated that in comparisons with models of similar size, its multilingual and multidialect recognition performance is generally superior, achieving SOTA results on the Fleurs Chinese test set. For specific recognition capabilities of a particular language or dialect, it is recommended to test with corresponding audio files on the experience page.

Q: How should we understand the model's MoE architecture and its 30B-A3B parameter configuration?
A: "30B-A3B" indicates that the total parameter count of the model is 30 billion, and it uses a MoE (Mixture of Experts, mixed expert) architecture. During inference, only about 3 billion parameters are activated per token (i.e., 3B). The MoE architecture divides parameters into multiple expert modules and activates only the most relevant expert combination for the current input during each inference. This achieves the effect of "large parameter count, low computational cost." It is a key design that allows the model to approach the performance of larger models with relatively low computational overhead.

Q: Does this model support commercial use and secondary development?
A: At present, the official release is in preview mode, offering online experience only, and has not yet announced specific policies regarding commercial licensing, open-source plans, or API services. From iFLYTEK's "1+N" system perspective, Spark-Audio, as a speech foundation model, is expected to support further fine-tuning to develop specialized models such as speech recognition, speech simultaneous interpretation, and speech interaction. However, the specific methods of openness, timeline, and licensing terms must await the official release.

Q: How does the model perform when processing ultra-long audio content?
A: The official has not yet disclosed the maximum input length of context for the model. In competitive comparisons, Qwen3.5-Omni-Flash supports a maximum input of 196,000 tokens, capable of handling audio content over 10 hours, while no official data is available for Spark-Audio in this regard. For long audio processing needs exceeding several minutes, it is recommended to first upload long audio files to the experience page for practical testing, or to follow the official technical documentation for accurate parameters in future releases.

Q: Does a model trained on domestic computing power require domestic chips for inference?
A: The training phase is completed using a domestic computing power cluster, but this does not necessarily bind the inference phase to specific hardware requirements. After the model is released, it can be deployed on both domestic chip environments and mainstream general-purpose GPU environments. However, to fully leverage the model's advantages in autonomy and controllability, it is recommended to deploy and infer it on domestic computing power platforms. Specific hardware compatibility lists and configuration requirements should be referenced in the official deployment documentation to be released later.

Q: Can the model recognize non-speech audio content such as music or animal sounds?
A: The model was trained using a diverse set of audio data, including environmental sounds and music, and has a certain level of understanding of non-speech audio content. Its core strength lies in understanding the relationship between human speech and background noise, such as separating and recognizing speech in noisy environments or identifying the context in which sounds occur. For precise music analysis or specific sound event classification, the base model may have limited capabilities, and targeted fine-tuning on the base model is recommended to achieve better specialized performance.

9. Project Links

• Official Experience Page: https://iflytekresearch.iflytek.com/experience/spark-audio