AI News (2026/7/31): Tencent HunYuan Team Open Sources AngelSpec: End-to-End Speculative Decoding Training Framework

Release Overview

The Tencent HunYuan team has recently open-sourced AngelSpec, an end-to-end speculative decoding training framework built on TorchSpec. The framework decouples inference from training: the inference engine streams hidden states to training workers via Mooncake/RDMA. It supports unified training across six types of draft architectures, including MTP autoregressive and DFly block-parallel. According to the arXiv preprint, the DFly draft achieved 1.98–2.40× end-to-end acceleration on the Hy3-A21B model, with a throughput improvement of 10.5–11.8% over DFlash. Alongside the framework, the project released the code, MTP and DFly draft weights for Hy3-A21B and Qwen3-8B, training configurations, and technical reports.

Image source: official article

Core Features

AngelSpec integrates six speculative decoding draft architecture variants—DFly, DFlash, DFlare, Eagle3, DSpark, and MTP—into a single training pipeline; switching between them requires only a configuration change. The MTP draft model is trained using a multi-depth autoregressive expansion with shared parameters, resulting in memory overhead comparable to that of a single causal forward pass, and supports context lengths up to 128k via Ulysses sequence parallelism.

Image source: official article

For training optimization, the framework provides a variety of acceptance rate alignment objective functions, including CE, top-k KL, LK Loss, D-PACE weighted, and end-to-end TV loss, all configurable through flexible combinations. It also implements document-aware sequence packing using Megatron-style fixed-length packing with strict cross-document isolation, covering both DFlash and MTP paths.

AngelSpec includes an online real-time evaluation mechanism: during training, it executes actual speculative decoding using inference engines such as vLLM and reports the average acceptance length and bit-wise acceptance rate in real time. The inference backend is compatible with mainstream engines including vLLM, SGLang, and HuggingFace.

Getting Started

AngelSpec is open-sourced on GitHub and comes with a complete installation and usage guide. To set up the environment, run pip install -e ".[vllm]" and pip install mooncake-transfer-engine. For a single-node quick start, divide 8 GPUs into 4 for inference and 4 for training, then execute ./examples/qwen3-8b-dfly/run.sh to launch the full pipeline. Configuration can be overridden directly via the CLI, for example: training.learning_rate=5e-5 training.num_train_steps=500. A Conda environment can be built with one command using ./tools/build_conda.sh 1 vllm. For multi-node deployment, the Inference Controller and Training Controller are orchestrated via Ray, combined with Mooncake’s hidden state storage, to achieve decoupled training across nodes. Details regarding the open-source license type and the specific GPU models required for training have not yet been disclosed.

Sources and Project Links

• Project Website: https://angelspec.readthedocs.io/
• GitHub Repository: https://github.com/Tencent/AngelSpec
• HuggingFace Model Library: https://huggingface.co/collections/AngelSlim/angelspec
• arXiv Technical Paper: https://arxiv.org/pdf/2607.25852