Keye-VL-2.0-30B-A3B – Kuaishou's Open Multimodal Large Model

1. What Is Keye-VL-2.0-30B-A3B

Keye-VL-2.0-30B-A3B is Kuaishou's fully open multimodal large language model—30B total parameters (MoE, ~3B active)—and the first to bring DeepSeek Sparse Attention (DSA) into multimodal understanding. It supports a 256K context window for near-lossless end-to-end temporal reasoning on hour-long video, beating Gemini-2.5-Pro and Gemini 3 Flash on multiple TimeLens video metrics. It also debuts Code Interpreter, Tool Use, and Search in the Keye line, evolving the model from passive observer to active agent.

Technical positioning and domain: MLLM focused on long-video understanding, temporal causal reasoning, and agentic multimodal execution—sparse attention for ultra-long visual context plus agent tooling in a perceive→reason→act loop.

Development background: Kwai-Keye team, grounded in short video, live streaming, and content understanding. Targets context limits, compute explosion, and lack of active execution on hour-scale video.

Core value: End-to-end hour video without chunking; linear-ish cost via DSA; first-class agent actions (code, tools, search) extending AI from "understand" to "do."

Technical characteristics: DSA in multimodal production; frame-level action boundaries and audio-visual modeling for millisecond localization; MoE 30B/3B active; integrated Code/Tool/Search agent loop.

2. Key Features

• Ultra-long video understanding: Up to 256K tokens without segmentation—coherent parsing of 60+ minute vlogs into structured summaries (gear, budget, POIs, safety tips).

• Temporal causal reasoning: Beyond frame labels—infers strategies from event chains (e.g., snow accident → "group tour safer than self-drive") via action boundaries and dynamic visual parsing.

• Millisecond frame localization: Timestamp-level breakdown of industrial processes or esports highlights.

• Cross-modal fusion: Vision, audio, and text in a shared space with emotional cues from soundtrack enhancing narrative understanding.

• Agent collaboration: Code generation, tool calls, and search in one loop—e.g., write analysis scripts and call APIs from video understanding.

• Noise robustness: Key-frame extraction and dynamics in surveillance/live streams via sparse attention filtering redundancy.

3. How to Use

1. Obtain model: GitHub https://github.com/Kwai-Keye/Keye, Hugging Face Kwai-Keye/Keye-VL-2.0-30B-A3B, or ModelScope. Apache 2.0.

2. Hardware: 2× NVIDIA H800 (80GB+) recommended (~160GB total); Linux Ubuntu 22.04+; CUDA 12.1+; PyTorch 2.1+.

3. Docker (recommended): docker pull kwai-keye/keye-vl:latest—includes custom SGLang, DeepGEMM, EffectiveKernels.

4. Source install: Clone and build Keye SGLang, DeepGEMM, EffectiveKernels; launch with tensor parallel and --trust-remote-code.

5. Serve: python -m sglang.launch_server --model-path ./Keye-VL-2.0-30B-A3B --tp 2 --trust-remote-code on port 8000, OpenAI-compatible API.

6. Infer: POST video (Base64/URL) + instruction; receive structured JSON with timestamps, causal conclusions, or agent outputs.

Notes: ~60GB weights on first load; ensure NVLink/PCIe bandwidth; Docker most stable for dependency hell.

4. Pros and Cons

5. Comparison with Similar Tools

Selection guidance: Hour-long video + millisecond localization + open weights → Keye. General multimodal + managed ecosystem → Gemini. Chinese academia + dense VL → Qwen2.5-VL-72B (shorter context, weaker agent story).

6. Editor's Take

Keye-VL-2.0-30B-A3B advances long-video MLLMs: DSA migration from LLMs to video, systematic agent integration, and TimeLens wins over Gemini on key slices—all as open weights.

Direct value for content platforms, industrial QA, esports, and education where agent automation cuts manual work. Hardware and deploy complexity limit mass adoption; hope for quant/distill variants.

Audience: researchers, video engineers, platform teams. Future: 512K context, multi-video reasoning, richer agent + RAG/multi-agent stacks.

— −0.5 for hardware/deploy friction; strong tech and open spirit.

7. Use Cases

• Long-form summarization: Travel docs, tutorials—timelines, recommendations, safety structured reports.
• Industrial process analysis: Millisecond action nodes for SOP extraction and QC.
• Esports/sports highlights: Tactical moments beyond kill feeds—auto highlight reels.
• Agent automation: Scripts + APIs + search from experimental or operational video.
• Education: Student performance videos with step-level feedback and scoring support.

8. FAQ

Q: VRAM requirements?
A: 2× H800 80GB (~160GB total) recommended; single 80GB A100 works but slower; wait for quant builds for consumer GPUs.

Q: Video formats and limits?
A: MP4/AVI/MOV etc.; 256K ≈ 1–2 hours depending on fps/resolution; 720p or lower saves tokens.

Q: DSA vs dense attention?
A: Dense O(n²) explodes at 256K; DSA sparsifies to relevant frames/regions (~O(n log n) or O(n)) while preserving accuracy.

Q: Agent setup?
A: Built-in JSON action sequences (e.g., code_interpreter); sandbox executes code; search needs network egress.

Q: Chinese performance?
A: Good overall; slang/dialect weaker than English—use Chinese output prompts; team plans CN optimization.

Q: Live stream support?
A: Offline-first today; chunked processing approximates realtime; streaming API may come later.

9. Project Links

• GitHub: https://github.com/Kwai-Keye/Keye
• Hugging Face: https://huggingface.co/Kwai-Keye/Keye-VL-2.0-30B-A3B