CoinVE-200K – Tencent's Open-Source Large-Scale Composite Instruction Video Editing Dataset

1. What is CoinVE-200K

CoinVE-200K is a large-scale composite instruction video editing dataset open-sourced by Tencent Video's Intelligent Creation Team. It contains 200,000 pairs of 1080p high-definition video editing samples, with each pair carrying 2-5 atomic editing instructions covering operations such as addition, removal, modification, and stylization. The longest videos can reach up to 201 frames. The project simultaneously open-sources the 22B-parameter CoinVE-Edit editing model and the CoinVE-Bench evaluation benchmark, establishing a complete closed-loop system from data to models and evaluation standards, addressing the three major gaps in the composite instruction video editing domain: data scarcity, model execution precision, and evaluation standardization.

Image source: Official article
Image source: official article

Technical Positioning & Domain: Belongs to the video generation and editing domain, focusing specifically on video editing tasks under composite instruction conditions. Unlike traditional single-instruction editing datasets, CoinVE-200K requires models to simultaneously understand and execute multiple interrelated editing instructions within the same video segment. This design better aligns with complex editing demands in real industrial scenarios like film post-production and advertising creation, pushing video editing models toward multi-intent collaborative understanding from single-intent execution.

Development Context: Developed and open-sourced by Tencent Video's Intelligent Creation Team. This team has long-term technical expertise in video content production and intelligent creation, having previously released multiple research achievements in video generation and editing. Composite instruction video editing has historically faced three major bottlenecks: scarce training data, models' difficulty in precisely executing multi-intent edits, and the absence of evaluation standards. The team therefore created a comprehensive open-source solution covering data, models, and evaluation to reduce research and development barriers in this field.

Core Value: Solves the problem of scarce high-quality large-scale training data in composite instruction video editing. The 200,000 pairs of 1080p samples provide ample supervisory signals for training multi-intent collaborative editing models. The accompanying CoinVE-Edit model enables execution of multiple editing instructions through a single inference pass. The CoinVE-Bench evaluation set offers standardized performance assessment methods for composite editing tasks. Together, these three components form a complete technical stack, eliminating the need for researchers to build data pipelines and evaluation systems from scratch.

Technical Features: The dataset employs an automated construction pipeline combining "atomic operation composition + multi-stage generation + secondary filtering." Based on a predefined editing classification system, Qwen3.6-27B analyzes source video content and identifies editable subjects. Multiple atomic operations are then combined to generate composite instructions. Subsequently, SAM3/SAM2 generates editing region masks, HunyuanImage-3.0 and Nano Banana synthesize editing keyframes, which are then expanded into full videos via Wan2.1-Animate or VACE diffusion. Finally, Gemini performs secondary quality filtering. On the model side, a region-decoupled attention mechanism is introduced, utilizing Mask Predictor and GateNet to precisely bind multiple instructions to their corresponding spatiotemporal regions while maintaining temporal consistency in unedited areas during composite editing.

2. Key Features

• Massive Compositional Editing Dataset: Provides 200,000 pairs of 1080p compositional instruction video editing samples, each containing source video and edited video. Supports multi-intent joint training for models. The dataset size leads in its category, offering ample supervisory signals for training large-scale video editing models.

• Diverse Atomic Operation Coverage: Covers six atomic editing operation types including addition, removal, replacement, and stylization. Enables flexible combination of multiple atomic operations into composite editing instructions. This design allows models to learn collaborative relationships between different editing operations rather than being confined to single-operation modes, enhancing generalization capabilities for complex tasks.

• Universal Target Support Across Scenarios: Editing targets span multiple subjects including people, objects, and backgrounds, with samples covering rich scene diversity and semantic variations. After training, the model can accurately locate and modify regions corresponding to different editing targets, improving applicability in real-world scenarios.

• Ready-to-Use Editing Model: Accompanied by the open-source CoinVE-Edit model with 22B parameters, designed based on the DiT backbone architecture. Capable of precisely executing multiple editing instructions in one pass. Users can directly load official weights for inference or fine-tuning without requiring from-scratch training, significantly lowering usage barriers.

• System Evaluation Benchmark: Constructs the CoinVE-Bench evaluation set using a framework combining MLLM-based Checklist with dedicated evaluators. Covers 11 fine-grained metrics across four core dimensions: editing accuracy, physical naturalness, semantic preservation, aesthetic quality, technical quality, and temporal stability, providing quantitative assessment methods for compositional editing performance.

• Rigorous Quality Filtering: Implements Gemini secondary quality filtering during dataset construction, screening qualified samples across multiple dimensions including instruction-following fidelity, visual quality, temporal consistency, and integrity of unedited regions. Ensures overall dataset quality and minimizes negative impacts of low-quality samples on model training.

• Region-Decoupled Editing Mechanism: CoinVE-Edit predicts spatiotemporal editing region masks for each instruction via the Mask Predictor. Combines GateNet dynamic gating with Q-Blending cross-attention mechanisms to precisely bind different editing intentions to corresponding regions while suppressing interference from irrelevant regions and maintaining temporal consistency.

3. How to Use

1. Environment Preparation: Install the ModelScope CLI as the dataset download tool and prepare a runtime environment with sufficient storage capacity. The dataset contains video files and mask files; full download requires significant disk space. Reserve adequate storage before initiating the download process.

2. Download Dataset: Execute the modelscope download command to retrieve the dataset. You may choose to perform a full download or initially download only the metadata file metadata_coinve200k.jsonl for preliminary testing. Confirm the data format and field structure before deciding whether to proceed with a full download.

3. Extracting Sharded Files: The dataset is distributed in tar-sharded format. Extract the shards into four corresponding directories: src_videos, tgt_videos, combined_masks, and instruction_masks. Ensure the directory structure matches the path mappings in the metadata to avoid path reading errors during subsequent training.

4. Parsing Metadata: Use Python to read the metadata_coinve200k.jsonl file and obtain the source video path, edited video path, and 2-5 combined editing instruction texts for each sample. Construct an index structure for training samples.

5. Accessing Mask Information: Extract the instruction_mask_video_paths and combined_mask_video_path fields from the metadata to retrieve spatiotemporal editing region masks corresponding to each instruction. These masks are used for region-aware conditional injection during training.

6. Initiating Model Training: Input the parsed video pairs, combined instructions, and masks into the DiT framework. Use the masks as region-aware conditional signals to supervise the training and fine-tuning of the combined instruction video editing model. Adjust batch size and gradient accumulation steps according to GPU memory capacity during training.

7. Reproducing Baseline Results: Directly load the official open-source CoinVE-Edit model weights and run inference on the CoinVE-Bench evaluation dataset to verify combined editing performance and reproduce the results reported in the paper.

Important Notes: Training the combined instruction editing model requires substantial GPU memory; the 22B parameter-scale model is recommended for multi-GPU parallel configurations. Mask files and video files must maintain consistent path relationships, or training data loading will fail. Validate the data loading workflow using the metadata file before initiating large-scale training tasks.

4. Pros and Cons Analysis

5. Comparative Analysis of Similar Tools

Selection Recommendations: If the research focus is on combinatorial instruction video editing—requiring the model to execute multiple editing instructions within a single video—CoinVE-200K is currently the most suitable option. Its 2.55 average instruction count, 1080p high-resolution support, and 201-frame long video capability provide unique advantages for training complex editing models, while the accompanying CoinVE-Edit model and CoinVE-Bench evaluation dataset eliminate the need for researchers to build their own baseline systems.

For research emphasizing larger-scale data coverage or more diverse editing types, OpenVE-3M's 3,000,000-sample size and 8-category editing operation coverage offer distinct advantages. However, its predominantly single-instruction sample structure is unsuitable for combinatorial editing scenarios. ReCo-Data is appropriate for teams with limited resources conducting lightweight model training, but its lack of a dedicated evaluation dataset requires researchers to independently construct assessment frameworks.

6. Editor's Summary

The release of CoinVE-200K provides systematic infrastructure for the niche domain of compositional instruction video editing. From a technological innovation perspective, the project's design of an automated pipeline combining "atomic operation composition + multi-stage generation + secondary filtering" at the data construction level addresses the challenge of scaling the production of compositional instruction training data. At the model level, the region-decoupled attention mechanism achieves precise spatial binding for multiple instructions through mask prediction and dynamic gating, a design approach with reference value for multi-condition control generation tasks.

In terms of practical value, the open-source format integrating data, models, and evaluation benchmarks significantly lowers research barriers in this field. Researchers can directly perform fine-tuning on CoinVE-Edit or use CoinVE-Bench to conduct standardized assessments of proprietary models, avoiding redundant infrastructure development costs. The accuracy improvements over closed-source solutions like Seedance 2.0 and Kling O3 in compositional instruction editing also validate the feasibility of open-source approaches for this task.

This dataset is suitable for researchers specializing in video generation/editing, industrial users engaged in film post-production and advertising creation, and AI development teams requiring high-quality video editing training data. As demand for compositional instruction editing grows and more researchers participate, this field is poised to further refine data scale, editing precision, and evaluation frameworks, accelerating the deployment of video editing models into more complex industrial applications.

7. Application Scenarios

• Film Post-Production: Directors or post-production professionals can complete multi-element replacement, background modification, and visual style transfer operations through natural language composite instructions, significantly reducing the time-consuming frame-by-frame editing process and enhancing post-production efficiency.

• Advertising Content Iteration: Brand teams can rapidly execute bulk edits on the same video material, including character outfit changes, product substitutions, and background adjustments, generating multiple advertising variations for A/B testing while minimizing re-shoot costs.

• Short Video Creation: Creators can input multiple editing instructions and achieve complex effects like object removal, element addition, and visual stylization with a single click, eliminating the need for professional video editing software and lowering the technical barrier for content creation.

• AI Model Training: Serves as high-quality supervised data for training composite instruction video editing large models, enhancing the system's ability to understand and precisely execute multi-intent commands. Also functions as fine-tuning data to improve existing editing models' performance.

• Evaluation Benchmark Research: Provides academia with the standardized CoinVE-Bench evaluation dataset, enabling systematic assessment of video editing models across composite instruction-following, physical plausibility, and visual quality metrics, advancing the standardization of evaluation methodologies.

8. FAQ

Q: What is the core difference between CoinVE-200K and other video editing datasets?
A: The core distinction lies in its composite instruction design. Each sample in CoinVE-200K contains 2-5 atomic editing instructions, requiring the model to perform multiple editing operations within the same video. In contrast, datasets like OpenVE-3M and ReCo-Data primarily focus on single-instruction samples. Additionally, CoinVE-200K provides 1080p high-resolution video samples with a maximum length of 201 frames, and is paired with a 22B model and dedicated evaluation set, forming a complete open-source closed-loop.

Q: How can I download the CoinVE-200K dataset?
A: The dataset is distributed via the ModelScope platform. First install the ModelScope CLI, then execute the modelscope download command to retrieve it. Full dataset downloads are supported, or you can download only the metadata file metadata_coinve200k.jsonl to preview the data format. Detailed usage instructions are available on the GitHub repository and HuggingFace page.

Q: What hardware configuration is required to run the CoinVE-Edit model?
A: CoinVE-Edit has 22B parameters, making it a large-scale video editing model. Both inference and training require substantial GPU memory. Refer to the official repository for specific configuration recommendations. Generally, a multi-GPU parallel setup is needed to achieve practical inference speeds, as consumer-grade GPUs struggle to run the model directly.

Q: What license applies to the dataset and model?
A: The project is released under an open-source format. The exact license terms are determined by the official GitHub repository and HuggingFace page. Licensing scope for the dataset, model weights, and evaluation set may vary, so we recommend carefully reviewing the license information from each source before use.

Q: Can CoinVE-200K be used to train custom video editing models?
A: Yes. The dataset provides source videos, edited videos, composite instructions, and corresponding spatiotemporal editing region masks. Users can input these data into a self-built DiT framework for training or fine-tuning. The metadata file's instruction_mask_video_paths and combined_mask_video_path fields specify mask path information, facilitating regional condition injection during training.

Q: How does the CoinVE-Bench evaluation set assess model performance?
A: CoinVE-Bench employs a dual-track complementary evaluation framework: The MLLM-based Checklist evaluates instruction-following accuracy across three dimensions—editing accuracy, physical plausibility, and semantic preservation—using Gemini 3.6 Flash. A dedicated evaluator measures aesthetic quality (via Aesthetic Predictor), technical quality (via DOVER++), comprehensive quality (via VisualQuality-R1), and temporal stability (via optical flow fields), covering 4 core dimensions and 11 metrics in total.

9. Project Links

1. Project Website: https://coinve200k.github.io/
2. GitHub Repository: https://github.com/coinve200k/CoinVE-200K
3. HuggingFace Model Library: https://huggingface.co/datasets/FireCRT/CoinVE-200K
4. arXiv Technical Paper: https://arxiv.org/pdf/2608.17566