One-sentence Summary

Researchers from Shanghai Jiao Tong University and Ant Group address image editing limitations by constructing a hierarchical taxonomy of over 1,000 fine-grained edit concepts and ConceptEdit-12M, a 12-million-pair dataset, and by proposing a dense supervision training strategy that synthesizes multiple non-interfering concepts into single image pairs to improve training efficiency and performance, alongside ConceptEdit-Bench for granular evaluation.

Key Contributions

• The paper proposes edit concept scaling, shifting data scaling from source image variety to edit concept richness through a hierarchical taxonomy of over 1,000 fine-grained categories. This taxonomy underlies ConceptEdit-12M, a 12-million-pair editing dataset built with an LLM-distilled synthesis framework and instance-specific VQA filtering for coverage and fidelity.
• A dense supervision training strategy composites multiple non-interfering edits into single image pairs, providing richer learning signals. This accelerates training convergence by 1.5× and improves single-concept and multi-concept editing performance.
• ConceptEdit-Bench is introduced as a granular evaluation suite spanning the fine-grained taxonomy, and the proposed approach achieves state-of-the-art results, outperforming baselines such as ScaleEdit and UnicEdit across diverse real-world scenarios.

Introduction

Recent text-to-image diffusion models have made instruction-based image editing practical, enabling localized modifications while preserving the original image context. However, prior editing datasets and training pipelines have mostly scaled source image diversity while paying little attention to the diversity and granularity of edit concepts. They also treat editing as dense global synthesis even though edits are spatially sparse, which limits training efficiency. The authors argue that edit concept richness is the real bottleneck for generalization. They introduce a structured paradigm with a hierarchical taxonomy of over 1,000 fine-grained edit concepts, a dense supervision strategy that composes multiple localized edits into single image pairs, and an LLM-guided synthesis framework with instance-specific VQA filtering. This yields ConceptEdit, a 12M-sample dataset and fine-grained benchmark aimed at improving editing capability and training efficiency.

Dataset

The authors introduce two related dataset artifacts:

• ConceptEdit-12M: 12 million verified image editing pairs, generated through an improved synthesis framework. The dataset is built around an Edit Concept Library with more than 1,000 fine-grained edit concepts.
• ConceptEdit-Bench: an evaluation benchmark constructed from 1,000 distinct editing categories selected from the same concept library. Source images are sampled from high-quality open-source datasets and cover diverse visual categories.

Dataset construction and processing

• Edit Concept Library construction: The process starts from a small, manually initialized seed taxonomy. An LLM iteratively expands it by merging or pruning redundant concepts, extrapolating new intermediate subcategories, and populating specific leaf nodes. Human experts then refine the taxonomy to resolve semantic overlaps and add missing edge cases.
• Semantic matching and instruction generation: For a source image, the authors sample a candidate concept subset of size $N$N. A VLM then outputs matched concepts, editing instructions, and VQA verification criteria. Category frequencies are tracked, and sampling weights are adjusted adaptively to prevent distribution collapse. A stochastic exploration mechanism can also allow the VLM to propose new concepts outside the candidate set.
• Image synthesis: An editing model generates target images based on the textual editing instructions.
• Instance-specific VQA filtering: The generated question-answer pairs are used as instance-specific verification. A VLM inspects localized regions prone to editing failures, performs chain-of-thought reasoning, and supports discarding low-quality samples or refining instructions through recaptioning.

How the data is used

• ConceptEdit-12M serves as a large-scale source of high-quality image editing pairs, with each sample connecting a source image, a target image, an editing instruction, and generated VQA verification criteria.
• ConceptEdit-Bench is used for fine-grained evaluation across over 1,000 editing concepts, enabling selective monitoring of specific capabilities rather than relying only on a single aggregate score.
• The provided text does not specify explicit train, validation, or test split ratios for ConceptEdit-12M.

Method

The authors identify a critical bottleneck in image editing generalization: the distribution collapse of edit concepts. Current pipelines rely on Vision-Language Models (VLMs) to stochastically generate instructions based on limited coarse-grained categories. This unconstrained dependency leads to severe biases. For instance, in the "style transfer" category, stochastic sampling causes the top 5 styles to dominate 74.6% of the generated instructions, leaving dozens of others at less than 1%.

To overcome this, the authors propose a paradigm shift from stochastic VLM generation to a structured, library-driven approach. Explicitly populating the instruction space with over 1,000 fine-grained categories ensures uniform exposure to diverse visual transformations and establishes a robust conceptual foundation.

Building on these insights, the authors propose an improved synthesis framework designed to generate high-fidelity image editing pairs with a controllable concept distribution. The pipeline consists of four key stages.

The core of the framework is the construction of a comprehensive Edit Concept Library. Moving beyond a handful of human-predefined coarse categories, the authors scale operations into over 1,000 fine-grained categories to ensure high conceptual density. Starting with a lightweight, manually initialized seed taxonomy, an LLM is prompted to continuously evaluate and dynamically expand the classification tree. During each iteration, the LLM merges or prunes redundant concepts, extrapolates new intermediate subcategories, and populates highly specific leaf nodes across diverse domains. This self-expanding loop repeats until semantic expansion converges. Finally, human experts refine the taxonomy to resolve semantic overlaps and supplement missing edge cases.

The extreme granularity of the library necessitates rigorous semantic grounding to ensure compatibility between concepts and image contexts. The authors employ a VLM generator $\Phi$Φ. For a source image $I$I, a candidate concept subset $\mathcal{C}_{\mathrm{cand}} \subset \mathcal{C}_{\mathrm{lib}}$Ccand​⊂Clib​ of size $N$N is sampled, and the matching process is formalized as:

$$\Phi \big(\mathbf{I}, \mathcal{C}_{\mathrm{cand}} \big) \mapsto \big \{(c_{k}, t_{k}, v_{k}) \big \}_{k=1}^{M}, \quad \text{s.t.} M \leq N$$Φ(I,Ccand​)↦{(ck​,tk​,vk​)}k=1M​,s.t.M≤N

where $c_{k} \in \mathcal{C}_{\mathrm{cand}}$ck​∈Ccand​ represents the matched concept, $t_{k}$tk​ is the generated editing instruction, and $v_{k}$vk​ denotes the accompanying VQA verification criteria. By tracking the frequencies of over 1,000 categories, sampling weights for $\mathcal{C}_{\mathrm{cand}}$Ccand​ are adaptively adjusted to prevent distribution collapse. Additionally, a stochastic exploration mechanism allows the VLM to autonomously propose novel concepts with a predefined probability. Following instruction generation, an editing model is invoked to generate target images based on the textual instructions.

To ensure the quality of synthesized editing pairs, the authors implement instance-specific VQA filtering. Utilizing the customized question-answer pairs $v_{k}$vk​ generated during the instruction phase, targeted queries direct the VLM to inspect localized regions prone to editing failures. This structured inquiry facilitates Chain-of-Thought reasoning, enabling the VLM to systematically evaluate the correspondence between the instruction and the visual modification, thereby improving the detection of subtle misalignments.

The inherent sparsity of single-concept edits limits overall training efficiency, as modified regions typically occupy only a small fraction of the image. To resolve this, the authors propose integrating multiple non-interfering edit concepts into a single image pair, a strategy formulated as dense supervision via composition. A VLM-driven aggregator $\Psi$Ψ performs compositional selection and instruction aggregation. For a source image $I$I, a candidate concept subset $\{(c_{n}, m_{n})\}_{n=1}^{N}${(cn​,mn​)}n=1N​ is sampled, where $c_{n} \in \mathcal{C}_{\mathrm{lib}}$cn​∈Clib​ is an edit concept and $m_{n}$mn​ is its corresponding edit region. The process is formalized as:

$$\Psi \big(\mathbf{I}, \{(c_{n}, m_{n})\}_{n=1}^{N} \big) \mapsto \big(T_{\mathrm{comp}}, V_{\mathrm{comp}}, \{(c_{k}, m_{k})\}_{k=1}^{M} \big), \quad \text{s.t.} m_{i} \cap m_{j} = \emptyset, \quad i \neq j, \quad M \leq N$$Ψ(I,{(cn​,mn​)}n=1N​)↦(Tcomp​,Vcomp​,{(ck​,mk​)}k=1M​),s.t.mi​∩mj​=∅,i=j,M≤N

where $M$M denotes the number of successfully selected concepts. The constraint $m_{i} \cap m_{j} = \emptyset$mi​∩mj​=∅ ensures that selected edit regions are spatially disjoint, preventing visual or conceptual interference. The mapping produces a single unified instruction $T_{\mathrm{comp}}$Tcomp​, a global verification checklist $V_{\mathrm{comp}}$Vcomp​, and the set of selected concept pairs. This strategy functions as a form of spatial data compression, significantly increasing the information entropy per sample. By providing dense supervision signals within a single forward pass, the model is forced to allocate more representation capacity to learning structural transformations rather than background preservation, empirically accelerating convergence.

Experiment

The experiments evaluate the proposed ConceptEdit dataset using the Z-Image framework on ImgEdit-Bench and GEdit-Bench at 2M and 5M training scales. Comparative results show that ConceptEdit consistently outperforms UnicEdit and ScaleEdit, with gains concentrated in instruction following accuracy, reflecting the benefit of fine-grained edit concepts. Ablations further validate that scaling concept diversity improves generalization, mixing composite edits provides dense supervision and reduces data requirements, and instance-specific VQA filtering detects localized artifacts and hallucinations more effectively than generic validation.

ConceptEdit with dense supervision achieves the highest overall ImgEdit-Bench scores among compared methods at both 2M and 5M training scales, and its lead over the strongest baseline widens at the larger scale. Expanding edit concept diversity from 10 concepts to 1000 concepts yields steady gains, while adding composite edits provides a further consistent improvement across scales. The benefit of composite supervision extends to categories such as Adjust, Replace, and Act at the 5M scale. The full ConceptEdit framework with dense supervision leads overall performance at both 2M and 5M scales, with a larger margin over ScaleEdit at 5M. Scaling edit concepts from 10 to 1000 improves overall performance at both scales, and ConceptEdit1000 clearly outperforms ConceptEdit10 at 5M. Adding composite edits consistently raises overall scores across scales, with notable 5M gains in Adjust, Replace, and Act beyond the composite task itself. ConceptEdit shows clear category-level advantages in Add, Style, Background, and Action relative to prior baselines.

At the 2M scale, ConceptEdit variants lead ScaleEdit and UnicEdit on instruction following and overall GEdit-Bench scores in both English and Chinese. The full configuration with composite dense supervision records the strongest overall and instruction-following results, while scaling concept categories from coarse to fine yields notable gains. Results at 5M similarly show concept scaling improving instruction following in both languages. The full ConceptEdit configuration with composite supervision achieves the highest overall and instruction-following scores in both English and Chinese at the 2M scale. Dense supervision improves English instruction following over the ConceptEdit1000 baseline by a clear margin. Scaling from 10 to 500 or 1000 concept categories increases instruction following, with 5M results showing gains in both English and Chinese.

The instance-specific VQA filtering pipeline outperforms a generic VLM validation baseline across precision, recall, F1-score, and accuracy when judged against Gemini-3-Pro pseudo-labels. The largest improvement is in recall, indicating that tailored region-aware checks catch more localized hallucinations and subtle failures than uniform holistic prompts. The filtered results move closer to the pseudo-label reference while stopping short of perfect agreement. Recall shows the largest gain among the evaluated metrics, consistent with better detection of localized artifacts. Precision, F1-score, and accuracy all improve relative to generic validation, but remain below the Gemini-3-Pro pseudo-label ceiling.

The experiments evaluate ConceptEdit at 2M and 5M training scales, testing dense supervision, scaling concept diversity from 10 to 1000 categories, and adding composite edits. Dense supervision, broader concept coverage, and composite supervision consistently improve overall image editing and instruction following in English and Chinese, with category-level gains in Add, Style, Background, and Action. The proposed instance-specific VQA filtering also outperforms generic validation, especially in recall, indicating better detection of localized hallucinations and subtle failures.