One-sentence Summary

Researchers from South China University of Technology, Tsinghua University, and colleagues propose LayerEditor, a unified framework that couples instruction-guided image editing with controllable generation of reusable RGBA layers through In-Context RGB–Alpha Token Modeling and Compositional Edit-Prior Distillation, and, supported by the LayerEditor-100K dataset, achieves strong editing fidelity and flexible layer-level manipulation.

Key Contributions

• The paper introduces LayerEditor, a unified framework that couples instruction-guided image editing with controllable generation of reusable RGBA layers via in-context RGB-alpha token modeling of the edited canvas and ordered layer sequence.
• The paper presents Compositional Edit-Prior Distillation (C-EPD), an on-policy distillation strategy that transfers a pretrained editor's prior to both the edited canvas and the image composed from predicted layers to preserve editing fidelity during layered adaptation.
• The paper constructs LayerEditor-100K, a large-scale paired image-editing dataset with ordered RGBA layer annotations, and introduces dedicated layered-editing benchmarks; experiments report strong editing fidelity, accurate RGBA layer generation, and flexible layer-level manipulation.

Introduction

Instruction-driven image editing is important for tasks such as e-commerce content production and advertising design, where users need to modify only specified content while preserving unrelated regions. Recent multimodal diffusion transformers often synthesize a single flattened canvas that merges text, objects, and background, making fine-grained editing, reuse, and later revision difficult. Prior layered workflows such as Agent Banana treat editing and decomposition as separate stages with different objectives, so even visually plausible edits can produce incomplete or poorly organized layers and require costly manual refinement. The authors introduce LayerEditor, a unified model that couples instruction-driven editing with generation of reusable RGBA layers. It uses In-Context RGB-Alpha Token Modeling and a dedicated alpha decoder to represent ordered layers and fine transparency, while Compositional Edit-Prior Distillation helps maintain the pretrained editor's generation quality during layered adaptation. The authors also construct the LayerEditor-100K dataset and two benchmarks for systematic evaluation.

Dataset

The authors introduce LayerEditor-100K, a dataset of 100K editing pairs built from three sources: Crello, Prism-LayersPro, and ScaleEdit. Each example provides a source image, an editing instruction, an edited target image, and an ordered full-canvas RGBA layer stack that reconstructs the target.

Dataset composition and processing:

• Native Crello and PrismLayersPro designs retain their original layered structure during edit construction.
• Flattened ScaleEdit pairs are converted into target layers through planning, decomposition, refinement, and quality filtering.
• All pairs are verified for instruction execution, layer isolation, and recomposition.
• The dataset contains 48,329 text edits, 36,858 object edits, and 14,813 poster-level appearance edits.
• Text edits include movement, recoloring, replacement, and translation.
• Object edits include addition, removal, movement, recoloring, and replacement.
• Each target has 2 to 10 layers, ranging from compact layouts to fine-grained organizations of text, objects, and decorations.

Evaluation benchmarks and usage:

• Crello-Testset contains 2,994 cases derived from native Crello and PrismLayersPro designs.
• ScaleEdit-Testset contains 1,000 cases derived from flattened editing pairs with recovered target layers.
• Both benchmarks are kept strictly disjoint from the training data, including source documents and all derived editing or layer variants.
• The authors pool the 3,994 evaluation cases to create controlled settings for layered controllability, including layer-count control and image-to-layer decomposition.
• Image-to-layer decomposition is evaluated on held-out original Crello examples with at most ten layers.

Method

The authors propose LayerEditor, a generative framework that extends a pretrained image editing model to jointly produce an edited canvas and a stack of reusable RGBA layers. Given a source image $I^{\mathrm{src}}$Isrc, an editing instruction $u$u, and a requested layer count $N$N, the model generates the edited canvas $\mathcal{D}$D and an ordered layer stack $\mathcal{S}$S:

Here, $R_i$Ri​ and $A_i$Ai​ represent the RGB appearance and Alpha opacity of layer $i$i, respectively. The layer stack is ordered from back to front, and its exact alpha-over rendering produces the composited image $\mathcal{C} = \mathrm{Comp}(\mathcal{S})$C=Comp(S).

To achieve this joint generation, the authors introduce In-Context RGB-Alpha Token Modeling. Frozen encoders provide source-image and instruction conditions, while the requested layer count $N$N determines the number of output pairs. At noise level $\sigma_t$σt​, the joint latent state is constructed as:

All output fields are initialized from Gaussian noise and jointly denoised within a unified sequence. The model represents appearance and opacity as distinct interacting fields, using shared attention to capture dependencies across the edited canvas and the layer stack. To distinguish fields sharing the same canvas coordinates, the authors adopt layer-aware rotary coordinates $(s, x, y)$(s,x,y). Source-image tokens use $s = -1$s=−1, the edited canvas uses $s = 0$s=0, and the RGB and Alpha fields of layer $i$i use $s = 2i + 1$s=2i+1 and $s = 2i + 2$s=2i+2, respectively. This formulation supports variable layer counts within a shared architecture. Furthermore, a dedicated Alpha decoder is introduced to recover continuous opacity $A_i$Ai​. Initialized from the pretrained RGB decoder and trained separately with opacity and compositional reconstruction objectives, this decoder preserves the pretrained RGB pathway while specializing transparency reconstruction for fine structures and semitransparent boundaries.

Adapting a pretrained image editing model to layered outputs can weaken its original editing capability, potentially introducing text corruption, structural distortions, and unintended changes to surrounding content.

To mitigate this degradation, the authors propose Compositional Edit-Prior Distillation (C-EPD), which uses the frozen pretrained editing model as an on-policy teacher to preserve the original editing prior. The teacher guides both the Direct output and the composed layered representation. Following DiffusionOPD, C-EPD queries the frozen teacher at intermediate states from a detached student denoising trajectory. At a query state $\bar{Z}_q$Zˉq​ with noise level $\sigma_q$σq​, the student recomputes Direct, RGB, and Alpha velocities with gradients. The teacher receives the same Direct latent $\bar{z}_q^{\mathcal{D}}$zˉqD​, source image, and editing instruction to predict $v_{q,T}^{\mathcal{D}}$vq,TD​. For each student output field $X \in \{R_i, A_i\}$X∈{Ri​,Ai​}, the local clean latent is estimated as $\hat{z}_{0|q}^X = \bar{z}_q^X - \sigma_q v_{q,S}^X$z^0∣qX​=zˉqX​−σq​vq,SX​. Frozen decoders recover $\hat{R}_i = \mathcal{D}_R(\hat{z}_{0|q}^{R_i})$R^i​=DR​(z^0∣qRi​​) and $\hat{A}_i = \mathcal{D}_A(\hat{z}_{0|q}^{A_i})$A^i​=DA​(z^0∣qAi​​). Starting from a white background $\widehat{\mathcal{C}}_{-1} = B$C−1​=B, the predictions are composed in back-to-front order:

The student's predicted Alpha controls each layer's visible contribution through standard source-over composition. The authors retain Direct velocity matching and supervise the decoded composite using the loss:

Here, $d_{\mathcal{D}}$dD​ and $d_{\mathcal{C}}$dC​ count the elements of the Direct velocity field and decoded composite, respectively, and $\mathrm{sg}$sg stops gradients through the teacher target. The first term aligns the student's Direct velocity with the frozen teacher, while the second transfers the same editing prior to the layered output through its decoded composition.

For the training process, LayerEditor adapts the frozen pretrained Multimodal Diffusion Transformer (MMDiT) with LoRA. Given the clean target sequence $Z^* = [z^{\mathcal{D},*}, z^{R_0,*}, z^{A_0,*}, \dots, z^{R_{N-1},*}, z^{A_{N-1},*}]$Z∗=[zD,∗,zR0​,∗,zA0​,∗,…,zRN−1​,∗,zAN−1​,∗], the joint noisy state is constructed as $Z_t = (1 - \sigma_t)Z^* + \sigma_t \epsilon$Zt​=(1−σt​)Z∗+σt​ϵ with target velocity $v_t^* = \epsilon - Z^*$vt∗​=ϵ−Z∗. The model predicts the joint velocity $v_\theta$vθ​ conditioned on the source image, editing instruction, requested layer count, and noise level. The authors use the flow-matching loss to supervise the complete velocity sequence:

After the initial flow-matching warm-up, C-EPD introduces teacher guidance for the Direct velocity and decoded layer composition while flow matching remains active. The combined objective is:

This training is supported by the LayerEditor-100K dataset, which provides 100K editing pairs containing source images, editing instructions, edited targets, and ordered full-canvas RGBA stacks that reconstruct the target, covering text, object, and poster-level appearance edits.

Experiment

The experiments evaluate LayerEditor on held-out Crello and ScaleEdit benchmarks for layered image editing, a controlled layer-count setting, a pure image-to-layer decomposition benchmark, and ablations on Crello-Testset. These setups validate editing fidelity after recomposition, layer reusability and opacity accuracy, controllability over layer granularity and assigned content, and transfer to decomposition without decomposition-specific fine-tuning. LayerEditor consistently improves over cascaded, sequential, and joint baselines, showing smaller drops from direct to recomposed outputs and better-separated, reusable RGBA layers. Ablations confirm that the Alpha decoder and both components of C-EPD jointly improve edit consistency, layer segmentation, and layer reconstruction quality.

Decomposition fine-tuning improves composite reconstruction fidelity in FR-based cascades, raising composite SSIM on both benchmarks while leaving direct editing metrics unchanged. LayerEditor achieves the best composite similarity, composite FID, and alpha overlap among the compared methods, with Crello composite SSIM 5.7% higher and composite FID 38.8% lower than Seedream5-Pro Joint. Rubric scores point in the same direction, with LayerEditor earning a higher mean Crello score than the strongest baseline. LayerEditor outperforms Seedream5-Pro and FR-based cascades on composite SSIM, composite FID, and alpha IoU across both benchmarks. Fine-tuning the decomposition stage raises composite SSIM from 0.5584 to 0.6343 on Crello and from 0.6451 to 0.6927 on ScaleEdit without changing direct metrics.

In a controlled layer-count setting, LayerEditor attains the best alpha overlap and rubric-based layer quality, outperforming the strongest baseline in Alpha IoU and semantic grouping. Among the listed baselines, Seedream5-Pro (Joint) provides the strongest overall quality but with relatively high observed latency, while QIL-based variants show clear trade-offs between layer fidelity, semantic quality, and speed. LayerEditor exceeds Seedream5-Pro (Joint) by a clear margin in Alpha IoU and semantic grouping, making it the best at matching requested layer structure. Seedream5-Pro (Joint) is the strongest baseline for alpha and rubric quality but is slower than several QIL-based approaches. FR+QIL-Control improves substantially over FR+QIL in alpha overlap, semantic grouping, and composition while also reducing observed latency. FR+QIL-Decomp-FT achieves relatively high semantic and composition scores but has the weakest RGBAL1 and a very low Alpha score, highlighting a layer fidelity trade-off.

The full LayerEditor configuration achieves the best edit consistency, visual quality, layer segmentation, and RGBA reconstruction error among all ablated variants. Removing compositional editing guidance substantially weakens edit consistency and layer segmentation, and the base model trails all LayerEditor variants on every metric. The dedicated alpha decoder and combined guidance terms together improve layer accuracy and editing fidelity. Removing C-EPD causes a marked drop in edit consistency and layer segmentation compared with the full model. Using either guidance term alone improves edit consistency over the no C-EPD setting but lowers visual quality, layer segmentation, and RGBA L1. Combining both guidance terms improves all four metrics, and the dedicated alpha decoder further reduces RGBA L1 while improving layer segmentation.

The experiments evaluate LayerEditor against FR-based cascades and Seedream-style baselines on composite reconstruction, layer structure alignment, and ablation settings. Decomposition fine-tuning improves composite fidelity without changing direct editing quality, and LayerEditor consistently achieves better composite similarity, alpha overlap, and semantic grouping than the compared methods. Controlled layer-count results show LayerEditor best matches requested layer structure, while Seedream5-Pro Joint is strong but slower, and QIL variants present clear trade-offs among fidelity, semantics, and speed. Ablations confirm that compositional editing guidance, combined guidance terms, and a dedicated alpha decoder are important for edit consistency, visual quality, layer segmentation, and RGBA reconstruction.