Effect examples

1. Tutorial Introduction

EasyControl is a project that aims to add efficient and flexible control to the Diffusion Transformer. It was jointly developed by Tiamat AI, ShanghaiTech University, National University of Singapore and Liblib AI in 2025. The related paper results are "EasyControl: Adding Efficient and Flexible Control for Diffusion Transformer". As the diffusion model architecture shifts from a Unet-based structure to a Diffusion Transformer (DiT), the existing DiT ecosystem lacks mature plug-in support and faces problems such as efficiency bottlenecks, multi-condition coordination conflicts, and insufficient model adaptability. To address these problems, EasyControl proposes an efficient and flexible unified conditional DiT framework. By introducing a lightweight conditional injection LoRA module, a location-aware training paradigm, and combining causal attention mechanisms with KV caching technology, EasyControl significantly improves model compatibility, supports plug-and-play functionality and lossless style control; at the same time, it enhances generation flexibility, can support multiple resolutions, aspect ratios, and multi-condition combinations; and optimizes inference efficiency, making the model more efficient at runtime.

This tutorial uses the stylized Img2Img control model, which is able to transform portraits into Hayao Miyazaki-style artworks. It was trained using only 100 real Asian faces and their corresponding Ghibli-style images generated by GPT-4o, while retaining facial features and applying the iconic anime aesthetic. In addition, EasyControl has collaborated with the CFG-Zero team to further improve image fidelity and controllability. The team also plans to continue to optimize model performance and explore more application scenarios. More pre-trained weights and training codes may be released in the future to support further research and development in the community.

This tutorial uses EasyControl as a demonstration, and the computing resource uses A6000.

2. Operation steps

1. After starting the container, click the API address to enter the Web interface

If "BadGateway" is displayed, it means the model is initializing. Since the model is large, please wait about 1-2 minutes and refresh the page.

2. Once you enter the website, you can start using

How to use

Demo

Exchange and discussion

🖌️ If you see a high-quality project, please leave a message in the background to recommend it! In addition, we have also established a tutorial exchange group. Welcome friends to scan the QR code and remark [SD Tutorial] to join the group to discuss various technical issues and share application effects↓