OmniGen: New Breakthrough in Unified Image Generation Models

In this paper, we introduce OmniGen, a novel unified image generation diffusion model. Unlike popular diffusion models such as Stable Diffusion, OmniGen does not require additional modules like ControlNet or IP-Adapter to handle different control conditions. This new model boasts several key features: Unified Capabilities: OmniGen excels not only in generating images from text but also supports a variety of downstream tasks seamlessly. These include image editing, subject-driven generation, and visual condition generation. Additionally, OmniGen can tackle traditional computer vision tasks, such as edge detection and human pose estimation, by converting them into image generation tasks. Simplicity: The architecture of OmniGen is remarkably streamlined, eliminating the need for extra text encoders. Compared to existing diffusion models, OmniGen is user-friendly, enabling complex tasks through simple instructions without the necessity for additional preprocessing steps, such as estimating human poses. This simplification significantly enhances the efficiency of the image generation workflow. Knowledge Transfer: By learning in a unified format, OmniGen effectively transfers knowledge across different tasks. This capability allows the model to handle unseen tasks and domains, demonstrating new and innovative abilities. We also explore the potential applications of OmniGen's reasoning and chain-of-thought mechanisms. This work represents the first significant attempt to create a general-purpose image generation model that can handle a wide array of tasks with minimal overhead. However, there are still some unresolved issues that require further investigation. To foster progress and encourage collaboration in this field, we have open-sourced the relevant resources on GitHub at https://github.com/VectorSpaceLab/OmniGen. OmniGen marks a step forward in the evolution of image generation technology, offering a versatile and simplified approach to handling diverse tasks. Its ability to convert various visual recognition tasks into image generation problems opens up new possibilities in fields like computer vision and beyond. By making these resources freely available, we aim to accelerate research and development, ultimately leading to more advanced and practical image generation models.