💡 HiDiffusion: Unlocking Higher-Resolution Creativity and Efficiency in Pretrained Diffusion Models

Introduction to HiDiffusion

HiDiffusion is an innovative framework developed by Megvii Technology to enhance the creativity and efficiency of pre-trained diffusion models in generating high-resolution images. This is a method that can improve the resolution and speed of pre-trained diffusion models without training. By applying HiDiffusion to various pre-trained diffusion models, not only can the resolution of image generation be increased to 4096×4096, but the image generation speed can also be increased by 1.5 to 6 times. This method not only solves the problems of object duplication and high computational burden, but also achieves excellent results in the task of generating high-resolution images.

The project supports a variety of tasks including text to image, image to image, and image restoration.

Effect examples

(Faster, and better image details.)

(2K results of ControlNet and inpainting tasks.)

How to use

1. First clone the container and start the container according to the steps

2. Copy the generated API address to your browser and use it

3. Three ways to use

HiDiffusion supports the following three methods, each of which corresponds to a model. When using a method, the model is loaded first, and then the image is generated. If you switch methods, the model will be reloaded.

3.1 Method 1: Text to Image

Generate images by inputting positive prompt words and reverse prompt words.

Positive prompt words: describes what you want to see in the image,

For example: Standing tall amidst the ruins, a stone golem awakens, vines and flowers sprouting from the crevices in its body.

Reverse prompt word: Used to describe content that is not desired to appear in the image, and to optimize the generated results by excluding unnecessary elements.

For example: blurry, ugly, duplicate, poorly drawn face, deformed, mosaic, artifacts, bad limbs.

The usage steps and generation effects are shown in the figure below

3.2 Method 2: Using ControlNet to generate graphs

ControlNet: Generate an image based on the outline of the original image according to the prompt word. First, extract the outline of the original image, and then generate it based on this outline. The operation is shown in the figure below

3.3 Method 3: Image Inpainting

Original Image: Input the image to be repaired

Repair area: Input the area to be repaired, which is actually a binary image, where the white area is the area to be repaired, and the black area is the frozen area. When repairing, modify the white area according to the prompts of Zheng Dan!

For example, the positive prompt words are: A steampunk explorer in a leather aviator cap and goggles, with a brass telescope in hand, stands amidst towering ancient trees, their roots entwined with intricate gears and pipes.

The reverse prompt words are: blurry, ugly, duplicate, poorly drawn face, deformed, mosaic, artifacts, bad limbs.