URochester and NVIDIA Develop PixelDiT2 for Pixel-Level Image Control

Researchers at the University of Rochester and NVIDIA have developed PixelDiT2, a new image-generation model that operates directly in pixel space rather than the compressed latent space used by most modern text-to-image systems. The work addresses a common limitation of leading artificial intelligence image generators: while latent-space diffusion models are computationally efficient, they offer limited fine-grained control, making it difficult to correct small details without unintended changes elsewhere in the image. Led by Jiebo Luo, the Albert Arendt Hopeman Professor of Engineering at the University of Rochester, the team previously introduced PixelDiT at the 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition. That model allowed users to edit targeted regions, such as a person’s glasses or a car’s color, by manipulating individual pixels. PixelDiT2 extends the approach with improved efficiency, achieving performance nearly comparable to state-of-the-art latent-space diffusion models while preserving direct pixel-level controllability. The latest findings are being presented at the Neural Information Processing Systems conference in December and have been published on arXiv. The collaboration includes computer science Ph.D. student and NVIDIA intern Yongsheng Yu and NVIDIA senior researcher Wei Xiong, a 2022 University of Rochester computer science alumnus. By removing the abstraction layer between generation and final image pixels, the researchers aim to make AI image editing more precise and predictable. The team is now exploring how the same pixel-space approach can be applied to video generation, where temporal consistency across frames and larger data volumes present major challenges. Successful development could enable more controllable AI video creation with frame-level precision.