This paper presents Model-guidance (MG), a novel objective for trainingdiffusion model that addresses and removes of the commonly used Classifier-freeguidance (CFG). Our innovative approach transcends the standard modeling ofsolely data distribution to incorporating the posterior probability ofconditions. The proposed technique originates from the idea of CFG and is easyyet effective, making it a plug-and-play module for existing models. Our methodsignificantly accelerates the training process, doubles the inference speed,and achieve exceptional quality that parallel and even surpass concurrentdiffusion models with CFG. Extensive experiments demonstrate the effectiveness,efficiency, scalability on different models and datasets. Finally, we establishstate-of-the-art performance on ImageNet 256 benchmarks with an FID of 1.34.Our code is available at https://github.com/tzco/Diffusion-wo-CFG.