Model quantization is a promising approach to compress deep neural networksand accelerate inference, making it possible to be deployed on mobile and edgedevices. To retain the high performance of full-precision models, most existingquantization methods focus on fine-tuning quantized model by assuming trainingdatasets are accessible. However, this assumption sometimes is not satisfied inreal situations due to data privacy and security issues, thereby making thesequantization methods not applicable. To achieve zero-short model quantizationwithout accessing training data, a tiny number of quantization methods adopteither post-training quantization or batch normalization statistics-guided datageneration for fine-tuning. However, both of them inevitably suffer from lowperformance, since the former is a little too empirical and lacks trainingsupport for ultra-low precision quantization, while the latter could not fullyrestore the peculiarities of original data and is often low efficient fordiverse data generation. To address the above issues, we propose a zero-shotadversarial quantization (ZAQ) framework, facilitating effective discrepancyestimation and knowledge transfer from a full-precision model to its quantizedmodel. This is achieved by a novel two-level discrepancy modeling to drive agenerator to synthesize informative and diverse data examples to optimize thequantized model in an adversarial learning fashion. We conduct extensiveexperiments on three fundamental vision tasks, demonstrating the superiority ofZAQ over the strong zero-shot baselines and validating the effectiveness of itsmain components. Code is available at https://git.io/Jqc0y.