Automatic age and gender classification based on unconstrained images hasbecome essential techniques on mobile devices. With limited computing power,how to develop a robust system becomes a challenging task. In this paper, wepresent an efficient convolutional neural network (CNN) called lightweightmulti-task CNN for simultaneous age and gender classification. Lightweightmulti-task CNN uses depthwise separable convolution to reduce the model sizeand save the inference time. On the public challenging Adience dataset, theaccuracy of age and gender classification is better than baseline multi-taskCNN methods.