A-Lamp: Adaptive Layout-Aware Multi-Patch Deep Convolutional Neural Network for Photo Aesthetic Assessment

Deep convolutional neural networks (CNN) have recently been shown to generatepromising results for aesthetics assessment. However, the performance of thesedeep CNN methods is often compromised by the constraint that the neural networkonly takes the fixed-size input. To accommodate this requirement, input imagesneed to be transformed via cropping, warping, or padding, which often alterimage composition, reduce image resolution, or cause image distortion. Thus theaesthetics of the original images is impaired because of potential loss of finegrained details and holistic image layout. However, such fine grained detailsand holistic image layout is critical for evaluating an image's aesthetics. Inthis paper, we present an Adaptive Layout-Aware Multi-Patch ConvolutionalNeural Network (A-Lamp CNN) architecture for photo aesthetic assessment. Thisnovel scheme is able to accept arbitrary sized images, and learn from bothfined grained details and holistic image layout simultaneously. To enabletraining on these hybrid inputs, we extend the method by developing a dedicateddouble-subnet neural network structure, i.e. a Multi-Patch subnet and aLayout-Aware subnet. We further construct an aggregation layer to effectivelycombine the hybrid features from these two subnets. Extensive experiments onthe large-scale aesthetics assessment benchmark (AVA) demonstrate significantperformance improvement over the state-of-the-art in photo aestheticassessment.