One-sentence Summary

Researchers from indico Research and Facebook AI Research introduce DCGANs, a class of deep convolutional generative adversarial networks with specific architectural constraints that learn a hierarchy of representations from object parts to scenes and yield general-purpose image features applicable to novel tasks.

Key Contributions

• The paper introduces Deep Convolutional GANs, adding architectural constraints to convolutional GANs to stabilize training and connect CNN success in supervised vision with unsupervised representation learning.

• Experiments on image datasets show that the trained generator and discriminator learn hierarchical features from object parts to scenes, and discriminator features can serve as competitive general image representations for classification.

• The paper further shows that GAN filters correspond to recognizable objects and that generator latent arithmetic allows semantic manipulation of generated samples, supporting the usefulness of the learned features.

Introduction

Learning reusable features from large unlabeled image datasets is valuable in computer vision because it can reduce the need for labeled data and improve supervised tasks such as image classification. Generative Adversarial Networks are an attractive generative alternative, but prior GAN approaches were often unstable, produced noisy or blurry samples, and gave little insight into the internal representations learned by the network. The authors address this by introducing Deep Convolutional GANs, a constrained architecture that makes convolutional GAN training more stable. They leverage the trained discriminator as an unsupervised feature extractor for classification, visualize learned filters to reveal meaningful object patterns, and show that the generator supports semantic manipulations through latent space arithmetic.

Dataset

• The authors use three image datasets for unsupervised training of DCGANs: LSUN bedrooms, a newly assembled Faces dataset, and ImageNet-1k. No train/test split is described; the data are used as training examples.

• Common preprocessing is minimal. Images are scaled to the tanh activation range, and no data augmentation is applied for any dataset.

• LSUN bedrooms provides a little over 3 million training examples. The authors use it to show scaling and high-resolution generation, with samples from one epoch and after convergence. They reduce memorization risk through image deduplication: an autoencoder is fit on 32x32 downsampled center crops, latent activations are binarized by thresholding ReLU, and about 275,000 near duplicates are removed.

• The Faces dataset is built by scraping web images from random queries of people’s names. Names are taken from dbpedia, restricted to people born in the modern era. The initial set contains 3 million images from 10,000 people. The authors run an OpenCV face detector and keep sufficiently high-resolution detections, yielding about 350,000 face boxes used for training.

• ImageNet-1k is used as a source of natural images. For training, the authors use 32x32 min-resized center crops. No augmentation is applied.

Method

The authors leverage a deep convolutional generative adversarial network designed for stable image generation at higher resolutions than earlier convolutional GAN approaches could reliably achieve. The model follows the adversarial training framework, where the generator learns to map latent noise to synthetic images and the discriminator learns to separate generated images from real images. The core optimization problem can be written as

where $z$z is sampled from a latent distribution and $G(z)$G(z) is the generated image.

The architecture is built around three main design choices: learned resampling, removal of hidden fully connected layers, and normalization. For the generator, deterministic pooling is replaced with fractional strided convolutions so that the network learns its own spatial upsampling. For the discriminator, spatial downsampling is performed with strided convolutions rather than fixed pooling. The input noise vector is projected by a linear layer and reshaped into a four dimensional tensor, which becomes the beginning of the convolutional stack. The discriminator flattens its final convolutional feature map and maps it to a sigmoid output. Refer to the framework diagram of a generator for scene modeling:

As shown in the figure below:

The illustrated generator takes a 100 dimensional uniform latent vector, projects it into a $4 \times 4$4×4 feature representation with 1024 feature maps, and expands it through fractional strided convolutions into a $64 \times 64$64×64 RGB image. Notably, the generator does not use pooling layers or hidden fully connected layers. This fully convolutional structure allows the network to learn spatial upsampling directly from data.

Batch normalization is applied to most convolutional layers in both the generator and discriminator to stabilize gradient flow and reduce sensitivity to initialization. However, batch normalization is intentionally omitted from the generator output layer and the discriminator input layer, because applying it directly to those layers was found to cause oscillation and instability. In the generator, ReLU activations are used for internal layers, while the output layer uses $\tanh$tanh so that generated pixels fall within the target image range. In the discriminator, LeakyReLU activations are used throughout, with a leak slope of 0.2.

The adversarial training pipeline begins with minimal image preprocessing. Training images are resized and scaled into the range expected by the $\tanh$tanh output activation. For face data, candidate images are processed with a face detector, and only sufficiently high resolution detections are retained as training samples. For large bedroom image data, the authors also use a deduplication pipeline before training to reduce the chance that the generator memorizes repeated training examples.

The deduplication workflow fits an autoencoder with a 128 dimensional code on downsampled center crops. The code layer activations are binarized by thresholding the ReLU activation, producing compact binary signatures for each image. These signatures are used for efficient semantic hashing and near duplicate removal. This filtering step improves the quality of the training pool by suppressing redundant examples that could otherwise encourage shortcut learning.

For optimization, all models are trained with stochastic gradient descent using batches of 128 examples. The Adam optimizer is used with a learning rate of 0.0002, while the first moment coefficient is reduced from 0.9 to 0.5 to improve stability. Network weights are initialized from a normal distribution with standard deviation 0.02. No data augmentation is applied during the core training runs.

Training progress is monitored by comparing generated samples at early and later stages. The early samples already reveal recognizable scene structure despite being produced after only one pass through the data. As shown by the generated bedrooms after one training pass:

After continued adversarial training, the samples become more coherent and visually refined, although some local artifacts can remain. The later generated bedrooms after additional epochs are shown:

Experiment

The experiments evaluate the model on a large, high-resolution bedroom dataset to assess scaling and memorization, and test its learned features through transfer classification on CIFAR-10 and SVHN. These evaluations show that strong generation is not simply due to overfitting, that the representations transfer across domains and remain useful in low-label settings, and that performance is not explained by the network architecture alone. Additional latent-space, feature visualization, and generator manipulation analyses reveal smooth, semantically organized representations that support interpretable editing of objects and attributes.

The table compares feature-based methods for CIFAR-10 classification, including DCGAN features learned on ImageNet and applied to CIFAR-10 without CIFAR-10 pretraining. The DCGAN plus L2-SVM setup is competitive with baselines while using comparatively few feature units, particularly in the 400-per-class setting. Exemplar CNN has the highest overall accuracy, and the DCGAN method performs better than the K-means baselines in that setting. ImageNet-trained DCGAN features transfer to CIFAR-10 classification. DCGAN plus L2-SVM uses fewer feature units than the K-means baselines. Exemplar CNN achieves the strongest overall accuracy.

The SVHN classification results with 1000 labels show that the SWWAE models achieve substantially lower error rates than the KNN, TSVM, and intermediate combinations. Adding dropout to SWWAE further reduces the error rate, indicating an improvement over the variant without dropout. SWWAE with dropout has the lowest error rate among the reported models SWWAE without dropout still outperforms the baseline and intermediate methods Combining M1 with KNN or TSVM gives moderate gains over the original KNN and TSVM baselines M1+M2 improves over M1+TSVM but remains less accurate than the SWWAE variants

The experiments compare transferable feature representations for CIFAR-10 and probabilistic models for SVHN. On CIFAR-10, ImageNet-trained DCGAN features combined with an L2-SVM are competitive with baselines despite using fewer feature units and outperform K-means methods in the limited-label setting, although Exemplar CNN remains the strongest overall. On SVHN, SWWAE models substantially outperform KNN, TSVM, and their intermediate combinations, with dropout providing further improvement.