One-sentence Summary

By adapting conditional generative adversarial networks for 3D model generation and introducing a training-step guidance mechanism, the authors enable the consistent synthesis of paired 3D models at controllable rotation angles without modifying the standard architecture, as demonstrated by experimental results and visual comparisons.

Key Contributions

• This work adapts conditional generative adversarial networks to automate the generation of 3D voxel models across varying rotation angles.
• A novel training guidance mechanism enforces cross-condition consistency, enabling the network to output identical 3D samples at different controllable rotations without modifying the standard conditional GAN architecture.
• Experimental results and visual comparisons demonstrate successful generation of paired 3D models. The framework additionally supports flexible training using paired, unpaired, or non-corresponding sample sets.

Introduction

Automated 3D model generation streamlines digital content creation by bypassing the constraints of manual scanning and labor-intensive design. While Conditional Generative Adversarial Networks enable controlled generation, they suffer from latent entanglement that causes the model to produce entirely different objects when conditions change. Existing solutions for generating consistent paired outputs typically require custom network architectures, complex modeling, or strictly paired training datasets. The authors leverage a novel training strategy that integrates directly into standard CGAN pipelines to overcome these limitations. This approach guides the network to generate identical 3D voxel models across controllable rotation angles without modifying the underlying architecture, effectively enabling consistent paired generation using paired, unpaired, or domain-independent data.

Method

The authors leverage the conditional generative adversarial network (CGAN) framework as the foundation for their method, extending it with an additional training step to address the challenge of generating consistent, paired outputs under varying conditions. The overall system operates within a min-max game between a generator $G$G and a discriminator $D$D, where both models are trained simultaneously. The core architecture is based on a conditional GAN, as shown in the framework diagram below, where the generator receives both a random noise vector $\mathbf{z}$z and a condition value $\mathbf{c}$c as input. This conditioning allows the generator to produce samples tailored to specific attributes, such as different rotations of a 3D object.

The generator function is defined as $G(\mathbf{z}|\mathbf{c})$G(z∣c), taking the input vector $\mathbf{z}$z and condition $\mathbf{c}$c to produce a generated sample. The discriminator $D$D is similarly conditioned, receiving both the sample and the condition $\mathbf{c}$c to evaluate its authenticity. The standard CGAN objective function is formulated as a minimax game, where the generator aims to minimize the log probability of the discriminator correctly identifying its outputs as fake, while the discriminator aims to maximize its ability to distinguish real from generated samples.

To overcome the limitation of standard CGANs, which generate different independent samples for the same input vector $\mathbf{z}$z under different conditions, the authors introduce a novel training step. This step is designed to enforce the generation of similar, consistent samples for the same input across multiple conditions. The process, illustrated in the diagram below, involves generating $n$n samples $S_0, S_1, \cdots, S_n$S0​,S1​,⋯,Sn​ using the same input vector $\mathbf{z}$z but different condition values $\mathbf{y}_0, \mathbf{y}_1, \cdots, \mathbf{y}_n$y0​,y1​,⋯,yn​. These samples are then processed by a domain-specific merge operator $M$M, which first aligns the generated samples and then combines them, typically by averaging voxel values, to produce a single merged output.

This merged sample is fed into the discriminator to assess its realism. The key innovation lies in the objective function, which incorporates this merge step as an additional term. The total objective function becomes the sum of the standard CGAN loss and a new term that penalizes the generator if the merged result is not realistic. This forces the generator to not only produce realistic samples but also to ensure that these samples are highly similar for the same input vector across different conditions, thereby enabling the creation of paired or consistent outputs. The training algorithm iterates through both the standard CGAN updates and this new merge-based update to refine the model.

Experiment

The evaluation utilized the ModelNet dataset to generate 3D voxel models for chairs, beds, and sofas, testing both a standard conditional GAN and the proposed method across two- and four-rotation conditions. These experiments validated the models' ability to maintain structural consistency when generating objects under different rotational inputs. Qualitatively, the baseline network consistently produced unrelated shapes for identical inputs across conditions, whereas the proposed framework successfully generated highly similar models that aligned coherently when merged. While increased rotational complexity introduced minor noise in the four-condition setup, the proposed method fundamentally outperformed the baseline by reliably enforcing cross-condition consistency throughout training.

The authors compare their proposed method with a baseline conditional GAN for generating 3D models under different rotational conditions. Results show that the proposed method produces more consistent outputs across rotations compared to the baseline, which generates significantly different models for the same object class under different orientations. The proposed approach achieves higher similarity between generated samples, as measured by both average absolute difference and voxel agreement ratio. The proposed method generates more consistent 3D models across different rotations compared to the baseline method. The proposed method achieves higher similarity between generated samples, as indicated by improved average absolute difference and voxel agreement ratio metrics. The baseline method produces significantly different models for the same object under different orientations, while the proposed method maintains consistency.

The authors compare their proposed method with a baseline conditional GAN for generating 3D models under different rotational conditions. Results show that the proposed method generates more consistent models across rotations compared to the baseline, as evidenced by lower average absolute differences and higher voxel agreement ratios. The improvement is more pronounced for certain object classes, with consistent performance across both two- and four-condition setups. The proposed method generates more consistent 3D models across different rotations compared to the baseline method. The proposed method achieves lower average absolute differences and higher voxel agreement ratios, indicating better similarity between generated models. The improvement in consistency is more significant for some object classes, with the proposed method maintaining consistent outputs across multiple rotational conditions.

The experiments evaluate the proposed method against a baseline conditional GAN for generating 3D models under varying rotational conditions to validate rotational consistency and structural stability. Results demonstrate that the proposed approach maintains highly consistent outputs across different orientations, whereas the baseline produces significantly divergent models for identical objects. This rotational invariance holds across multiple experimental setups and proves particularly robust for specific object classes, confirming the method's superior ability to preserve geometric coherence regardless of viewing angle.