FOF: Learning Fourier Occupancy Field for Monocular Real-time Human Reconstruction

The advent of deep learning has led to significant progress in monocularhuman reconstruction. However, existing representations, such as parametricmodels, voxel grids, meshes and implicit neural representations, havedifficulties achieving high-quality results and real-time speed at the sametime. In this paper, we propose Fourier Occupancy Field (FOF), a novelpowerful, efficient and flexible 3D representation, for monocular real-time andaccurate human reconstruction. The FOF represents a 3D object with a 2D fieldorthogonal to the view direction where at each 2D position the occupancy fieldof the object along the view direction is compactly represented with the firstfew terms of Fourier series, which retains the topology and neighborhoodrelation in the 2D domain. A FOF can be stored as a multi-channel image, whichis compatible with 2D convolutional neural networks and can bridge the gapbetween 3D geometries and 2D images. The FOF is very flexible and extensible,e.g., parametric models can be easily integrated into a FOF as a prior togenerate more robust results. Based on FOF, we design the first 30+FPShigh-fidelity real-time monocular human reconstruction framework. Wedemonstrate the potential of FOF on both public dataset and real captured data.The code will be released for research purposes.