In 3D human pose estimation one of the biggest problems is the lack of large,diverse datasets. This is especially true for multi-person 3D pose estimation,where, to our knowledge, there are only machine generated annotations availablefor training. To mitigate this issue, we introduce a network that can betrained with additional RGB-D images in a weakly supervised fashion. Due to theexistence of cheap sensors, videos with depth maps are widely available, andour method can exploit a large, unannotated dataset. Our algorithm is amonocular, multi-person, absolute pose estimator. We evaluate the algorithm onseveral benchmarks, showing a consistent improvement in error rates. Also, ourmodel achieves state-of-the-art results on the MuPoTS-3D dataset by aconsiderable margin.