We present an approach which takes advantage of both structure and semanticsfor unsupervised monocular learning of depth and ego-motion. More specifically,we model the motion of individual objects and learn their 3D motion vectorjointly with depth and ego-motion. We obtain more accurate results, especiallyfor challenging dynamic scenes not addressed by previous approaches. This is anextended version of Casser et al. [AAAI'19]. Code and models have been opensourced at https://sites.google.com/corp/view/struct2depth.