Length-Aware DETR for Robust Moment Retrieval

Video Moment Retrieval (MR) aims to localize moments within a video based ona given natural language query. Given the prevalent use of platforms likeYouTube for information retrieval, the demand for MR techniques issignificantly growing. Recent DETR-based models have made notable advances inperformance but still struggle with accurately localizing short moments.Through data analysis, we identified limited feature diversity in shortmoments, which motivated the development of MomentMix. MomentMix employs twoaugmentation strategies: ForegroundMix and BackgroundMix, each enhancing thefeature representations of the foreground and background, respectively.Additionally, our analysis of prediction bias revealed that short momentsparticularly struggle with accurately predicting their center positions ofmoments. To address this, we propose a Length-Aware Decoder, which conditionslength through a novel bipartite matching process. Our extensive studiesdemonstrate the efficacy of our length-aware approach, especially in localizingshort moments, leading to improved overall performance. Our method surpassesstate-of-the-art DETR-based methods on benchmark datasets, achieving thehighest R1 and mAP on QVHighlights and the highest [email protected] on TACoS andCharades-STA (such as a 2.46% gain in [email protected] and a 2.57% gain in mAP averagefor QVHighlights). The code is available athttps://github.com/sjpark5800/LA-DETR.