Transformer-Capsule Model for Intent Detection

Intent recognition is one of the most crucial tasks in NLUsystems, which are nowadays especially important for design-ing intelligent conversation. We propose a novel approach to intent recognition which involves combining transformer architecture with capsule networks. Our results show that such architecture performs better than original capsule-NLU net-work implementations and achieves state-of-the-art results on datasets such as ATIS, AskUbuntu , and WebApp.