RWKV – A Powerful Combination of Transformer and RNN

This article will introduce a new network architecture that combines the advantages of both RNN and transformer - RWKV.