Language Modelling On Penn Treebank Character

Metrics

Bit per Character (BPC)

Number of params

Results

Performance results of various models on this benchmark

			Paper Title
Bipartite Flow	1.38	-	Discrete Flows: Invertible Generative Models of Discrete Data
TCN	1.31	5.9M	Seq-U-Net: A One-Dimensional Causal U-Net for Efficient Sequence Modelling
Temporal Convolutional Network	1.31	-	An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
Seq-U-Net	1.3	5.9M	Seq-U-Net: A One-Dimensional Causal U-Net for Efficient Sequence Modelling
STAR	1.30	-	Gating Revisited: Deep Multi-layer RNNs That Can Be Trained
R-Transformer	1.24	-	R-Transformer: Recurrent Neural Network Enhanced Transformer
2-layer Norm HyperLSTM	1.219	14.4M	HyperNetworks
NAS-RL	1.214	16.3M	Neural Architecture Search with Reinforcement Learning
FS-LSTM-2	1.193	27M	Fast-Slow Recurrent Neural Networks
IndRNN	1.19	-	Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN
FS-LSTM-4	1.190	27M	Fast-Slow Recurrent Neural Networks
6-layer QRNN	1.187	13.8M	An Analysis of Neural Language Modeling at Multiple Scales
Dense IndRNN	1.18	-	Deep Independently Recurrent Neural Network (IndRNN)
3-layer AWD-LSTM	1.175	13.8M	An Analysis of Neural Language Modeling at Multiple Scales
Past Decode Reg. + AWD-LSTM-MoS + dyn. eval.	1.169	13.8M	Improved Language Modeling by Decoding the Past
Feedback Transformer	1.160	10.7M	Addressing Some Limitations of Transformers with Feedback Memory
Trellis Network	1.158	13.4M	Trellis Networks for Sequence Modeling
GAM-RHN-5	1.147	16.0M	Recurrent Highway Networks with Grouped Auxiliary Memory
Mogrifier LSTM	1.120	24M	Mogrifier LSTM
Mogrifier LSTM + dynamic eval	1.083	24M	Mogrifier LSTM

0 of 20 row(s) selected.

Command Palette

Language Modelling On Penn Treebank Character

Metrics

Results