HyperAI초신경

홈 뉴스 최신 연구 논문 튜토리얼 데이터셋 백과사전 SOTA LLM 모델 GPU 랭킹 컨퍼런스

한국어

HyperAI초신경

Language Modelling On Hutter Prize

평가 지표

Bit per Character (BPC)

Number of params

평가 결과

이 벤치마크에서 각 모델의 성능 결과

모델 이름	Bit per Character (BPC)	Number of params	Paper Title	Repository
Large RHN	1.27	46M	Recurrent Highway Networks
Large FS-LSTM-4	1.245	47M	Fast-Slow Recurrent Neural Networks
Transformer-XL + RMS dynamic eval	0.94	277M	Dynamic Evaluation of Transformer Language Models
18-layer Transformer-XL	1.03	88M	Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
Large mLSTM +emb +WN +VD	1.24	46M	Multiplicative LSTM for sequence modelling	-
Mogrifier LSTM	1.122	96M	Mogrifier LSTM
12-layer Transformer-XL	1.06	41M	Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
64-layer Character Transformer Model	1.06	235M	Character-Level Language Modeling with Deeper Self-Attention
3-layer AWD-LSTM	1.232	47M	An Analysis of Neural Language Modeling at Multiple Scales
Longformer Small	1.00	41M	Longformer: The Long-Document Transformer
12-layer Character Transformer Model	1.11	44M	Character-Level Language Modeling with Deeper Self-Attention
FS-LSTM-4	1.277	27M	Fast-Slow Recurrent Neural Networks
mLSTM + dynamic eval	1.08	46M	Dynamic Evaluation of Neural Sequence Models
Longformer Large	0.99	102M	Longformer: The Long-Document Transformer
RHN - depth 5 [zilly2016recurrent]	1.31	-	Recurrent Highway Networks
24-layer Transformer-XL	0.99	277M	Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
Compressive Transformer	0.97	-	Compressive Transformers for Long-Range Sequence Modelling
Mogrifier LSTM + dynamic eval	0.988	96M	Mogrifier LSTM

0 of 18 row(s) selected.