HyperAI

Language Modelling On Openwebtext

Metrics

eval_loss
eval_perplexity
parameters

Results

Performance results of various models on this benchmark

Model Name
eval_loss
eval_perplexity
parameters
Paper TitleRepository
GPT2-Tropical2.9218.64124MLearnable polynomial, trigonometric, and tropical activations
GPT2-GELU2.9519.24124MLearnable polynomial, trigonometric, and tropical activations
GPT2-Fourier2.9318.72124MLearnable polynomial, trigonometric, and tropical activations
GPT2-Hermite2.9118.39124MLearnable polynomial, trigonometric, and tropical activations
0 of 4 row(s) selected.