LLM Based on First Principles, a New Training Paradigm POET

Reparameterized Training via Orthogonal Equivalence Transformation (POET) is a novel reparameterized training algorithm proposed by the Max Planck Institute in Germany and the Chinese University of Hong Kong on June 9, 2025. It uses orthogonal equivalence transformation to optimize neurons. The related paper results are "Reparameterized LLM Training via Orthogonal Equivalence Transformation".

POET works by reparameterizing each neuron using two learnable orthogonal matrices and a fixed random weight matrix. Because POET provably preserves the spectral properties of the weight matrix, it can stably optimize the objective function and improve generalization. The research team developed efficient approximation methods that make POET flexible and scalable for training large-scale neural networks.