Researchers have introduced CODA, a new GPU kernel abstraction designed to optimize the performance of Transformer training systems. While modern deep learning frameworks rely heavily on dense linear algebra, a significant portion of training time is consumed by memory-bound operations surrounding the core computation. These operations, including normalization, activation functions, residual updates, and reductions, involve moving large intermediate tensors through global memory while performing relatively few arithmetic calculations. This pattern has emerged as a critical bottleneck in otherwise highly optimized training stacks. CODA addresses this inefficiency by reparameterizing these specific computations as GEMM-plus-epilogue programs. The approach is based on the observation that many Transformer operators, typically implemented as separate framework kernels, can be algebraically transformed to execute while a GEMM output tile remains on the GPU chip. By keeping data on-chip for these subsequent operations before writing to memory, the system significantly reduces the latency associated with global memory access. The abstraction works by fixing the GEMM main loop, which handles the primary matrix multiplication, and exposing a small set of composable epilogue primitives. These primitives allow for scaling, reductions, pairwise transformations, and accumulation to occur seamlessly within the GEMM structure. This constrained interface is designed to preserve the high performance of expert-written GEMM kernels while remaining flexible enough to cover nearly all non-attention computation in both the forward and backward passes of a standard Transformer block. Testing across representative Transformer workloads demonstrated that both human-engineered and Large Language Model-authored CODA kernels achieve high performance. These results suggest that the GEMM-plus-epilogue programming model offers a practical pathway to combine the productivity of high-level frameworks with the efficiency of low-level hardware optimization. The study, available on arXiv, highlights the importance of rethinking how auxiliary operations are executed in AI training pipelines. As models grow larger and more complex, the relative cost of data movement continues to rise, making the optimization of these non-matrix-multiplication operations increasingly vital. By integrating these computations directly into the GEMM workflow, CODA minimizes redundant memory transfers and maximizes the utilization of GPU compute resources. This innovation has the potential to streamline the development of future AI models by reducing training times and lowering the barrier for achieving hardware-level efficiency. The method allows developers to focus on model architecture without sacrificing the performance benefits that usually require deep, manual kernel optimization. As the field of machine learning continues to evolve, approaches like CODA represent a necessary evolution in system design, ensuring that training infrastructure keeps pace with the demands of next-generation large language models.