Analyzing energy consumption during the training of AI models is crucial for optimizing efficiency and reducing environmental impact. However, the skewed nature of energy consumption data often requires statistical techniques to improve model fit. In a recent project, the author encountered this challenge while working with data from Epoch AI, focusing on the energy consumption (in kWh) of various AI models. The data was right-skewed and showed extreme outliers (Fig. 1). To address this, the author considered log transformations and log link functions within Generalized Linear Models (GLMs) to normalize the distribution and stabilize variance. The initial approach involved log-transforming the energy consumption variable (log(Energy)). The resulting distribution appeared more normal (Fig. 2), and the Shapiro-Wilk test confirmed this with a p-value of approximately 0.5. Four models were then fitted using R's GLM framework: 1. Gaussian distribution with log link (log-linked Gaussian) 2. Gaussian distribution with log-transformed response (log-transformed Gaussian) 3. Gamma distribution with log link (log-linked Gamma) 4. Gamma distribution with log-transformed response (log-transformed Gamma) Model comparison using the Akaike Information Criterion (AIC) indicated that the log-transformed models had significantly lower AIC values: - Log-transformed Gaussian: AIC = 311.5963 - Log-linked Gaussian: AIC = 1780.8524 - Log-transformed Gamma: AIC = 352.5450 - Log-linked Gamma: AIC = 1775 The diagnostic plots further supported the selection of log-transformed models. For the log-linked Gaussian model, the Residuals vs Fitted plot suggested linearity, but the Q-Q plot showed non-normality (Fig. 4). In contrast, the log-transformed Gaussian model had a better Q-Q plot, indicating normality, though the Residuals vs Fitted plot showed a dip to -2, hinting at potential non-linearity (Fig. 5). The log-linked Gamma model, while having acceptable Q-Q plots, exhibited clear signs of non-linearity in the Residuals vs Fitted plot (Fig. 6). Conversely, the log-transformed Gamma model had a relatively good Residuals vs Fitted plot, with only a small dip (Fig. 7). Based on these findings, the author initially chose the log-transformed Gamma model. However, when interpreting the coefficients, the results seemed implausible. The slopes for continuous variables like training time and hardware quantity were almost zero, and the intercepts for different hardware types were around 1 kWh, which did not align with the observed high energy consumption. Switching to the log-linked Gamma model, the author found more sensible results. The coefficients for training time and hardware quantity indicated a multiplicative effect on energy consumption, increasing by 0.18% and 0.07% per additional hour and chip, respectively. The interaction term, however, showed a slight decrease in energy use by 2.651 × 10⁻⁷% per additional chip-hour combination. To visualize these differences, two plots comparing the predictions from both models were generated (Fig. 8). The log-transformed Gamma model's predictions (left panel) were nearly flat and did not align with the actual data. In contrast, the log-linked Gamma model's predictions (right panel) closely matched the fitted lines of the raw data, confirming its better fit. Why Log Transformation Fails The key issue lies in the fundamental differences between log transformation and log link functions. When applying a log transformation to the response variable, the model effectively compresses both the true relationship (f(X)) and the error term. This transformation creates a new response variable (log(Y)), which the model tries to predict using the input function (g(X)). As a result, the model captures a distorted version of the original relationship, leading to unreliable coefficient estimates. On the other hand, a log link function keeps the original response variable (Y) intact. Instead, it exponentiates the input function (g(X)) to predict Y. This maintains the error terms on the original scale, allowing the model to minimize the difference between the actual and predicted values of Y. Therefore, the log-linked Gamma model provides a more accurate representation of the data's underlying relationships. Industry Insights and Company Profiles Industry experts emphasize the importance of choosing the correct model to avoid misleading interpretations, especially when dealing with highly skewed data. The selection of the log-linked Gamma model in this case highlights the significance of understanding the mathematical underpinnings of statistical techniques. Epoch AI, a leading resource in AI benchmarking, provides valuable datasets that help researchers and practitioners optimize the energy efficiency of AI models. The success of this project underscores the critical role of statistical rigor in ensuring that AI technologies are sustainable and eco-friendly.