Command Palette
Search for a command to run...
PROLA: Principal-Orthogonale Niedrigrang-Adaption für prädiktives räumlich-zeitliches Wetter-Downscaling
PROLA: Principal-Orthogonale Niedrigrang-Adaption für prädiktives räumlich-zeitliches Wetter-Downscaling
Minseo Yoon Minseong Bae Sojin Lee Hyunwoo J. Kim
Zusammenfassung
Hochauflösende Wettervorhersagen sind wichtig, um lokale atmosphärische Muster zu erfassen, die von groben globalen Prognosen übersehen werden. Große vortrainierte Wetter-Backbones haben die globale Vorhersage verbessert, doch ihre Anpassung zur Erzeugung künftiger hochauflösender Felder aus jüngsten groben Zuständen bleibt herausfordernd und kostspielig. Wir formulieren dieses Szenario als Predictive Spatiotemporal Weather Downscaling (PSWD), eine Aufgabe, die räumliche und zeitliche Auflösung gemeinsam verfeinert und zugleich künftige hochauflösende Trajektorien vorhersagt. Zu diesem Zweck schlagen wir ein Framework vor, das die Vorhersage mithilfe eines gemeinsamen vortrainierten Backbones, fester numerischer Gerüste und leichtgewichtiger Residualköpfe in räumliche und zeitliche Stufen zerlegt. Für die Backbone-Adaption führen wir PROLA (PRincipal-Orthogonal Low-rank Adaptation) ein, das ein festes Niedrigrang-Budget zwischen den Hauptrichtungen der vortrainierten Gewichte und ihrem orthogonalen Komplement aufteilt. PROLA skaliert darüber hinaus Rang-1-Optimierer-Updates mithilfe von Gradientensignalen neu und verbessert so die Adaption, ohne den trainierbaren Rang zu erhöhen. Experimente im PSWD-Szenario zeigen, dass PROLA repräsentative Niedrigrang-Adaptions-Baselines übertrifft. Weitere Experimente über verschiedene Datensätze und Aufgaben hinweg stützen die breitere Anwendbarkeit von PROLA und zeigen, dass seine Vorteile über das primäre Wetter-Downscaling-Szenario hinausgehen.
One-sentence Summary
Researchers at Korea Advanced Institute of Science and Technology propose PROLA, a principal-orthogonal low-rank adaptation that allocates a fixed low-rank budget between pretrained weights' principal directions and their orthogonal complement while rescaling rank-1 optimizer updates using gradient signals, thereby improving predictive spatiotemporal weather downscaling and outperforming representative low-rank adaptation baselines across diverse datasets and tasks.
Key Contributions
- The paper formalizes Predictive Spatiotemporal Weather Downscaling (PSWD) as predicting future high-resolution, temporally denser weather trajectories from recent coarse-resolution global states, and introduces a spatial-to-temporal scaffold-and-correct pipeline using a shared pretrained backbone with fixed numerical scaffolds and lightweight residual heads.
- It proposes PROLA (Principal-Orthogonal Low-rank Adaptation), which splits a fixed low-rank budget between pretrained weights' principal directions and their orthogonal complement, and rescales rank-1 optimizer updates using gradient signals without increasing trainable rank.
- Experiments show PROLA outperforms representative low-rank adaptation baselines under matched low-rank budgets, and further tests across diverse datasets and tasks support broader applicability beyond weather downscaling.
Introduction
Weather modeling has largely separated forecasting, spatial downscaling, and temporal downscaling, but applications such as minute-scale wind-power forecasting, sub-hourly precipitation nowcasting, and aviation turbulence guidance require predicting future states at both finer spatial and temporal resolution from low-resolution inputs. Prior work has not addressed this joint regime, and adapting large weather foundation models to richer output grids is constrained by the cost of full fine-tuning. The authors formalize this setting as Predictive Spatiotemporal Weather Downscaling (PSWD), introduce a scaffold-and-correct pipeline built around a shared pretrained backbone, and propose PROLA, a parameter-efficient low-rank adaptation method that splits a fixed rank budget between principal and orthogonal branches while rescaling optimizer updates with gradient-based scores. Experiments on WeatherBench 2 show that PROLA improves over representative low-rank PEFT baselines under matched budgets and narrows the gap to full fine-tuning.
Method
The authors formulate Predictive Spatiotemporal Weather Downscaling as a coupled spatiotemporal prediction task that jointly requires forecasting, spatial downscaling, and temporal downscaling. Given coarse-step past and present low-resolution inputs, the goal is to predict fine-step future high-resolution targets. To address the challenges of this coupled shift, the authors introduce a unified scaffold-and-correct pipeline.
As shown in the figure below:
The scaffold-and-correct architecture organizes prediction into spatial and temporal stages, utilizing a coarse scaffold built primarily from pretrained or fixed components, followed by learnable correction heads. In the spatial stage, a fixed spatial upsampler Us aligns the coarse low-resolution inputs to the target high-resolution grid. A parameter-efficient fine-tuning adapted weather foundation model backbone B then processes these upsampled inputs to produce a spatial scaffold at the final lead time. A spatial correction head Cs subsequently adds a learned residual to this scaffold, yielding the refined spatial endpoint.
For the temporal stage, the predicted spatial endpoint anchors the temporal downscaling. A fixed temporal interpolator Ut bridges the upsampled present input and the predicted spatial endpoint to form temporal scaffolds across the target lead times. A lead-time-conditioned temporal correction head Ct then adds another residual to produce the final spatiotemporal outputs. This design concentrates the adaptation burden on the backbone and the lightweight correction heads, while the fixed interpolators preserve large-scale spatial and temporal structures. The entire architecture is trained end-to-end using a variable-weighted state loss applied uniformly to all trajectory outputs, ensuring both stages co-adapt for the joint trajectory.
To efficiently adapt the large weather foundation model backbone within a limited parameter budget, the authors propose PROLA, a Principal-Orthogonal Low-rank Adaptation method. PROLA addresses how the limited capacity is allocated in the pretrained weight space and how updates are rescaled during optimization.
As shown in the figure below:
PROLA first allocates the fixed low-rank budget through a dual-branch mechanism. The total rank is split into a principal branch and an orthogonal branch. The principal branch is initialized from the top singular components of the pretrained weight matrix, supporting pretrained-aligned adaptation. The orthogonal branch is dedicated to capturing residual corrections outside the selected principal directions, ensuring that the model can learn task-specific deviations without disrupting the core pretrained knowledge.
Furthermore, PROLA introduces gradient-guided scaling of the optimizer updates. For each rank-1 component within both branches, the method computes a component signal based on the gradient magnitude. These signals are normalized into component and branch scores, which are then used to rescale the raw updates proposed by the base optimizer. This post-optimizer scaling reallocates the update magnitude toward high-signal components while preserving the average layer-level update scale, allowing the model to effectively direct its limited trainable capacity toward the most critical task-specific corrections.
Experiment
The experiments evaluate PROLA across global probabilistic spatiotemporal weather downscaling, ERA5-to-CERRA regional downscaling, and regional precipitation forecasting, comparing it with low-rank adaptation baselines and full fine-tuning. PROLA consistently achieves the best accuracy and calibration among parameter-efficient methods, outperforming full fine-tuning on several variables while adapting very few parameters and improving fine-scale spectral fidelity. Ablations show that the scaffold-and-correct pipeline benefits from both correction heads, especially temporal correction, and that combining principal and orthogonal branches with update scaling under a balanced rank split yields the strongest results. Analysis of update geometry further indicates that PROLA uses more orthogonal update energy than PiSSA while still retaining meaningful corrections within pretrained principal subspaces.
On WeatherBench 2, PROLA achieves the lowest RMSE and highest ACC among PEFT methods across all reported variables and lead times. It uses only 3.92M trainable parameters, under 0.4% of full fine-tuning, yet outperforms full fine-tuning on Z500 at both leads and on T850 and T2M at 6h. PROLA attains the best RMSE and ACC among PEFT methods for all reported variables and lead times. Relative to AdaLoRA, PROLA reduces Z500 RMSE by 5.3% at 3h and 6.7% at 6h, with consistent but smaller gains on other variables. Despite using under 0.4% of Full FT trainable parameters, PROLA surpasses Full FT on Z500 at both lead times and on T850 and T2M at 6h.
PROLA records the lowest latitude-weighted RMSE at 3h for all five reported ERA5-to-CERRA variables. Among the low-rank baselines, PiSSA performs best on every variable, ahead of LoRA and OPLoRA. The relative improvement is largest for Z500 and T2M. PROLA is the only method that ranks first on all five reported downscaling variables at 3h. PiSSA consistently outperforms LoRA and OPLoRA across the five variables, making it the strongest baseline in this comparison. Relative to PiSSA, PROLA provides the largest RMSE reductions for Z500 and T2M.
For 6h precipitation totals at a 12h lead, PROLA achieves the best performance among the compared parameter efficient fine tuning methods, with the lowest SEEPS, lowest RMSE, and highest ACC. Relative to OPLoRA, PROLA improves all three metrics, indicating better prediction of precipitation categories and accumulated amounts as well as closer agreement with spatial anomaly patterns. These gains are consistent with PROLA outperforming low-rank baselines across regional weather tasks. PROLA records the lowest error and highest skill among all compared methods at the 12h lead. Compared with OPLoRA, PROLA improves SEEPS, RMSE, and ACC, reflecting better precipitation category prediction, accumulated amount prediction, and spatial pattern agreement.
Direct prediction with PROLA achieves lower latitude-weighted RMSE than LoRA, PiSSA, and OPLoRA across the reported variables at 3h and 6h lead times. The full scaffold-and-correct pipeline further improves accuracy over direct PROLA prediction, while removing the temporal correction head sharply degrades 3h Z500 RMSE and removing both heads shows that uncorrected scaffolds are insufficient. Spatial correction removal has a much smaller effect, indicating temporal correction is the main driver of the intermediate lead improvement. Direct prediction with PROLA outperforms LoRA, PiSSA, and OPLoRA on all reported variables at both lead times. The full scaffold-and-correct pipeline lowers 3h Z500 RMSE from 53.13 with direct PROLA prediction to 50.30. Temporal correction matters more than spatial correction at 3h: removing the temporal head raises Z500 RMSE to 84.53, while removing the spatial head leaves it at 53.35. Removing both correction heads raises 3h Z500 RMSE to 87.73, showing scaffolds alone are insufficient.
Ablations show that combining principal and orthogonal branches improves forecast skill over either branch alone, and update scaling provides further gains. With scaling enabled, a balanced rank split between the two branches performs best across all variables at both lead times. Full PROLA, which combines branch and component scores, achieves the lowest RMSE across the evaluated variables, while removing either score degrades performance. Combining the principal and orthogonal branches lowers 3h Z500 RMSE relative to either branch alone, and update scaling improves both single-branch and combined configurations. A balanced rank allocation outperforms uneven allocations across all five variables at both 3h and 6h leads when scaling is enabled. Using only the branch score or only the component score degrades 6h Z500 RMSE compared with the full PROLA configuration, which achieves the lowest errors across variables.
The experiments evaluate PROLA as a parameter-efficient fine-tuning method across global WeatherBench2 forecasting, ERA5-to-CERRA regional downscaling, 12h precipitation prediction, and ablations of its scaffold-and-correct and low-rank design. Across these settings, PROLA consistently outperforms LoRA, PiSSA, OPLoRA, and AdaLoRA baselines while using under 0.4% of full fine-tuning parameters, and it can surpass full fine-tuning on several variables and lead times. The correction pipeline mainly benefits from temporal rather than spatial correction, and ablations confirm that combining principal and orthogonal branches, update scaling, balanced rank allocation, and both branch and component scores all contribute to the best forecast skill.