Command Palette
Search for a command to run...
PROLA: 예측적 시공간 기상 다운스케일링을 위한 주성분-직교 저계수 적응
PROLA: 예측적 시공간 기상 다운스케일링을 위한 주성분-직교 저계수 적응
Minseo Yoon Minseong Bae Sojin Lee Hyunwoo J. Kim
초록
고해상도 기상 예측은 거친 전지구 예보가 놓치는 국지적 대기 패턴을 해석하는 데 중요하다. 대규모 사전 학습된 기상 백본은 전지구 예측을 개선했지만, 최근의 거친 상태로부터 미래의 고해상도 장을 생성하도록 이들을 적응시키는 것은 여전히 어렵고 비용이 많이 든다. 우리는 이 설정을 예측적 시공간 기상 다운스케일링(PSWD)으로 정식화하는데, 이는 미래의 고해상도 궤적을 예측하면서 공간 및 시간 해상도를 공동으로 정제하는 과제이다. 이를 위해 공유된 사전 학습 백본, 고정된 수치적 스캐폴드, 경량 잔차 헤드를 사용하여 예측을 공간 및 시간 단계로 분해하는 프레임워크를 제안한다. 백본 적응을 위해 사전 학습된 가중치의 주성분 방향과 그 직교 여공간 사이에 고정된 저계수 예산을 할당하는 PROLA(PRincipal-Orthogonal Low-rank Adaptation)를 소개한다. PROLA는 또한 기울기 신호를 사용하여 랭크-1 최적화기 업데이트를 재조정함으로써 학습 가능한 랭크를 늘리지 않고도 적응을 개선한다. PSWD 설정에서의 실험은 PROLA가 대표적인 저계수 적응 기준선을 능가함을 보여준다. 다양한 데이터셋과 과제에 걸친 추가 실험은 PROLA의 더 넓은 적용 가능성을 뒷받침하며, 그 이점이 주요 기상 다운스케일링 설정을 넘어 확장됨을 보여준다.
One-sentence Summary
Researchers at Korea Advanced Institute of Science and Technology propose PROLA, a principal-orthogonal low-rank adaptation that allocates a fixed low-rank budget between pretrained weights' principal directions and their orthogonal complement while rescaling rank-1 optimizer updates using gradient signals, thereby improving predictive spatiotemporal weather downscaling and outperforming representative low-rank adaptation baselines across diverse datasets and tasks.
Key Contributions
- The paper formalizes Predictive Spatiotemporal Weather Downscaling (PSWD) as predicting future high-resolution, temporally denser weather trajectories from recent coarse-resolution global states, and introduces a spatial-to-temporal scaffold-and-correct pipeline using a shared pretrained backbone with fixed numerical scaffolds and lightweight residual heads.
- It proposes PROLA (Principal-Orthogonal Low-rank Adaptation), which splits a fixed low-rank budget between pretrained weights' principal directions and their orthogonal complement, and rescales rank-1 optimizer updates using gradient signals without increasing trainable rank.
- Experiments show PROLA outperforms representative low-rank adaptation baselines under matched low-rank budgets, and further tests across diverse datasets and tasks support broader applicability beyond weather downscaling.
Introduction
Weather modeling has largely separated forecasting, spatial downscaling, and temporal downscaling, but applications such as minute-scale wind-power forecasting, sub-hourly precipitation nowcasting, and aviation turbulence guidance require predicting future states at both finer spatial and temporal resolution from low-resolution inputs. Prior work has not addressed this joint regime, and adapting large weather foundation models to richer output grids is constrained by the cost of full fine-tuning. The authors formalize this setting as Predictive Spatiotemporal Weather Downscaling (PSWD), introduce a scaffold-and-correct pipeline built around a shared pretrained backbone, and propose PROLA, a parameter-efficient low-rank adaptation method that splits a fixed rank budget between principal and orthogonal branches while rescaling optimizer updates with gradient-based scores. Experiments on WeatherBench 2 show that PROLA improves over representative low-rank PEFT baselines under matched budgets and narrows the gap to full fine-tuning.
Method
The authors formulate Predictive Spatiotemporal Weather Downscaling as a coupled spatiotemporal prediction task that jointly requires forecasting, spatial downscaling, and temporal downscaling. Given coarse-step past and present low-resolution inputs, the goal is to predict fine-step future high-resolution targets. To address the challenges of this coupled shift, the authors introduce a unified scaffold-and-correct pipeline.
As shown in the figure below:
The scaffold-and-correct architecture organizes prediction into spatial and temporal stages, utilizing a coarse scaffold built primarily from pretrained or fixed components, followed by learnable correction heads. In the spatial stage, a fixed spatial upsampler Us aligns the coarse low-resolution inputs to the target high-resolution grid. A parameter-efficient fine-tuning adapted weather foundation model backbone B then processes these upsampled inputs to produce a spatial scaffold at the final lead time. A spatial correction head Cs subsequently adds a learned residual to this scaffold, yielding the refined spatial endpoint.
For the temporal stage, the predicted spatial endpoint anchors the temporal downscaling. A fixed temporal interpolator Ut bridges the upsampled present input and the predicted spatial endpoint to form temporal scaffolds across the target lead times. A lead-time-conditioned temporal correction head Ct then adds another residual to produce the final spatiotemporal outputs. This design concentrates the adaptation burden on the backbone and the lightweight correction heads, while the fixed interpolators preserve large-scale spatial and temporal structures. The entire architecture is trained end-to-end using a variable-weighted state loss applied uniformly to all trajectory outputs, ensuring both stages co-adapt for the joint trajectory.
To efficiently adapt the large weather foundation model backbone within a limited parameter budget, the authors propose PROLA, a Principal-Orthogonal Low-rank Adaptation method. PROLA addresses how the limited capacity is allocated in the pretrained weight space and how updates are rescaled during optimization.
As shown in the figure below:
PROLA first allocates the fixed low-rank budget through a dual-branch mechanism. The total rank is split into a principal branch and an orthogonal branch. The principal branch is initialized from the top singular components of the pretrained weight matrix, supporting pretrained-aligned adaptation. The orthogonal branch is dedicated to capturing residual corrections outside the selected principal directions, ensuring that the model can learn task-specific deviations without disrupting the core pretrained knowledge.
Furthermore, PROLA introduces gradient-guided scaling of the optimizer updates. For each rank-1 component within both branches, the method computes a component signal based on the gradient magnitude. These signals are normalized into component and branch scores, which are then used to rescale the raw updates proposed by the base optimizer. This post-optimizer scaling reallocates the update magnitude toward high-signal components while preserving the average layer-level update scale, allowing the model to effectively direct its limited trainable capacity toward the most critical task-specific corrections.
Experiment
The experiments evaluate PROLA across global probabilistic spatiotemporal weather downscaling, ERA5-to-CERRA regional downscaling, and regional precipitation forecasting, comparing it with low-rank adaptation baselines and full fine-tuning. PROLA consistently achieves the best accuracy and calibration among parameter-efficient methods, outperforming full fine-tuning on several variables while adapting very few parameters and improving fine-scale spectral fidelity. Ablations show that the scaffold-and-correct pipeline benefits from both correction heads, especially temporal correction, and that combining principal and orthogonal branches with update scaling under a balanced rank split yields the strongest results. Analysis of update geometry further indicates that PROLA uses more orthogonal update energy than PiSSA while still retaining meaningful corrections within pretrained principal subspaces.
On WeatherBench 2, PROLA achieves the lowest RMSE and highest ACC among PEFT methods across all reported variables and lead times. It uses only 3.92M trainable parameters, under 0.4% of full fine-tuning, yet outperforms full fine-tuning on Z500 at both leads and on T850 and T2M at 6h. PROLA attains the best RMSE and ACC among PEFT methods for all reported variables and lead times. Relative to AdaLoRA, PROLA reduces Z500 RMSE by 5.3% at 3h and 6.7% at 6h, with consistent but smaller gains on other variables. Despite using under 0.4% of Full FT trainable parameters, PROLA surpasses Full FT on Z500 at both lead times and on T850 and T2M at 6h.
PROLA records the lowest latitude-weighted RMSE at 3h for all five reported ERA5-to-CERRA variables. Among the low-rank baselines, PiSSA performs best on every variable, ahead of LoRA and OPLoRA. The relative improvement is largest for Z500 and T2M. PROLA is the only method that ranks first on all five reported downscaling variables at 3h. PiSSA consistently outperforms LoRA and OPLoRA across the five variables, making it the strongest baseline in this comparison. Relative to PiSSA, PROLA provides the largest RMSE reductions for Z500 and T2M.
For 6h precipitation totals at a 12h lead, PROLA achieves the best performance among the compared parameter efficient fine tuning methods, with the lowest SEEPS, lowest RMSE, and highest ACC. Relative to OPLoRA, PROLA improves all three metrics, indicating better prediction of precipitation categories and accumulated amounts as well as closer agreement with spatial anomaly patterns. These gains are consistent with PROLA outperforming low-rank baselines across regional weather tasks. PROLA records the lowest error and highest skill among all compared methods at the 12h lead. Compared with OPLoRA, PROLA improves SEEPS, RMSE, and ACC, reflecting better precipitation category prediction, accumulated amount prediction, and spatial pattern agreement.
Direct prediction with PROLA achieves lower latitude-weighted RMSE than LoRA, PiSSA, and OPLoRA across the reported variables at 3h and 6h lead times. The full scaffold-and-correct pipeline further improves accuracy over direct PROLA prediction, while removing the temporal correction head sharply degrades 3h Z500 RMSE and removing both heads shows that uncorrected scaffolds are insufficient. Spatial correction removal has a much smaller effect, indicating temporal correction is the main driver of the intermediate lead improvement. Direct prediction with PROLA outperforms LoRA, PiSSA, and OPLoRA on all reported variables at both lead times. The full scaffold-and-correct pipeline lowers 3h Z500 RMSE from 53.13 with direct PROLA prediction to 50.30. Temporal correction matters more than spatial correction at 3h: removing the temporal head raises Z500 RMSE to 84.53, while removing the spatial head leaves it at 53.35. Removing both correction heads raises 3h Z500 RMSE to 87.73, showing scaffolds alone are insufficient.
Ablations show that combining principal and orthogonal branches improves forecast skill over either branch alone, and update scaling provides further gains. With scaling enabled, a balanced rank split between the two branches performs best across all variables at both lead times. Full PROLA, which combines branch and component scores, achieves the lowest RMSE across the evaluated variables, while removing either score degrades performance. Combining the principal and orthogonal branches lowers 3h Z500 RMSE relative to either branch alone, and update scaling improves both single-branch and combined configurations. A balanced rank allocation outperforms uneven allocations across all five variables at both 3h and 6h leads when scaling is enabled. Using only the branch score or only the component score degrades 6h Z500 RMSE compared with the full PROLA configuration, which achieves the lowest errors across variables.
The experiments evaluate PROLA as a parameter-efficient fine-tuning method across global WeatherBench2 forecasting, ERA5-to-CERRA regional downscaling, 12h precipitation prediction, and ablations of its scaffold-and-correct and low-rank design. Across these settings, PROLA consistently outperforms LoRA, PiSSA, OPLoRA, and AdaLoRA baselines while using under 0.4% of full fine-tuning parameters, and it can surpass full fine-tuning on several variables and lead times. The correction pipeline mainly benefits from temporal rather than spatial correction, and ablations confirm that combining principal and orthogonal branches, update scaling, balanced rank allocation, and both branch and component scores all contribute to the best forecast skill.