HyperAIHyperAI

Command Palette

Search for a command to run...

PROLA:予測的時空間気象ダウンスケーリングのための主成分直交低ランク適応

Minseo Yoon Minseong Bae Sojin Lee Hyunwoo J. Kim

概要

高解像度の気象予測は、粗い全球予報では捉えられない局所的な大気パターンを解像するために重要である。大規模な事前学習済み気象バックボーンは全球予報を改善してきたが、最近の粗い状態から将来の高解像度場を生成するようにこれらを適応させることは依然として困難でコストがかかる。我々はこの設定を予測的時空間気象ダウンスケーリング(PSWD)として定式化する。これは、将来の高解像度軌道を予測しながら空間的・時間的解像度を同時に精緻化するタスクである。この目的のために、共有の事前学習済みバックボーン、固定された数値的スキャフォールド、および軽量な残差ヘッドを用いて、予測を空間段階と時間段階に分解するフレームワークを提案する。バックボーン適応のために、我々はPROLA(PRincipal-Orthogonal Low-rank Adaptation)を導入する。これは、事前学習済み重みの主成分方向とその直交補空間の間に固定された低ランク予算を配分する。PROLAはさらに、勾配信号を用いてランク1のオプティマイザ更新を再スケーリングし、学習可能なランクを増やすことなく適応を改善する。PSWD設定における実験では、PROLAが代表的な低ランク適応ベースラインを上回ることが示された。多様なデータセットとタスクにわたるさらなる実験は、PROLAのより広い適用可能性を支持し、その利点が主要な気象ダウンスケーリング設定を超えて広がることを示している。

One-sentence Summary

Researchers at Korea Advanced Institute of Science and Technology propose PROLA, a principal-orthogonal low-rank adaptation that allocates a fixed low-rank budget between pretrained weights' principal directions and their orthogonal complement while rescaling rank-1\mathrm{rank}\text{-}1rank-1 optimizer updates using gradient signals, thereby improving predictive spatiotemporal weather downscaling and outperforming representative low-rank adaptation baselines across diverse datasets and tasks.

Key Contributions

  • The paper formalizes Predictive Spatiotemporal Weather Downscaling (PSWD) as predicting future high-resolution, temporally denser weather trajectories from recent coarse-resolution global states, and introduces a spatial-to-temporal scaffold-and-correct pipeline using a shared pretrained backbone with fixed numerical scaffolds and lightweight residual heads.
  • It proposes PROLA (Principal-Orthogonal Low-rank Adaptation), which splits a fixed low-rank budget between pretrained weights' principal directions and their orthogonal complement, and rescales rank-1 optimizer updates using gradient signals without increasing trainable rank.
  • Experiments show PROLA outperforms representative low-rank adaptation baselines under matched low-rank budgets, and further tests across diverse datasets and tasks support broader applicability beyond weather downscaling.

Introduction

Weather modeling has largely separated forecasting, spatial downscaling, and temporal downscaling, but applications such as minute-scale wind-power forecasting, sub-hourly precipitation nowcasting, and aviation turbulence guidance require predicting future states at both finer spatial and temporal resolution from low-resolution inputs. Prior work has not addressed this joint regime, and adapting large weather foundation models to richer output grids is constrained by the cost of full fine-tuning. The authors formalize this setting as Predictive Spatiotemporal Weather Downscaling (PSWD), introduce a scaffold-and-correct pipeline built around a shared pretrained backbone, and propose PROLA, a parameter-efficient low-rank adaptation method that splits a fixed rank budget between principal and orthogonal branches while rescaling optimizer updates with gradient-based scores. Experiments on WeatherBench 2 show that PROLA improves over representative low-rank PEFT baselines under matched budgets and narrows the gap to full fine-tuning.

Method

The authors formulate Predictive Spatiotemporal Weather Downscaling as a coupled spatiotemporal prediction task that jointly requires forecasting, spatial downscaling, and temporal downscaling. Given coarse-step past and present low-resolution inputs, the goal is to predict fine-step future high-resolution targets. To address the challenges of this coupled shift, the authors introduce a unified scaffold-and-correct pipeline.

As shown in the figure below:

The scaffold-and-correct architecture organizes prediction into spatial and temporal stages, utilizing a coarse scaffold built primarily from pretrained or fixed components, followed by learnable correction heads. In the spatial stage, a fixed spatial upsampler UsU_sUs​ aligns the coarse low-resolution inputs to the target high-resolution grid. A parameter-efficient fine-tuning adapted weather foundation model backbone B\mathcal{B}B then processes these upsampled inputs to produce a spatial scaffold at the final lead time. A spatial correction head CsC_sCs​ subsequently adds a learned residual to this scaffold, yielding the refined spatial endpoint.

For the temporal stage, the predicted spatial endpoint anchors the temporal downscaling. A fixed temporal interpolator UtU_tUt​ bridges the upsampled present input and the predicted spatial endpoint to form temporal scaffolds across the target lead times. A lead-time-conditioned temporal correction head CtC_tCt​ then adds another residual to produce the final spatiotemporal outputs. This design concentrates the adaptation burden on the backbone and the lightweight correction heads, while the fixed interpolators preserve large-scale spatial and temporal structures. The entire architecture is trained end-to-end using a variable-weighted state loss applied uniformly to all trajectory outputs, ensuring both stages co-adapt for the joint trajectory.

To efficiently adapt the large weather foundation model backbone within a limited parameter budget, the authors propose PROLA, a Principal-Orthogonal Low-rank Adaptation method. PROLA addresses how the limited capacity is allocated in the pretrained weight space and how updates are rescaled during optimization.

As shown in the figure below:

PROLA first allocates the fixed low-rank budget through a dual-branch mechanism. The total rank is split into a principal branch and an orthogonal branch. The principal branch is initialized from the top singular components of the pretrained weight matrix, supporting pretrained-aligned adaptation. The orthogonal branch is dedicated to capturing residual corrections outside the selected principal directions, ensuring that the model can learn task-specific deviations without disrupting the core pretrained knowledge.

Furthermore, PROLA introduces gradient-guided scaling of the optimizer updates. For each rank-1 component within both branches, the method computes a component signal based on the gradient magnitude. These signals are normalized into component and branch scores, which are then used to rescale the raw updates proposed by the base optimizer. This post-optimizer scaling reallocates the update magnitude toward high-signal components while preserving the average layer-level update scale, allowing the model to effectively direct its limited trainable capacity toward the most critical task-specific corrections.

Experiment

The experiments evaluate PROLA across global probabilistic spatiotemporal weather downscaling, ERA5-to-CERRA regional downscaling, and regional precipitation forecasting, comparing it with low-rank adaptation baselines and full fine-tuning. PROLA consistently achieves the best accuracy and calibration among parameter-efficient methods, outperforming full fine-tuning on several variables while adapting very few parameters and improving fine-scale spectral fidelity. Ablations show that the scaffold-and-correct pipeline benefits from both correction heads, especially temporal correction, and that combining principal and orthogonal branches with update scaling under a balanced rank split yields the strongest results. Analysis of update geometry further indicates that PROLA uses more orthogonal update energy than PiSSA while still retaining meaningful corrections within pretrained principal subspaces.

On WeatherBench 2, PROLA achieves the lowest RMSE and highest ACC among PEFT methods across all reported variables and lead times. It uses only 3.92M trainable parameters, under 0.4% of full fine-tuning, yet outperforms full fine-tuning on Z500 at both leads and on T850 and T2M at 6h. PROLA attains the best RMSE and ACC among PEFT methods for all reported variables and lead times. Relative to AdaLoRA, PROLA reduces Z500 RMSE by 5.3% at 3h and 6.7% at 6h, with consistent but smaller gains on other variables. Despite using under 0.4% of Full FT trainable parameters, PROLA surpasses Full FT on Z500 at both lead times and on T850 and T2M at 6h.

PROLA records the lowest latitude-weighted RMSE at 3h for all five reported ERA5-to-CERRA variables. Among the low-rank baselines, PiSSA performs best on every variable, ahead of LoRA and OPLoRA. The relative improvement is largest for Z500 and T2M. PROLA is the only method that ranks first on all five reported downscaling variables at 3h. PiSSA consistently outperforms LoRA and OPLoRA across the five variables, making it the strongest baseline in this comparison. Relative to PiSSA, PROLA provides the largest RMSE reductions for Z500 and T2M.

For 6h precipitation totals at a 12h lead, PROLA achieves the best performance among the compared parameter efficient fine tuning methods, with the lowest SEEPS, lowest RMSE, and highest ACC. Relative to OPLoRA, PROLA improves all three metrics, indicating better prediction of precipitation categories and accumulated amounts as well as closer agreement with spatial anomaly patterns. These gains are consistent with PROLA outperforming low-rank baselines across regional weather tasks. PROLA records the lowest error and highest skill among all compared methods at the 12h lead. Compared with OPLoRA, PROLA improves SEEPS, RMSE, and ACC, reflecting better precipitation category prediction, accumulated amount prediction, and spatial pattern agreement.

Direct prediction with PROLA achieves lower latitude-weighted RMSE than LoRA, PiSSA, and OPLoRA across the reported variables at 3h and 6h lead times. The full scaffold-and-correct pipeline further improves accuracy over direct PROLA prediction, while removing the temporal correction head sharply degrades 3h Z500 RMSE and removing both heads shows that uncorrected scaffolds are insufficient. Spatial correction removal has a much smaller effect, indicating temporal correction is the main driver of the intermediate lead improvement. Direct prediction with PROLA outperforms LoRA, PiSSA, and OPLoRA on all reported variables at both lead times. The full scaffold-and-correct pipeline lowers 3h Z500 RMSE from 53.13 with direct PROLA prediction to 50.30. Temporal correction matters more than spatial correction at 3h: removing the temporal head raises Z500 RMSE to 84.53, while removing the spatial head leaves it at 53.35. Removing both correction heads raises 3h Z500 RMSE to 87.73, showing scaffolds alone are insufficient.

Ablations show that combining principal and orthogonal branches improves forecast skill over either branch alone, and update scaling provides further gains. With scaling enabled, a balanced rank split between the two branches performs best across all variables at both lead times. Full PROLA, which combines branch and component scores, achieves the lowest RMSE across the evaluated variables, while removing either score degrades performance. Combining the principal and orthogonal branches lowers 3h Z500 RMSE relative to either branch alone, and update scaling improves both single-branch and combined configurations. A balanced rank allocation outperforms uneven allocations across all five variables at both 3h and 6h leads when scaling is enabled. Using only the branch score or only the component score degrades 6h Z500 RMSE compared with the full PROLA configuration, which achieves the lowest errors across variables.

The experiments evaluate PROLA as a parameter-efficient fine-tuning method across global WeatherBench2 forecasting, ERA5-to-CERRA regional downscaling, 12h precipitation prediction, and ablations of its scaffold-and-correct and low-rank design. Across these settings, PROLA consistently outperforms LoRA, PiSSA, OPLoRA, and AdaLoRA baselines while using under 0.4% of full fine-tuning parameters, and it can surpass full fine-tuning on several variables and lead times. The correction pipeline mainly benefits from temporal rather than spatial correction, and ablations confirm that combining principal and orthogonal branches, update scaling, balanced rank allocation, and both branch and component scores all contribute to the best forecast skill.


AIでAIを構築

アイデアからローンチまで — 無料のAIコーディング支援、すぐに使える環境、最高のGPU価格でAI開発を加速。

AI コーディング補助
すぐに使える GPU
最適な料金体系

HyperAI Newsletters

最新情報を購読する
北京時間 毎週月曜日の午前9時 に、その週の最新情報をメールでお届けします
メール配信サービスは MailChimp によって提供されています