Nature, Cambridge University and Others Released the First End-to-end Data-driven Weather Forecasting System, Which Increases the Prediction Speed by Dozens of Times
At dawn every day, hundreds of supercomputers around the world cut the atmosphere into millions of virtual cubes, and by solving a set of partial differential equations involving thermodynamics, fluid mechanics, and radiation transfer, they deduce the trajectory of the Earth's atmosphere in the next 10 days. This technology, called numerical weather forecasting, has developed into the cornerstone of modern weather forecasting since the British meteorologist Richardson proposed the idea of a "weather forecast factory" in the mid-20th century. From typhoon path prediction to flight route planning, from agricultural sowing decisions to new energy power generation scheduling, the three-dimensional snapshot of the global atmosphere updated every 6 hours by the numerical forecast system is reshaping the way humans communicate with the weather with a spatial resolution of 0.01° longitude and latitude (about 1 kilometer).
However, faced with growing social demands, traditional numerical forecasting models based on physical equations are facing the dual dilemma of exponential growth in computational complexity and diminishing marginal benefits of improved forecast accuracy.In this context, the intervention of artificial intelligence technology is like installing a new sensory organ for atmospheric science. When deep neural networks begin to analyze the subtle textures of satellite cloud images, when reinforcement learning algorithms autonomously optimize parameterization schemes, and when generative adversarial networks rehearse extreme weather in virtual scenes, a technological revolution in weather forecasting is gradually taking shape.
Recently, the University of Cambridge, together with the Turing Institute, the University of Toronto, the Microsoft Center for Scientific Intelligence, the European Centre for Medium-Range Weather Forecasts, the British Antarctic Survey and Google DeepMind, jointly launched an end-to-end data-driven weather forecasting system called Aardvark Weather. The system integrates observational data to generate global gridded forecasts and local site forecasts at the same time.In tests of multiple key variables and different lead times, its global forecast performance surpassed the traditional operational numerical weather prediction (NWP) benchmark.For local site forecasts, the system demonstrates high forecast skill with a lead time of up to 10 days, and its performance is comparable to that of the global NWP benchmark optimized through post-processing and advanced end-to-end forecast systems incorporating input from human forecasters. Further end-to-end tuning significantly improves the accuracy of local forecasts.
The relevant research results are titled "End-to-end data-driven weather prediction" and have been published in an unedited version in Nature.
Paper address:
https://www.nature.com/articles/s41586-025-08897-0
The open source project "awesome-ai4s" brings together more than 100 AI4S paper interpretations and also provides massive data sets and tools:
https://github.com/hyperai/awesome-ai4s
Datasets: Cross-platform, multi-source integration, building innovative data fusion strategies
To ensure that the Aardvark Weather system is completely independent of traditional numerical weather prediction (NWP) models at work, the researchers carefully constructed a cross-platform, multi-source fusion observation dataset.The system only uses satellite data with processing levels of 1B or 1C.Level 1B data refers to data that has been calibrated and geolocated, while level 1C data has been further calibrated and geometrically processed to ensure that the observation accuracy meets the requirements of business deployment. All data covers the time span from 2007 to 2020 and supports real-time acquisition, providing a stable data stream for the system.
Land observation data comes from the HadISD dataset of the UK Meteorological Office.It includes surface temperature at 8,719 stations, air pressure at 8,016 stations, wind speed at 8,721 stations, and dew point temperature at 8,617 stations, updated every 6 hours. Ocean observations rely on the ICOADS data set of NOAA in the United States, integrating real-time observations from ships and buoys around the world. Upper atmosphere observation data comes from NCEI's IGRA archive, covering temperature, humidity and wind profiles at 1,375 sounding stations. The research team specially selected observation records of the surface and key pressure levels such as 200, 500, 700, and 850 hPa.
In addition, the geographical limitations of field observations prompted researchers to introduce multi-source remote sensing data. The study used the ASCAT scatterometer data from the MetOp series of satellites provided by the European EUMETSAT, and jointly built a microwave-infrared collaborative observation network with the ATOVS system provided by NCEI (including AMSUA, AMSUB, MHS and HIRS/4 instruments). The research team further used NCEI's IASI hyperspectral infrared detection data to supplement the detailed information on the vertical distribution of temperature and humidity. For geostationary satellite observations, the research team selected NCDC's GridSat gridded products, which eliminated the temporal and spatial heterogeneity of orbiting satellites by standardizing infrared and water vapor channel data.
It is worth noting that although the Aardvark system is completely independent of the NWP model when running, its modular architecture cleverly utilizes the ERA5 reanalysis dataset during the training phase. ERA5, the fifth generation of reanalysis of the European Centre for Medium-Range Weather Forecasts (ECMWF), is a state-of-the-art global atmospheric reanalysis dataset that provides a complete spatiotemporal series of parameters such as temperature, humidity, and wind fields from 1940 to the present. Through this innovative data fusion strategy, Aardvark Weather not only breaks through the limitations of traditional numerical models, but also maximizes the real-time and high-precision advantages of data-driven models, opening up a new paradigm for global weather forecasting technology.
Aardvark Weather: Requires less computing power and increases forecast speed by dozens of times
Aardvark Weather is an innovative deep learning-driven end-to-end weather forecasting system.Its core architecture consists of three modules: encoder module, processing module and decoder modules.The system was designed to get rid of the dependence on traditional numerical weather forecast products during the deployment phase, while making full use of high-quality historical reanalysis data to improve model performance during the training phase.
The encoder is responsible for integrating multi-source observation data, whether gridded or non-gridded, to generate the initial atmospheric state. This process uses a non-recursive method, which effectively avoids problems such as gradient instability that may occur in the recursive update of the Kalman filter in traditional numerical weather forecasting. The processing module generates a 24-hour forecast based on the initial state and extends it to a longer lead time with the help of an autoregressive mechanism. The decoder converts the gridded forecast into a local site forecast, supports multi-task expansion, and further expands the application scope of the system.
In terms of model architecture, the encoder and processing modules use the visual Transformer (ViT) as the backbone, and the decoder uses a lightweight convolutional network. This combination ensures both processing efficiency and model flexibility. In terms of training strategy, Aardvark first uses high-fidelity historical reanalysis data (such as ERA5) for pre-training to learn to correct observation biases, and then optimizes each module through staged fine-tuning to ensure consistency between training and deployment. Finally, joint fine-tuning is used to further improve the forecast accuracy of specific regions and variables. This modular design and training method allows Aardvark to be completely independent of traditional numerical models when deployed, giving full play to the real-time and high-precision advantages of data-driven models.
In evaluating the global gridded forecast performance of Aardvark Weather, the research team compared it with four baseline models. The simplest baselines are persistence and hourly climate, which are used to evaluate whether the forecast system has basic capabilities. The more challenging comparison objects are the two most widely used deterministic operational global NWP systems: the Integrated Forecast System (IFS) configured with high-resolution HRES of the European Center for Medium-Range Weather Forecasts (ECMWF) and the Global Forecast System (GFS) of the National Centers for Environmental Prediction. The results show that Aardvark is comparable to or even better than GFS in most lead times. In addition, for most variables, Aardvark's performance is close to HRES. Although Aardvark has larger errors in the higher atmosphere and shorter lead times, which may be due to the more concentrated observations close to the surface, for longer lead times, Aardvark's forecasts tend to become spectrally blurred, which is a common phenomenon in data-driven weather forecast systems.
The research results also show that Aardvark Weather, as the first fully end-to-end data-driven weather forecasting system, does not rely on NWP products at all when deployed by learning a direct mapping from raw observation data to forecast outputs. It relies solely on observation data to complete forecast generation, and the generation speed is several orders of magnitude faster than existing systems. It takes only about one second on four NVIDIA A100 GPUs to generate a complete forecast from observation data. In comparison, the traditional European Centre for Medium-Range Weather Forecasts (ECMWF) High Resolution (HRES) system requires about 1,000 node hours to perform data assimilation and forecasting alone. Even though the amount of observation data used is an order of magnitude less than the operational baseline, and the computing resource requirements are several orders of magnitude less than traditional methods,Aardvark still produces forecasts on a 1.5 degree global grid and outperforms operational NWP systems in root mean square error (RMSE) across multiple variables and lead times.
AI has been widely used in the field of numerical weather forecasting
At present, global meteorological science is undergoing a paradigm shift led by artificial intelligence technology, and my country's scientific research strength has shown a strong breakthrough trend in this technological revolution. Relying on the iterative innovation of deep learning algorithms,Domestic research institutions have built an intelligent model system covering the entire chain of weather forecasting.It not only significantly improved the forecast accuracy and efficiency, but also promoted the deep integration of meteorology and AI.
For example, as the world's first three-dimensional neural network model adapted to the earth's coordinate system, Huawei's Pangu Meteorological Model breaks through the physical equation dependence of traditional numerical forecasts. Its innovative hierarchical time-domain aggregation strategy effectively suppresses the accumulation of iterative errors, making the global medium-term weather forecast resolution reach an industry-high 0.25°. Fudan University's Fuxi Meteorological Model takes a different approach, achieving collaborative forecasting of meteorological elements through a multi-task learning framework. Its latitude-weighted loss function design cleverly solves the industry pain point of attenuated forecast accuracy in high-latitude areas, especially in achieving significant improvements in the forecast accuracy of small to medium precipitation events.
In the field of special forecasting, the "Fengwu" system of the Shanghai Artificial Intelligence Laboratory has compressed the 120-hour forecast error of the typhoon path to a breakthrough accuracy of 121.4 kilometers through multimodal fusion technology, which is a leapfrog improvement over the 293.8-kilometer error of the European Center for Medium-Term Weather Forecasts ECMWF. The "Wenhai" ocean forecasting model of the Laoshan Laboratory has pioneered the embedding of ocean dynamics principles into the neural network architecture, building the world's first intelligent ocean environment forecasting system with a resolution of 1/12°.
Not only that, The "AI Weather Forecast Model Demonstration Program" led by the China Meteorological Administration is accelerating the transformation of technological achievements. The Fengqing, Fenglei, and Fengshun systems have been released.This marks the transformation of AI technology from research to application. Among them, the "Wind and Thunder" system pioneered a hybrid architecture of physical guidance and data drive. This "mathematical fusion" technology path effectively makes up for the shortcomings of pure AI models in the explanation of extreme weather events.The "Fengqing" large model generates a 15-day global forecast at an astonishing speed of 3 minutes, extending the effective forecast time of core elements to 10.5 days.The "Fengshun" sub-seasonal forecasting system under construction marks the beginning of the extension of AI technology into the more difficult field of climate forecasting.
Although the research on AI meteorological big models in China has moved from technical verification to business application, its advantages of high efficiency and low cost have brought a new paradigm to global weather forecasting. However, the interpretability of physical mechanisms, the real-time nature of data, and the modeling of extreme events are still difficult problems that need to be overcome. With the continuous deepening of the physics-data fusion framework and the strengthening of interdisciplinary cooperation,AI technology is expected to push weather forecasting into a new era of higher accuracy and wider application, from short-term forecasts at the minute level to climate forecasts at the seasonal scale.