HyperAI超神经

Contents at a glance:Landslide prediction has always been very difficult due to the multiple temporal and spatial variations involved. Deep neural networks (DNNs) can improve prediction accuracy, but they are not interpretable by themselves. In this paper, UCLA researchers introduced SNNs. SNNs have the characteristics of full interpretability, high accuracy, high generalization ability, and low model complexity, which further improves the prediction ability of landslide risk.

Keywords： Landslide SNN DNN

This article was first published on HyperAI WeChat public platform~

The occurrence of landslides is affected by a combination of factors, such as topography, slope, soil, rock and other material characteristics, as well as environmental conditions such as climate, rainfall, and hydrology. Therefore, relevant predictions have always been very difficult. Typically, geologists use physical and statistical models to estimate the risk of landslides.Although these models can provide fairly accurate predictions, training physical models requires a lot of time and resources and is not suitable for large-scale applications.

In recent years, researchers have been training machine learning models for predicting landslides, especially deep neural networks (DNNs). As a high-precision prediction model, DNNs have been effective in many fields, including image recognition, speech recognition, natural language processing, computational biology, and financial big data.However, it has multiple layers of hidden structures outside the input layer and output layer and lacks interpretability. This black box problem has always troubled researchers.

Recently, researchers at the University of California, Los Angeles (UCLA) developed a superposable neural network (SNN). Unlike DNN, SNN can separate the results of different data inputs to better analyze the influencing factors in natural disasters. The SNN model outperforms physical and statistical models and achieves performance similar to the most advanced DNN.Currently, the research results have been published in the journal Communications Earth & Environment, titled "Landslide susceptibility modeling by interpretable neural network".

Figure 1: The research results have been published in Communications Earth & Environment

Read the full paper:

https://www.nature.com/articles/s43247-023-00806-5#Sec4

Select landslide data from the easternmost Himalayas

Through data analysis, researchers found that casualties caused by landslides from 2004 to 2016 occurred mainly in Asia.The easternmost region of the Himalayas is highly prone to steep slope landslides, extreme precipitation, floods and other events. The researchers generated an inventory of landslides (a record or dataset of landslide events) in the easternmost Himalayas by combining manually delineated landslide areas with a semi-automated detection algorithm.The total number of mapped landslides was 2,289, ranging in area from 900 to 1.96 × 106 m2, throughout the study area of 4.19 × 109 m2.

Figure 2: Study area in the easternmost part of the Himalayas

The colors represent the elevation, and the yellow box indicates the study area in the NS (Dibang), NW-SE (range front), and EW (Lohit) directions.

The inset shows the eastern Himalayas, the black box indicates the study area, and the dark grey line indicates the national border (upper right corner).

As shown in the figure above, researchers selected three areas with different environmental conditions (Dibang, Lohit and range front) in the easternmost part of the Himalayas to test the performance and application of the SNN model.Hereinafter, Dibang, Lohit and range front areas are referred to as NS, EW and NW-SE, respectively.

Dataset address:

https://doi.org/10.25346/S6/D5QPUA

Model development: 6 steps to train an SNN

In this study, in order to circumvent the lack of interpretability of DNN while ensuring accuracy, the researchers combined model extraction and feature-based methods to generate a fully interpretable additive ANN optimization framework. Additive ANN is a type of generalized additive models (GAM). Model extraction methods aim to train an interpretable student model to mimic the teacher model. Feature-based methods aim to analyze and quantify the impact of each input feature.

The researchers call this additive ANN architecture Superposable Neural Network (SNN) optimization.Different from DNN, which establishes the interdependence between features through connections between different layers, SNN establishes the interdependence between features through the product function of the original input features.The comparison between the two is shown in the figure below:

Figure 3: Traditional DNN vs. SNN

x1, x2, …, xn denotes a set of n original features, χ1, χ2, …, χM denotes a set of M combined features, and Y and St refer to the susceptibility results in DNN and SNN, respectively.

As shown in Figure 3, in traditional DNNs, features are represented and learned through connections in the network. This dependency is tightly embedded in the network structure, which is very complex and difficult to separate.In SNNs, researchers find and explicitly assign features that will contribute to the output to separate inputs in advance, and each neuron is connected to only one input.

The SNN training flow chart is as follows:

Figure 4: Flowchart of training SNN

As shown in the figure, the researchers adopted two main methods, feature-selection model and multistage training.The feature selection model is used to select the most relevant features for subsequent analysis and modeling; multi-stage training means that the training process is divided into multiple stages, each stage has specific goals and training strategies, and gradually optimizes model performance.

The training process can be summarized into the following steps:

Multivariate polynomial expansion:Create composite features.
Tournament ranking:An automatic feature selection method for finding the features most relevant to the model.
Multi-Stage Training (MST):A second-order deep learning technique for generating high-performance teacher networks.
Fractional knowledge distillation:Used to separate the contribution of each feature to the final output.
Parallel knowledge distillation:Standard knowledge distillation techniques are applied to the network corresponding to each feature individually.
Network superposition:The single-layer networks corresponding to each feature are merged into one SNN.

Experimental Results

SNN highest accuracy exceeds 99%

Based on the highest level composite features used in model training, the researchers divided SNN into three different levels of models, namely Level-1, Level-2 and Level-3.Experiments show that the accuracy of Level-3 SNN can reach more than 99% of SOTA teacher DNN, and the accuracy of Level-2 SNN exceeds 98%.Given the small difference in accuracy between the two, the researchers assume that the interpretability of the Level-2 SNN is sufficient for analysis.

Next, the researchers compared Level-1 and Level-2 SNNs with the SOTA DNN teacher model (MST, DNN based on second-order optimization), as well as traditional methods (LogR and LR).All methods were applied to the same area and using the same data, and the results are shown in the figure below.

Figure 5: Performance comparison of various models

MST: SOTA DNN Teacher Model

LogR: Logistic regression (traditional method)

LR: Likelihood ratio (traditional method)

As shown in the figure, the performance of SNN is comparable to that of MST model and outperforms the commonly used traditional models. The average values of the three study areas were calculated, and the AUROCs of Level-1 and Level-2 SNNs were 0.856 and 0.890, respectively. The AUROC of Level-2 SNN was about 8% higher than that of LogR (AUROC = 0.848) and LR (AUROC = 0.823).

AUROC (area under the receiver operating characteristic): A performance metric used to evaluate classification models. The closer the AUROC is to 1, the better the model performance.

SNNs are fully interpretable

SNN is a fully interpretable model with a level of interpretability comparable to linear regression.

The researchers divided the study area into landslide (ld) and non-landslide (nld) areas. SNN provides the exact contribution of individual features to susceptibility, making it possible to quantify the impact of each feature on landslide susceptibility.By calculating the differences in individual characteristics between the ld and nld regions, the main controlling factors of the landslide and their relative contributions can be determined.

As shown in the figure below,MAP*Slope (the product of mean annual precipitation and slope), NEE*Slope (the product of the number of extreme rainfall events and slope), Asp*Relief (the product of aspect and local supply wind), and Asp (aspect) have large effects in all three regions.

Figure 6: Effect of various characteristics on landslide susceptibility

(a, d): NS study area; (b, e): NW-SE study area; (c, f): EW study area.

The bar graphs in (a–c) represent the difference in magnitude of each feature between landslide (ld) and non-landslide (nld) areas in descending order; the pie charts in (d–f) represent the average impact of each feature on landslide (ld) and non-landslide (nld) areas.

Mean annual precipitation (MAP), number of extreme rainfall events (NEE), aspect (Asp), elevation (Elev), mean curvature (CurvM), distance to river channel (DistC), all faults (DistF) and main frontal thrust and fracture zones (DistMFT), and local relief (Relief).

The asterisk * indicates the algebraic multiplication of two features.

Due to the unique capabilities of SNN,The researchers could isolate the spatial distribution of the main controlling features and their local effects.

Figure 7: Spatial distribution of each feature

ac: spatial distribution of main features.

df: Effects of climate and slope on susceptibility.

(a, d): NS study area; (b, e): NW-SE study area; (c, f): EW study area.

Areas with greater climate influence are blue, and areas with greater slope influence are red.

Mean annual precipitation (MAP), number of extreme rainfall events (NEE), aspect (Asp), elevation (Elev), mean curvature (CurvM), local relief (Relief).

The asterisk * indicates the algebraic multiplication of two features.

As shown in Figure df above, in the NS, NW-SE and EW regions,The locations of approximately 74%, 54%, and 54%, respectively, are more affected by climatic characteristics (such as number of extreme rainfall events, mean annual precipitation, and aspect) than by slope gradient, as shown by the larger blue area than the red area, indicating the importance of climatic characteristics in controlling landslides in the easternmost Himalayas.The eastern Himalayas have significant vertical climate variations due to increasing precipitation rates eastwards along the Himalayas. This climate gradient is likely to affect the landslide susceptibility in the eastern Himalayas.

SNN code GitHub address:

https://GitHub.com/geosnn/geosnn.git

SNN breaks through landslide prediction challenges

The study's authors, Louis Bouchard and Seulgi Moon, are both associate professors at UCLA, Khalid Youssef is a postdoctoral researcher at UCLA, and Kevin Shao is a doctoral student in earth, planetary and space sciences at UCLA.

Figure 8: From left to right: Louis Bouchard, Seulgi Moon, Khalid Youssef, Kevin Shao

Kevin Shao talked about “Deep neural networks (DNNs) can provide accurate estimates of the likelihood of landslides, but they cannot determine which specific variables will cause landslides to occur and why.”Co-first author Khalid Youssef said “The problem is that the various layers of a DNN constantly influence each other during the learning process, so it’s impossible to clearly analyze their results. This research hopes to be able to clearly separate the results of different data inputs, making it more useful in determining the most important factors affecting natural disasters.”

“Similar to using an autopsy to determine the cause of death, identifying the exact trigger for a landslide always requires field measurements and historical records of soil, hydrological, and climatic conditions, such as rainfall amount and intensity, which are difficult to obtain in remote areas like the Himalayas. But SNNs can identify key variables and quantify their contribution to landslide susceptibility.” Professor Seulgi Moon said. Louis Bouchard said “Unlike DNNs, which require powerful computer servers to train, SNNs are small enough to run on an Apple Watch.”

The researchers plan to expand their work to other areas of the world prone to landslides, such as California.In California, where frequent wildfires and earthquakes have heightened the risk of landslides, SNNs could help develop early warning systems that take into account multiple signals and predict a range of other surface hazards.

Reference articles:

[1]https://phys.org/news/2023-06-geologists-artificial-intelligence-landslides.html

[2]https://newsroom.ucla.edu/releases/artificial-intelligence-can-predict-landslides

[3]https://www.bccn3.com/news/ucla-geologists-develop-ai-model-to-predict-landslides

[4]https://static-content.springer.com/esm/

art10.1038s43247-023-00806-5/MediaObjects/43247_2023_806_MOESM1_ESM.pdf