Transfer Learning Helps! Chengdu University of Technology Builds SCDUNet++ Model for Landslide Mapping
Landslides are one of the most common natural disasters, usually triggered by earthquakes and rainfall, causing serious property losses and casualties. The damage caused by landslides triggered by earthquakes is sometimes more serious than the damage caused by the earthquake itself. After a large earthquake, rapid and accurate landslide mapping (LM) is essential for emergency rescue, timely quantitative disaster assessment, and post-disaster reconstruction.
In recent years, a lot of research has been done on methods for automatically drawing landslide maps from remote sensing images. However, due to the large differences in characteristics and scales of landslides and the similarities in optical remote sensing images, existing methods face various challenges in accurately mapping landslides.
to this end,Researchers from Chengdu University of Technology proposed a semantic segmentation model called SCDUNet++, which combines the advantages of convolutional neural networks (CNN) and Transformer to enhance the recognition and extraction of landslide features. Its performance is better than that of 8 other deep learning models such as FCN, DeepLabv3+, Segformer, etc., and it improves IoU by 1.91% to 24.42% and F1 by 1.26% to 18.54%.This result has been published in the International Journal of Applied Earth Observation and Geoinformation. Note: IoU is the intersection over union ratio, which measures the overlap between the predicted area and the true area; F1 is the harmonic mean of precision and recall, which measures the accuracy and completeness of the model.
Research highlights:
* Draw landslide maps using semantic segmentation models of multi-channel data
* Improving landslide mapping using topographic and spectral index factors
* After deep transfer learning, the performance of the model is greatly improved in data-scarce areas
* The proposed model outperforms other models in terms of drawing and transferability
Paper address:
https://doi.org/10.1016/j.jag.2023.103612 Dataset one-click download:
https://hyper.ai/datasets/29647
Follow the official account and reply "Landslide" to get the complete PDF
Experimental process: Building the SCDUNet++ model
Dataset: Luding and Jiuzhaigou earthquake data
The 6.8-magnitude Luding earthquake in 2022 and the 7.0-magnitude Jiuzhaigou earthquake in 2017 both caused serious landslide disasters, so the researchers chose these two areas for experimental research.
The dataset contains three parts, which are integrated and stored in HDF5 format.
Sentinel-2 multispectral data:The data for Luding area are from 2022/03/15 and 2022/11/25, and the data for Jiuzhaigou are from 2017/07/29 and 2017/09/07.
NASADEM (Digital Elevation Model) data:Download data from NASADEM's official website to obtain data such as slope, aspect, curvature, mountain shade, and terrain wetness index (TWI).
Landslide inventory:Sentinel-2 optical remote sensing images and Google Earth images were combined and manually annotated by experts.
Model architecture: three major modules
The following figure shows the overall architecture of the constructed SCDUNet++ model:
GLFE (Global Local Feature Extraction) module: As shown in Figure b, the GLFE module combines the shallow CNN structure and the deep Swin Transformer structure to effectively handle the problem of landslide image feature extraction and classification.
DSSA (Detailed Spatial Spectral Aggregation) module: By fusing the spectral and spatial features from the GLFE module, the ability to process complex information in multispectral images is enhanced. This module can perform more accurate spectral and spatial feature extraction, enhancing the efficiency and accuracy of the model for multispectral image analysis.
DSC (Dense Skip Connection) : As shown in Figure d, this module is able to restore the resolution of features at each stage.
Finally, the features of each stage are weighted and fused to obtain the final result.
Algorithm training: DTL method based on simple model
The researchers introduced the deep transfer learning (DTL) method to improve the migration performance of LM and the model. The algorithm training is designed to effectively handle the unbalanced distribution of landslides and backgrounds in remote sensing images while ensuring the efficiency of the model in multiple performance indicators.
Experimental results: After transfer learning, the model effectively carried out landslide mapping
Model evaluation: effectiveness of combining GLFE and DSSA modules
The researchers used UNet++, a commonly used method for landslide mapping, as the baseline, and Luding area I and II as the experimental groups. They conducted an ablation experiment to compare the models with different modules added, focusing on the two overall indicators of IoU and F1.
As can be seen from the following table:
* Effectiveness of the GLFE module: Compared with the baseline, the IoU and F1 are improved by 1.7% and 1.16% respectively.
* Effectiveness of the DSSA module: Compared with the baseline, IoU and F1 are improved by 1.88% and 1.28% respectively.
* SCDUNet++ model: Compared with the baseline, IoU and F1 are improved by 2.83% and 1.92% respectively.
In summary, the researchers added the GLFE module and the DSSA module to the baseline model UNet++ to form the SCDUNet++ model. The SCDUNet++ model retains small landslide images for better performance, improving the model precision by 0.46% and the recall by 4.06%.
a: Baseline
b: Baseline + GLFE
c: Baseline + DSSA
d: Baseline + GLFE + DSSA
Luding test: SCDUNet++ demonstrates superior performance
In the Luding I and II test areas, the researchers compared the landslide mapping work of the SCDUNet++ model with that of the other eight models, revealing the sensitivity of SCDUNet++ to specific geographical features and environmental complexity.
The results in the figure below show that in test area I, SCDUNet++ exhibits high performance in precision, recall, MCC, IoU, MIoU and F1 score.This shows that the model can still maintain high detection accuracy in relatively complex geographical environments.The experimental results of test area II with a relatively simple background also confirmed the adaptability of SCDUNet++ in different environments, especially in terms of recall rate and MIoU.
In the visualization results, the SCDUNet++ model shows excellent performance, as shown in the following:
Landslide Mapping Completeness:SCDUNet++ is able to generate relatively complete landslide maps. In contrast, models such as FCN, Segformer, TransUNet, and UNet miss some small landslide images.
Handling of FP (false positives, i.e. negative classes that are incorrectly predicted as positive classes): Since the bare soil area and landslide characteristics are very similar, all models have different degrees of FP, but SCDUNet++ can suppress the generation of FP to a certain extent based on a specific feature extraction module.
Processing of Boundary Areas: The researchers found a large number of FNs (false negatives, i.e. positive classes that were mistakenly predicted as negative classes) and FPs in the boundary areas of landslides and backgrounds, but SCDUNet++ uses a nested decoder that can recover and fuse features at all levels and perform deep supervision, effectively solving the problem of inaccurate boundaries.
The white rectangles in A, B, C, and D show the main differences between the results.
a: Test area I
b: Test Area II
In general, the SCDUNet++ model shows great potential for landslide mapping in Luding area, especially in handling complex environments and accurate boundary determination.
Jiuzhaigou test: Model predictions after transfer learning are more accurate
Directly use the model trained in Luding area
Indicator comparison: SCDUNet++ outperforms other models in most indicators. However, it should be considered that the characteristics of the Jiuzhaigou test area are relatively simple, and the model test results should be better than those in the Luding area, but the indicators are not very ideal.
Visualization results: The figure below shows that the mapping results have few FPs (blue areas), but many actual landslides are missed.
Model optimized with deep transfer learning
Improved indicators: After deep transfer learning, all indicators showed significant improvements in test areas III and IV, especially the recall rate and F1 score increased significantly. The SCDUNet++ model still performed best in multiple indicators.
Visualization results: After deep transfer learning, the number of FN (false negatives, i.e., positive classes that are incorrectly predicted as negative classes) is significantly reduced. The model effectively identifies and maps large landslides, reducing missed detections. At the same time, the model LM results include most small landslides.This shows that the SCDUNet++ model after deep transfer learning has better recognition ability for small and complex landslide features.
a: Test area III before DTL
b: Test area III after DTL
c: Test area IV before DTL
d: Test area IV after DTL
National Key Laboratory in the Field of Geological Hazard Prevention and Control
Chengdu is located in the western part of the Sichuan Basin, with a complex geological structure. The local area and surrounding areas are often affected by earthquake disasters. It can be said that Chengdu has always been fighting on the front line of disaster prevention and reduction, and Chengdu University of Technology is one of the earliest backbone forces to join the "battle".
In 1989, the former State Planning Commission and the State Education Commission approved the establishment of a national professional laboratory based on the national key discipline of geological engineering at Chengdu University of Technology (formerly Chengdu Institute of Geology).This is the predecessor of the State Key Laboratory of Geological Hazard Prevention and Geological Environment Protection. In 2001, the laboratory was approved by the Sichuan Provincial People's Government as a "Sichuan Provincial Key Laboratory". In 2002, it became a "National Key Laboratory Cultivation Base" jointly built by the Ministry of Science and Technology and Sichuan Province. In 2003, it was approved as a key laboratory of the Ministry of Land and Resources. In October 2007, it was approved by the Ministry of Science and Technology to be included in the National Key Laboratory Construction Plan. In December 2010, it passed the acceptance of the Ministry of Science and Technology.
In addition, the laboratory is currently the only national key laboratory in the field of geological disaster prevention and control in my country. In 2017, it was named the "National Land and Resources Science Popularization Base" by the Ministry of Land and Resources and the Ministry of Science and Technology, and in 2020 it was selected as the twelfth batch of provincial science popularization bases in Sichuan Province.
Once a landslide occurs on a mountain that has stood for thousands of years, rocks and soil will flow along the inclined mountain like flowing water, instantly burying a village several kilometers away. Behind the disaster is a mystery waiting for researchers to explore. Over the years, laboratory researchers have gradually accumulated a complete set of theoretical and technical systems for geological disaster prevention and control and geological environment protection from on-site investigations, simulation assessments, monitoring and early warning.
In November 2023, researchers published a paper in PNAS, making important progress in the study of stick-slip instability of earthquakes and landslides; in September 2023, a paper was published to study the stick-slip precursor characteristics of earthquakes or landslides, providing prediction information for upcoming earthquakes or landslides; in January 2023, the mystery of super-strong fluidization of high-speed long-distance landslides was solved...
Since ancient times, human beings have been "persecuted" by natural disasters many times, such as earthquakes, tsunamis, rainstorms, droughts, etc., which have caused countless families to be displaced. Even in today's rapidly changing technology, it is difficult for us to compete with natural forces, and we are more concerned with prevention and early warning. Especially with the rapid iteration of technologies such as AI and big data, the data accumulated by past disasters has become a valuable nutrient, helping various prediction models to improve their precision and accuracy, so as to better serve the current society.
We are always willing to believe that man can conquer nature, so humans have never retreated in the face of natural disasters. We believe that technologies such as AI will also become a solid shield for humans to resist natural disasters and protect the safety of life and property.
HyperAI's official website has uploaded data sets related to natural disasters and geology
* xBD Natural Disaster Image Dataset:
https://hyper.ai/datasets/13272
* RSSCN7 Data Set Remote Sensing Image Dataset:
https://hyper.ai/datasets/5440
* European Flood 2013 European flood dataset:
https://hyper.ai/datasets/21579
References:
https://www.sklgp.cdut.edu.cn/