Small Model, Big Breakthrough! Neural Network Sees Through Spatial Heterogeneity and Accurately Describes Complex Geographical Phenomena
To promote the universalization of AI4S, reduce the dissemination barriers of scientific research results of academic institutions, and provide a communication platform for more industry scholars, technology enthusiasts and industrial units,HyperAI has planned the "Meet AI4S" series of live broadcasts.We invite researchers or related units who are deeply engaged in the field of AI for Science to share their research results, methods and ideas in the form of videos.
In the first episode of the "Meet AI4S" series of live broadcasts,We are honored to have invited Ding Jiale, a PhD student in Remote Sensing and Geographic Information Systems from Zhejiang University,The Zhejiang Provincial Key Laboratory of Resources and Environmental Information Systems, where he works, has published a number of high-value research results in national high-tech fields such as digital earth and geographic information systems, remote sensing and global positioning systems.
This sharing,Dr. Ding Jiale gave a speech titled “Neural Networks Provide New Explanations for Spatial Heterogeneity of Housing Prices”.He shared his latest research results. This study further combined a spatial proximity measure (OSP) optimized by a neural network with a geographic neural network weighted regression method to construct an osp-GNNWR model. By solving the spatial non-stationary regression relationship between the dependent variable and the independent variable, the neural network training was realized, which can more accurately describe complex spatial processes and geographical phenomena.
Click here to view the full live replay⬇️
https://www.bilibili.com/video/BV14W42197on
HyperAI has compiled and summarized Dr. Ding Jiale’s in-depth sharing without violating the original intention.
Promoting the future development of science from the perspective of model interpretability
As an explorer of geographical science, if the model we come up with can only simply predict housing prices, then such results are boring to me.What we are pursuing is to make a reasonable scientific explanation of geographical processes or patterns by using a series of regression coefficients output by these models that vary with spatial locations.Such research is more forward-looking and practical. It is based on this vision that I chose the topic of "Neural Networks Provide New Explanations for Spatial Heterogeneity of Housing Prices" to share today.
Some time ago, our team published a research paper titled "A neural network model to optimize the measure of spatial proximity in geographically weighted regression approach: a case study on house price in Wuhan" in the International Journal of Geographical Information Science, a well-known journal in the field of geographic information science.
Paper address:
https://www.tandfonline.com/doi/full/10.1080/13658816.2024.2343771
In this study, we introduced a neural network method to nonlinearly couple multiple spatial proximity measures (such as Euclidean distance, travel time, etc.) between observation points.The optimized spatial proximity measure (OSP) is obtained, thereby improving the accuracy of the model's housing price prediction.
To solve the problems that abstract “spatial proximity” cannot construct loss functions and neural networks are difficult to train, we further combine OSP with the Geographically Neural Network Weighted Regression (GNNWR) method.The osp-GNNWR model was constructed.The neural network is trained by solving the spatial non-stationary regression relationship between the dependent variable and the independent variable. Ultimately, the model is proven to have better global performance and can more accurately describe complex spatial processes and geographical phenomena.
Next, I will use this achievement as an example to share with you the specific process of how neural networks provide new explanations for the spatial heterogeneity of housing prices.
Research background: Scientific research breakthroughs under dual challenges
"Spatial heterogeneity" is a key factor causing housing price fluctuations, but a single distance measurement method is insufficient in capturing the "spatial heterogeneity" of housing prices in a complex geographical environment; the traditional geographically weighted regression model (GWR) also faces challenges in measuring spatial proximity. It is precisely because of these factors that we chose to conduct this study.
Spatial heterogeneity: Different expressions in different spaces
First, let me give you some background on spatial heterogeneity and geographically weighted regression.
The ordinary linear regression model OLS is the most commonly used and basic statistical method to determine the regression relationship of variables. It uses a very concise formula to describe the relationship between the dependent variable and multiple independent variables. As shown in the figure below, y is equal to an intercept term plus the product of several regression coefficients and independent variables.
When we apply statistical methods such as OLS to geography,It is often necessary to consider the inherent spatial characteristics of some geographical issues.This gave rise to related research on spatial statistics and spatiotemporal modeling.
The ordinary linear regression model assumes that the regression coefficient is independent of the spatial and temporal location of the sample data, and the calculated coefficient of the independent variable is at the average level of the study area.
but,The regression relationship in real geographical processes will show differences in different spatial locations.Taking housing prices as an example, the main influencing factors of a house of the same size in the city center and the suburbs are different, so their regression relationships also have different forms.We call this characteristic spatial heterogeneity (spatial non-stationarity).
Spatial heterogeneity is an inherent feature of describing the relationship between geographic elements, and is the differential expression of the relationship or structure of geographic elements in different time and space locations. It means that the generation mechanism of data in different spatial locations is different, which will be manifested in the form of corresponding regression models, or the parameters will change with the spatial location.
Geographically Weighted Regression: Transforming from Spatial Proximity to Weights via Kernel Functions
Geographically Weighted Regression (GWR) is a modeling method for spatially heterogeneous processes proposed by American academician A. Stewart Fotheringham.
From the formula in the figure below, we can see that although the overall form of GWR is still a linear regression relationship, its intercept term and regression coefficient have become a mapping relationship with the coordinate position (ui, vi). In other words, its regression relationship is different at different coordinate positions.The regression relationship reflected by the entire formula will also change with different spatial positions.
The regression coefficients for GWR are difficult to determine.The most commonly used solution method now is similar to OLS, which uses a weighted least squares method for solution.
In the formula in the figure below, the diagonal weight matrix W is used to weight the samples, which can reflect the spatial correlation between the independent variables. Specifically,The weights between samples are calculated based on the spatial proximity of the samples.The closer two points are in space, the stronger the correlation will be. We will assign a larger weight to it and use it to build a model.
How to achieve the conversion from spatial proximity to weight?GWR uses a kernel function, such as the Gaussian kernel function, the bisquare kernel function, etc., to transform spatial proximity into a weight, thereby realizing the construction of the weight equation. However, this method has certain limitations.
In the past, the key to modeling spatially heterogeneous processes was to design and construct the spatiotemporal weight kernel function based on the spatiotemporal proximity measurement, and then use the local weighted regression theory to establish the non-stationary target solution function. By optimally solving the model evaluation criteria, the geographic modeling of spatiotemporal non-stationary relationships was achieved.
Existing research on this method also focuses on refining the scope of use of kernel functions, and then establishing a hybrid kernel model with multiple bandwidth parameters.However, the improvement and development of the structure of the kernel function itself is ignored:For example, the existing kernel function structure system with single parameter analysis as the core is relatively simple, and it is difficult to fully estimate the complex effect of spatiotemporal proximity on spatiotemporal weights, resulting in the inability to accurately solve the spatiotemporal non-stationary characteristics of complex geographical relationships.
With the continuous development of big data in recent years, we should give full play to the advantages of massive data in the big data environment and efficiently utilize the nonlinear fitting ability of deep neural networks.Using neural networks to explain spatial heterogeneity is a feasible solution to the current development dilemma of spatiotemporal relationship modeling methods.
How can neural networks be used to account for spatial heterogeneity?
Fusion of SWNN, GNNWR has stronger generalization ability
Previously, we proposed a geographic neural network weighted regression model GNNWR, which uses a deep neural network (spatial weighted neural network SWNN) to assign a series of spatial weights to samples at each location.
GNNWR paper address:
https://doi.org/10.1080/13658816.2019.1707834
Specifically, SWNN takes the distance vector from each sample point to other sample points as input and outputs a series of spatial weights at that position, that is, the weight matrix W.This enables the expression of spatial heterogeneity.
In order to have strong generalization ability on smaller samples and to make the model training converge faster, we use the GNNWR method.The weights output by SWNN are multiplied by the global regression coefficients obtained by OLS prior.Regression coefficients for spatial heterogeneity were formed.
The regression equation in the figure above is composed of independent variables, global regression coefficients, and adjustment parameters for spatial non-stationarity at the observation point. Based on this, we established a spatial regression model based on neural networks to solve the spatial non-stationary process.
Optimizing spatial proximity measures using neural networks
As mentioned above, SWNN takes the distance vector from each sample point to other sample points as input. In this process, we generally use Euclidean distance.For example, the length of the line connecting two points in space is used as a measure of distance.This is the most intuitive and easiest to understand way to express distance.
However, in urban environments,Euclidean distance is affected by natural and traffic conditions and is difficult to reflect actual spatial proximity.For example, if you want to go to the Qiantang River on the other side, if you can't take the highway bridge, you need to take a long detour to get there. In this case, although the straight-line distance between the two points is very close, they are very far apart in actual space, and the Euclidean distance cannot fully reflect their spatial proximity.
In the real world, constrained by natural landscapes and man-made objects, the exchange of people and materials often relies on road transportation networks.Road Network Distance (ND) and Travel Duration (TD) are also appropriate spatial proximity measures.
However,Due to traffic regulations and road capacity restrictions,The same length of road network distance and the same travel time represent different spatial proximity. For example, if you drive for 13 minutes, you can only travel a short distance if you follow the speed limit on campus, but you can travel a long distance if you are on an elevated bridge.
Therefore, if a single spatial proximity metric is used, there will be certain limitations.We attempt to establish a distance fusion function that couples multiple distance metrics together to optimally represent spatial proximity.
According to the above equation, we couple several "distances" between two points to form a better and more accurate value that represents the true spatial proximity between the two points.
But there is a problem with this equation. fsp is a distance representation that needs to be unified in multiple different dimensions. For example, the units of travel time and Euclidean distance are different, and the orders of magnitude may also be quite different. Simply relying on ordinary functions cannot fully achieve the coupling effect.We constructed a spatial proximity neural network SPNN,Map these distances into a unified spatial proximity metric.
Subsequently, by training this neural network, the calculation of a specific function can be transformed into a data-driven fitting process. This is our idea of using neural networks to optimize spatial proximity.
Connect two neural networks to form osp-GNNWR
Since spatial proximity is an abstract concept and has no true value, for example, given point a and point b, we cannot say that the spatial proximity between a and b is a definite value x. This makes it impossible to define the loss function of SPNN and thus to train it.
Our solution is,The output of SPNN is directly used as the distance input of GNNWR, and the two neural networks are connected to form a unified whole, which we call optimized spatial proximity measure geographic network weighted regression (osp-GNNWR).
According to this model, we can directly train the entire network through the error of the sample estimation value, and use the fitted value and the error of the added value of the final dependent variable y as the loss function to directly train the network. The entire network is trained, and the SPNN is also trained at the same time, thus solving the SPNN solution problem and completing the regression task.
Taking Wuhan housing prices as an example, osp-GNNWR provides a new explanation for the spatial heterogeneity of housing prices
Taking Wuhan housing prices as an example,We selected 968 independent second-hand housing transaction data in Wuhan and divided them into training set and test set in the ratio of 85:15.From these data, we selected 10 independent variables in three categories using the hedonic price method commonly used in housing price modeling, including basic information of these houses, surrounding supporting facilities, transportation convenience, etc. On this basis, we selected Euclidean distance and travel time as the input distance of SPNN to construct the osp-GNNWR model.
For the optimized spatial proximity metric, as shown in the figure below, the color of each point in the figure represents the residual difference of the fitting result; orange means that the fitting effect of osp-GNNWR is better than the original GNNWR model; the line represents the difference between the obtained optimized spatial proximity and the Euclidean distance.
As can be seen in Figure a, in the urban fringe area, the difference between OSP and Euclidean distance is large, and due to the influence of the road network structure, there is a certain directional difference; in particular, we can find a lower difference in the direction of the red arrow, which is mainly because this direction coincides with the Second Ring Expressway of Wuhan City.This is due to the small differences in the Euclidean distance and travel time used to construct the OSP.
Figure b shows that in the central area of the city, due to the perfect transportation facilities, no matter which direction you go, the spatial proximity in different directions is relatively balanced.Therefore, the difference between osp and Euclidean distance shows a more regular concentric circle distribution.
Through these differences between OSP and Euclidean distance,We were also able to demonstrate the practical significance of optimizing the spatial proximity measure.
Based on the modeling results of housing prices, we can further discuss the spatial heterogeneity of regression coefficients, such as studying the impact of university distance on housing prices.
As shown in the figure below, the UA parameters in the center of Hongshan District, Wuhan are significantly higher than those in other areas.This suggests that the university has a positive impact on house prices in the area.That is to say, the closer to educational institutions, the higher the housing prices. In addition, these universities and scientific research institutions also bring better living environments and create a more prosperous rental market.
Small models also have great significance
We did not use a large model in the above research. Although large neural network models and deep network models are very popular now, small models still have their practical significance. In the absence of so much computing power and rich data set samples, designing a small and beautiful model can also be of great help in solving certain problems.
Finally, there are some references. If you are interested, you can also check them out.
Call to action
HyperAI (hyper.ai) is China's largest search engine in the field of data science. It has long focused on the latest research results of AI for Science and has interpreted more than 100 academic papers in top journals.
Research groups and teams that are conducting research and exploration around AI for Science are welcome to contact us to share their latest research results, contribute in-depth interpretation articles, and participate in the Meet AI4S live broadcast column. More ways to promote AI4S are waiting for us to explore together!
Add WeChat: HyperaiXingXing (WeChat ID: Hyperai01)