Can a Smartphone Be Used As an Air Quality Monitor? A Guide From Several Engineering Students
In response to the problem of air quality detection, a team from the Indian Institute of Technology trained an AQI estimation model in the cloud. By taking pictures with a mobile phone, people can learn about the air quality and effectively obtain information on smog weather.
Written by Prerna Khanna et al., Indian Institute of Technology
Compilation | Little Walnut, Black Walnut Laboratory
This article is authorized to be reproduced from the official account "Black Walnut Laboratory"ID: blackwanutlabs, thanks to the author and translator!
Did you know that there is a kind of cough called "smog cough"? There is also a color called "smog color"? After struggling in the smog for a long time, everyone is also making fun of this headache-inducing air quality out of helplessness.
In addition to cities such as Beijing, Lanzhou, and Chengdu in China that have been troubled by air pollution for years, foreign cities such as Delhi, India, Seoul, South Korea, and Chiang Mai, Thailand are also facing serious air pollution problems, especially in winter.
In winter, air pollution not only limits people’s activities, but also causes health problems. According to media reports, air pollution has killed 8.8 million people worldwide.
The world’s air pollution problem is serious, and the first step to solving it is to enable people to monitor the quality of the air they breathe.
While monitoring air quality can be done with pollution sensors, large-scale deployment is expensive. A team of students from the Indian Institute of Technology has designed a reliable and cheap solution for assessing air quality, allowing people to monitor it with their smartphones.
PM 2.5 estimation using mobile devices
Real-time analysis of air quality based on mobile phone photos
Previous studies have demonstrated that machine learning can effectively use camera images to assess air quality, but they were usually limited to images from a few static cameras.
A team of students from India hopes to provide local, real-time air quality assessments on an Android-based mobile app using images taken with a mobile phone.
They chose to target PM 2.5 and smaller particles as the standard for air quality assessment. To visualize the results, the PM 2.5 forecast values were mapped onto a color gradient air quality index (AQI) table, which is a standard scale set by governments in various countries. The corresponding pollution warnings were then displayed based on the AQI value.
Predicting air quality using TensorFlow Lite
The specific steps are that the application collects images from the phone camera, and then Tensorflow Lite processes them on the device to provide AQI prediction values. Before developing the application, they trained an AQI estimation model in the cloud. The model can be automatically downloaded in the Android application using Firebase ML Kit.
The system consists of the following parts:
1. Mobile application: used to capture images and predict AQI levels, the application processes the images on the device.
2. TensorFlow Lite: used to make predictions using the trained model, and it itself is a very small binary file (which is important for download speed when bandwidth is limited).
3. Firebase: The parameters extracted from the image (described below) are sent to Firebase. A unique ID is created for each new user when they use the app. This can be used to customize the machine learning model for different geographic locations later.
4. Amazon EC2: The model was trained on Amazon EC2 using various geo-location parameters and PM values.
5. ML Kit: The trained model is hosted on ML Kit and automatically loaded onto the device, then run using TensorFlow Lite.
Two models
They trained two image-based machine learning models: one to predict AQI using features of user-uploaded photos and one to filter out images without skylines.
AQI Model
Based on user photos, they use the following features to predict the AQI: These features are extracted through traditional image processing techniques and combined through a linear model. The second model (discussed later) uses the image directly.
Transmission: describes the amount of light that enters the phone camera after scene attenuation and reflections from air particles. It can be described by this equation:
where I is the observed blurred image, t is the transmission from the scene to the camera, J is the scene radiance, and A is the air color vector.
Using the concept of dark channel, the transmission of a single hazy image is found by assuming that there are at least some pixels with zero or very low intensity in one color channel in all outdoor images. For the haze-free image J, the dark channel is:
where Jc is one of the color channels of J and Ω(x) is a local patch centered at x. The airlight can be estimated from the sky or the brightest region, so the transmission can be obtained as:
Where Ic(y)/A is the blurred image normalized by the air light A, and the second term on the right is the dark channel of the normalized blurred image.
Sky color: If the sky is grey, we consider it a day with polluted air. The blue color can be estimated using RGB segmentation.
Gradient of the sky: Considering that the sky may appear grey due to cloud cover, this feature is incorporated. The gradient is calculated by making a mask of the sky area and then calculating the Laplacian of this area.
Entropy, RMS contrast: These characteristics indicate the details contained in the image. If it is polluted, the image will lose its details. RMS contrast is defined as the standard deviation of the image pixel intensities. Here is the equation for RMS contrast:
Where Iij is the brightness at the (i, j) pixel of an image of size M×N, and avg(I) is the average brightness of all pixels in the image. Therefore, the contrast is inversely proportional to PM 2.5. To estimate the entropy, the following equation is used:
Where pi is the probability that the pixel intensity is equal to i, and M is the maximum brightness of the image. As the PM concentration increases, the details of the image are gradually lost and the image entropy decreases. Therefore, it is inversely proportional to PM 2.5.
Humidity: Studies have concluded that pollution levels rise on humid days because PM 2.5 absorbs moisture and reduces visibility.
Skyline Model
Can users use the app to predict the AQI inside their room just like it predicts the AQI outside their house?
The model can predict an image if it contains at least 50% of the skyline and will accept the skyline image by using a binary classifier.
They created this classifier using Transfer Learning and retrained the model on their labeled dataset using TensorFlow Hub. The dataset consists of 2 classes: 500 images with 50% skylines, 540 images without skylines (or less than 50% skylines), including rooms, offices, gardens, outdoor scenes, etc. Then using the MobileNet 0.50 architecture, 95% accuracy was achieved when testing 100 unseen samples. TF for Poets provided assistance for image retraining.
The confusion matrix of the retrained model is as follows:
Left: Image with a skyline larger than 50%. Right: Image with a skyline smaller than 50%.
User-defined models
Because each smartphone’s camera specs vary, each user needs a custom machine learning model. To train such a model, they collected images of each user.
They decided to combine the image-based model and the temporal model using meteorological parameters. The temporal model using meteorological parameters helped to achieve higher inference accuracy and provide some results to the users while training the image-based machine learning model, while the image-based machine learning model helped us to customize the model for a specific user, thereby improving the inference accuracy by reducing the prediction error.
To create a small training dataset for each user, they extract features from 7 images for training. The images must be taken over 7 consecutive days, with half of them covering the sky and without direct light sources such as the sun. After extracting features from the images, they are used to train a regression model. The model is linear because all image features are more or less linearly proportional to the PM 2.5 value.
After the training dataset and model are created, a second set of images for testing is created. Once the dataset has 7 days of different image features, testing begins. If the training RMSE is less than 5 days, the model is frozen and sent to ML Kit, which is downloaded in the app. If the RMSE is not less than 5 days, more training data is collected.
Relationship between image features and PM 2.5
Meteorological parameters
To improve the inference accuracy, they also used a temporal model of meteorological data to predict AQI based on the historical AQI of recent locations and supplemented the temporal model with an image-based model.
The team collected a meteorological dataset for Delhi from the Indian government website from 2015 to 2017 and performed ridge regression using LASSO optimization to select key parameters affecting PM 2.5 levels.
The key parameters selected were: PM 2.5 concentration in the previous hour, concentrations of various gases such as NO2, SO2, O3 and dew point. These data were then trained and tested separately. The accuracy of its data set reached 90%.
The line graph shows the RMS error values given by the three models over a 21-day period.
The next challenge was to host the adaptive image-based model for each user. To do this, they used a solution from Firebase ML Kit. It allows custom and adaptive ML models to be hosted in the cloud and on the device.
How far can this simple but effective system go in detecting or controlling smog? Let us look forward to their further exploration and improvement!