Remote sensing for road detection Part 2: Improving the model and detecting changes
Near Infrared and Normalized Differential Vegetation Index bands improve predictions of SkyTruth’s automated road detection model.
(This is the second in a two-part blog series. You can read Part 1 here.)
In Part 1 of this blog series, I discussed how the SkyTruth intern team altered a machine learning model that detects oil and gas well pads to detect roads. In this post, I’ll explore whether adding spectral bands such as Near or Shortwave Infrared to the road detection model would improve the model’s predictions. I will also illustrate the model’s ability to detect newly constructed roads in the Neuquen Basin of Argentina.
To add bands to the model, we first needed to determine if the training data can hold additional information and if the model itself can accommodate a larger input. It turns out that these are both simple adjustments: we can easily expand the dimensions of both the training data and the U-NET model. We also needed to make sure that all the included bands in the training data have the same pixel size, also known as spatial resolution. Sentinel-2’s Red, Green, and Blue bands have 10 meter wide pixels, so we could easily include the Near Infrared (NIR) band which also has 10 meter pixels. We also calculated an additional 10 meter resolution band of Normalized Differential Vegetation Index (NDVI) to give the model even more information about how vegetation contrasts with road features. NDVI is a common measure of vegetation greenness derived from satellite imagery by calculating the ratio between the Red band and the NIR band. The team also experimented with the Shortwave Infrared (SWIR) band, but because its spatial resolution was 20 meters, we needed to rescale all the other bands up to 20 meter wide pixels for consistency, and this resulted in losing critical details in the images. As a result, we did not incorporate the SWIR band into the model.
Second, we needed to determine if this additional information would enhance the model’s predictions. This comes down to examining what value the NIR and NDVI bands add. Both infrared and optical imagery rely on reflectance of light back to the satellite, but infrared light interacts with materials in ways the human eye can’t see because it is outside of the visible portion of the electromagnetic spectrum. For example, vegetation is highly reflective in this range of wavelengths, while concrete and asphalt are more absorbent. Additionally, soil has a slightly higher spectral reflectance than concrete in the NIR band, which explains why the paved and dirt roads are different shades in Figure 1.
Figure 1. A visualization of a Colorado landscape through Sentinel-2’s NIR band. The paved highway and the steel railroad tracks appear dark because those materials absorb near infrared light, so those pixels have lower reflectance values. The dirt roads appear white since they are more reflective of near infrared light, so their pixel values are high.
The NDVI band does not distinguish between the type of road surface like the NIR band, but it does offer an excellent measure of vegetation. The surface of a road almost always has less vegetation than its surroundings, so roads have very low NDVI values and therefore show up as black in images visualizing the NDVI band like Figure 2.
Figure 2. A visualization of the NDVI band in a forested region in Kentucky, where the paved roads are in sharp contrast with the vegetated surroundings.
Adding these extra bands of information to the training data increases the complexity of our model. The model will now try to detect patterns among six layers of pixels (red, green, blue, NIR, NDVI, and mask) instead of four. Figure 3 graphs the training loss of both the original model and the new model. While I was initially surprised to see that the new model recorded larger loss (i.e. error) values throughout training, it quickly struck me that since we had added more complexity and nuance to the model, the learning process would be more challenging and therefore incur slightly larger errors. Over time, as the model learned from its errors, its predictions improved and the two models’ losses nearly converged by the 75th epoch.
Figure 3. The sharp decrease of both loss curves tells us that the models quickly learn to generally distinguish roads within the first five epochs, and then gradually fine tune themselves to learn the exact shape and composition of roads over the rest of the training epochs.
Despite the higher errors, I still maintained hope that the new model would see improvements when it came to detecting roads in entirely new regions such as the test areas in Colorado, Kentucky, and California shown in the first blog post. But before we could assess the accuracy of the model in those areas, we needed to digitize the actual roads in the test images. From there we could compare the actual roads with the model’s predictions in QGIS to evaluate how well the two layers match up. (For a refresher on how we handle and process the model’s outputs, revisit Ry Covington’s blog post!) In order to quantify this accuracy we picked three metrics commonly used in classification tasks: precision, recall, and F1 score. Figure 4 provides a visual explanation of the layers involved in calculating those metrics.
Figure 4. The leftmost panel shows the actual, true roads of a test image in blue. The next panel shows the model’s predictions in orange. The third panel illustrates true positives, the road pixels the model predicted correctly, in green, and false positives, the pixels the model incorrectly classified as road, in magenta. The rightmost panel shows the true positives alongside the false negatives, the road pixels the model didn’t detect.
The equations under the last two panels in Figure 4 describe how precision and recall are calculated. Precision compares the true positives to the falsely predicted roads, effectively answering the question: out of all the model’s predictions, what percentage of them are correct?
Recall compares the true positives to the false negatives, answering the question: out of all the actual roads, what percentage did the model predict? In other words, precision measures specificity, while recall measures sensitivity.
The F1 score is derived as the harmonic mean of precision and recall, so it measures the model’s ability to be both specific in delineating roads and sensitive to detecting all the roads in the image. Figure 5 illustrates the F1 score equation after some arithmetic steps are completed.
I
Figure 5. Two variations of the F1 score equation. If the expression on the right looks familiar, it’s because we’ve seen it before in the first blog post as the Dice coefficient! This makes F1 score a good metric to evaluate the model’s outputs, since it is the same metric that was a component of the loss function the model optimized during training.
Figures 6-8 compare the two model’s predictions in three test images: Colorado, California, and Kentucky. In each figure, the ground truth roads are shown in blue, and the old model’s predictions on the left side are shown in yellow, while the new model’s predictions on the right side are shown in orange.
Figures 6, 7, 8. In all three images, the newer model notches a higher F1 score. The Colorado and Kentucky F1 scores only increased marginally because while the NIR-NDVI model’s recall was 15% higher than the Red-Green-Blue model in those areas, both saw a dip in the precision, bringing the F1 score back down. This informs us that the additional bands do help the model capture more of the roads in an image, but some precision is compromised as the new model detects more false positives.
One potential use for a tool like this is monitoring the development and expansion of fossil fuel infrastructure; specifically the roads of natural gas drilling projects. Natural gas, also known as methane, releases less carbon at its point of combustion than oil and coal. This fact has made methane an appealing fuel and has advanced the narrative of methane as a “bridge fuel” to catalyze the transition away from fossil fuels while renewable energy capacity scales up. However, methane creates its own problems that contribute to climate change. Persistent methane leaks from wells and pipes largely go unaccounted for in calculating emissions, and methane is a far more potent greenhouse gas than carbon dioxide. Modern gas drilling — commonly paired with hydraulic fracturing — also has been connected to a long list of health problems in local communities (see for example, some of the impacts documented in highly fracked counties in Pennsylvania). Scientific entities such as the International Energy Agency (IEA) and the Intergovernmental Panel on Climate Change (IPCC) have issued statements calling for a decrease in natural gas in the energy sector. In their 2021 Roadmap to Net Zero, the IEA states that no additional gas should be developed and that facilities should work on reducing emissions from their current operations. In Chapter 2 of their 2018 Special Report, the IPCC estimates that gas needs to drop its current share of the global energy sector from 23% to 13% by 2050 to limit global temperatures by 1.5 ℃ of pre-industrial levels.
Despite these documented harms and international warnings, natural gas projects are still being planned and funded by governments and global financial institutions across the world. One area where SkyTruth monitors this development is the Neuquen Basin in Patagonia, Argentina. The basin lost 20,000 hectares of former farmland to oil and gas extraction in the past 30 years, and this development has caused health problems and physical damage to the lands and homes of inhabitants such as the indigenous Mapuche community. Sadly, this expansion is still occurring, with more drilling well pads and access roads being constructed every year. We ran the revised road detection model on the region twice, once with imagery from 2019 and again with imagery from 2020 (See Figure 9). Once we had the two sets of road predictions, we took the difference between 2020 and 2019 to reveal new road construction (See Figure 10).
Figure 9. Annual composite images showing an increase in well pads and roads from 2019 to 2020.
Figure 10. The white layer depicts the difference between the 2020 predictions and the 2019 predictions, showing specifically where new roads have been made. In this region spanning 180 square kilometers, our model detected 39 kilometers of newly built roads in a year.
While it’s no secret that drilling in this region has been growing for years, a monitoring tool like our road detection model can visualize this expansion in a way that helps raise public awareness and inform regulatory processes. This ability to track the activity of oil and gas companies will be crucial in the coming years as the current path of development and extraction will increasingly conflict with the urgent need to decrease emissions across the globe.
As my year-long journey in the SkyTruth internship program comes to a close, I am proud to have taken this work in an important new direction. More so, I am grateful for the technical support and emotional reassurance that my colleagues Brendan Jarrell and Ry Covington offered throughout the process and how they trusted me to carry out this project even when I ran into challenges. I am also thankful for the support of my fellow interns Hillson Ghimire, Dylan Thomson, Dhiya Sathananthan, Cameron Reaves, and CJ Manna for helping create training data and brainstorming directions for improving the model. Through these two blog posts I hope I have communicated our technical workflow simply, triggered interest in how roads appear on imagery, and inspired readers to imagine other possibilities for combining remote sensing and machine learning. Ultimately, I am appreciative of the opportunity SkyTruth has given me to become a technical steward of our land, water, and air through various projects in the past year, and I plan to carry the lessons with me onward in my career and life.
(Note: Title was adjusted 7/20/21)
