Matthew Ibarra’s Final Project Was His Favorite

As a SkyTruth Intern, Matthew Ibarra learned new skills and helped protect Native lands.

As I finish up my internship at SkyTruth, I can honestly say that the experience has been everything I imagined it would be and more. My time here was a perfect amalgamation of what I love: namely, an organization that applies technology and gathers and analyzes data to protect the environment. 

When I started my internship at SkyTruth I was unsure of what to expect. I remember the first day I drove into the small town of Shepherdstown, West Virginia. I was worried. For the first time in my life I was working with like-minded individuals with special talents and skills far above my own. I thought that I would have to perform as well as my colleagues right off the bat. However, my fears quickly melted away upon meeting Brendan Jarrell, SkyTruth’s Geospatial Analyst and father to all us interns. Brendan assured me that I would be focusing on my own personal interests, developing practical skills, and applying them to the various projects happening at SkyTruth. Within my first week I became familiar with all the programs I needed for my internship, namely Google Earth Engine and QGIS. Both are programs that are critical in geospatial analysis that were completely new to me, despite having taken Geographic Information System (GIS) courses at West Virginia University. Interning at SkyTruth opened my eyes to the new possibilities of environmental monitoring and I was excited to get started.

My very first day I became familiar with the various satellites that orbit the Earth and provide the imagery that SkyTruth uses on a daily basis. The Landsat and Sentinel satellite missions provide imagery available for free to the public, allowing interns like myself to create maps and interactive data to track activity on Earth. My first task as an intern was to monitor Southeast Asian waters for bilge dumps — oily slicks of wastewater dumped in the ocean by ships. I used Google Earth Engine to access the necessary imagery easily. Then I used QGIS to create the various maps that we post on our Facebook page and blog posts. I found my first bilge dump on February 7, 2020. It was a 35 kilometer slick (almost 22 miles long) off the coast of Malaysia. 

Often, we can identify the likely polluter using Automatic Identification System (AIS) to track vessels at sea. Most vessels constantly send out a radio signal to broadcast their route. When those signal points align with a bilge dump it suggests that the ship is the likely source for that bilge slick. However, not all ships will transmit their signal at all times, and there have even been instances of ships spoofing their signal to different locations. For my first slick I was unable to match a ship’s AIS broadcast to the trail of the bilge dump, but I was able to do so several  times after that. We can’t know for certain who caused this slick, but imagery helps us paint a picture of potential suspects. My first slick pales in comparison to the many slicks I found in later months: later, I captured a few slicks that were over 100 kilometers (more than 62 miles) in length. I was also able to link a ship’s AIS broadcast to the trail of the slick. You can read more about slicks in the South East Asia region in my April 15 blog post here.

 

An example of likely bilge dumping from a vessel identified using Sentinel satellite imagery

Following my introduction to our bilge dumping detection work, I was thrilled to be assigned my first solo project for SkyTruth — updating SkyTruth’s FrackFinder. FrackFinder is an ongoing project at SkyTruth. It aims to keep track of the active oil and natural gas well pads in states such as West Virginia. Drilling permit data is often misleading; sites that are permitted to be drilled may not actually be drilled for several years. In the past, our FrackFinder app was hosted in Carto. Carto is a cloud-based mapping platform that provides limited GIS tools for analysis. I was tasked with giving the application an overhaul and bringing it into Google Earth Engine, a much more powerful and accessible program. 

Learning to code for Earth Engine was challenging for me. I had only one computer science course in college, and that was nearly three years ago. So I was surprised that my first project would revolve around coding. Initially, I was overwhelmed and I struggled to find a place to start. As time went on I slowly became more comfortable with spending large amounts of time solving tiny problems. Brendan was incredibly helpful and patient with teaching me everything I would need to know to be successful. He always made time for me and assisted me with my code numerous times. My finished app is far from perfect but I am proud of the work that I accomplished and I hope that it brings attention to the changing landscape of West Virginia caused by oil and natural gas drilling using hydraulic fracturing (fracking). 

 

The FrackTracker app for West Virginia

My second and final project was creating a visualization about the land surrounding Chaco Culture National Historical Park in New Mexico. Much like the update to the FrackFinder App, it involved the changing landscape surrounding the park due to the increase in fracking. I was tasked with creating a series of still images, an embeddable GIF which shows an animation of the rapid increase in drilling, and an app on Earth Engine that allows the user to zoom in and visually inspect each individual well surrounding the park. In the final months of my internship, I became comfortable using the programs that were foreign to me initially. I created a series of 19 images using QGIS from the years 2000-2018. You can see the collection of images for each year here. SkyTruth’s Geospatial Engineer Christian Thomas assisted me in creating the GIF. 

This project was special to me because I was able to help activists who are advocating for the passage of the Chaco Cultural Heritage Area Protection Act, legislation passed by the U.S. House of Representatives that would effectively create a 10-mile buffer zone surrounding the park and ensure the protection of the area for Native Americans and local communities for generations to come. The Senate has not yet passed the act. When I started my internship at SkyTruth I never would have believed that I would be advocating for protection of Native lands. I always believed issues like these were too big for one person to tackle, but if there’s anything I learned at SkyTruth is that only one person can create real change.

The growth of oil and gas wells within a 15-mile radius of Chaco Culture National Historical Park from 2000 – 2018

After interning at SkyTruth for the past eight months I am happy to say that I feel I have made a difference in the world. I accomplished so much that I thought would be impossible for me initially. I used to think oil slicks were tragedies that happened infrequently, limited to a few times a decade. I was shocked to learn that oily wastewater gets dumped into the ocean so frequently that I was able to capture more than  80 bilge dumps in my eight months at SkyTruth. 

In addition, one of my greatest passions is sustainable energy. I was thrilled to be an advocate for clean energy by showcasing the dangers of an ever-expanding oil and natural gas industry. West Virginia has been my home for the past five years during my time at West Virginia University and I was happy to be able to bring to light one of the growing concerns of the state through the 2018 FrackFinder update. Finally, I was able to advocate for the protection of Native lands through the most meaningful project to me — the Chaco Culture National Historical Park visualizations. It felt incredible fighting for something that was much bigger than myself. As I leave SkyTruth, I will miss contributing to the world in my own way.

SkyTruth has always been more than just a place to intern at for me. I have made unforgettable connections with my colleagues despite the various challenges that we all have to face every day, such as the ongoing COVID-19 pandemic. Never once did I feel that I was alone in my work. I always knew there were people supporting me and encouraging me in my projects even when I was working remotely. I will never forget Christian’s tour of Shepherdstown on my first day or Brendan’s talks about the best Star Wars movie. I cannot thank each of them enough for the patience and kindness they showed me in my short time with them. Everyone at SkyTruth has contributed to my success in some way. I will miss everyone, but I’ll carry my new skills and experiences with me for the rest of my life.   

On Considering The Larger World Around Us: The SkyTruth Intern Experience

Bilge dumping and more allowed Tatianna Evanisko to think big at SkyTruth.

SkyTruth seemed like a great fit. I had always been interested in data, the inductive route, experiencing things firsthand and then exploring my assumptions. I was compelled by computation as well as the natural world. Being active in environmental protection was important to me and I had always been drawn to vocations with a larger purpose; that allowed me to be visionary and have big dreams. Growing up on the tail end of the Millennial generation, I had experienced an explosion of technology, becoming what some have coined a “screenager.” Not only that, but I had grown up in the climate change generation. In recent decades we’d seen an increase in extreme weather events, environmental atrocities, and lost species. But notably, my generation also has been active in movements that strive to address these problems such as eating less meat, using alternative energy, and living more sustainably. Even at a time of climate conspiracies and fake news, several million people globally participated in the largest climate protest in history in 2019. That’s not to say I believe all that I hear. However, over time and by paying attention to environmental events occurring all over the world, I find the evidence overwhelming: the Earth is changing, and more and more people are bearing witness to it.

When I started as an intern at SkyTruth I was asked what issues I cared about to help me decide what to work on. My reaction was: everything of course! How can you ask such a thing? To a certain extent, my options were already defined: most of the SkyTruth staff were using satellite imagery, and despite the other issues we were working on, at some point we were all looking at the ocean — the eerie, non-terrestrial world — often in search of pollution, such as oil.  My work quickly became focused on searching for streaks of oil in the middle of vast oceans. Oil can appear on radar satellite imagery as a uniform dark and linear formation, called a slick. Many of these slicks come from cargo vessels and tankers that dump their untreated oily waste from the bottom of their ship (the bilge) into the ocean, an act called bilge dumping. Our team has been developing a solution that expands the capacity of SkyTruth to automate the detection of these slicks by using machine learning, a type of artificial intelligence. In a matter of months, I helped turn an empty spreadsheet into a collection of over 330 images of oil slicks — training data that we could use to “teach” computers to recognize the slicks in our prototype of a monitoring platform named Cerulean. Wow — intelligent and creative minds at work, which will soon enable anyone globally to monitor the sea to detect oil slicks with SkyTruth! 

Ocean monitoring thus became a routine event for me, and naturally I started to notice some patterns. I learned the locations of energy infrastructure as well as the largest shipping routes and ports, and this meant I also realized when the environment changed. For example, we found oil appearing in regular areas at sea all over the world. On a weekly basis, we discovered  obvious oil slicks where the normally smooth grey (on a radar image) ocean was instead splattered with black streaks. I also tracked  some of the vessels I believed were responsible for the oily waste and they shared something in common: many were registered under flags different from their country of ownership. I wanted to know more. Why were ships dumping in the same places — what was it about those areas that was attracting them? Did those vessels have something in common? Who was responsible for making the choice to dump pollution — the crew, the vessel operator, or the vessel company? This was the catalyst for my largest project at SkyTruth, a multi-month pursuit to understand the scale, impact, intentions, and potential solutions of the dumping of this untreated oily waste. 

Compilation of training data showing the various ways oil slicks appear on radar satellite images.

Bilge dumping isn’t the first environmental issue that people think of when they think of protecting the Earth. In fact, when I started at SkyTruth I had only ever heard of accidental, large scale oil spills, such as the 2010 BP spill in the Gulf of Mexico, and was unaware that smaller, more frequent and intentional acts of pollution occur. Additionally, there is little information about bilge dumping  online. One of the last large-scale reports, published by the National Academies Press, was released in 2003. My quest to know more had to be thorough. I had to read prolifically and search widely in order to piece together the true scale and impact of this issue. 

What did I find? I learned how vast the world’s ocean is (encompassing various oceans, composed of bodies of water such as seas and straits) and how little the ocean is regulated (legal authority depends on nearby countries). Promising international treaties don’t necessarily lead to legislation that allows for enforcement or meaningful measures to prosecute polluters. Vessels’ operators should know that polluting the oceans is wrong, but have little incentive to protect marine waters, especially when penalties are rare. I learned that some vessel operators choose to pollute the ocean — to harm coastal birds, dolphins, and coral reefs, to adversely impact human health, to harm the livelihoods of coastal businesses, and to leave beaches stained and tarred — all just to save money.  

But my research didn’t just uncover bad news. A lot of stakeholders are interested in initiatives supporting more sustainable seas. Not only citizen activists, non-profits, and coastal communities, but investors and technology providers.  Several indexes score vessels on how well they manage waste and emissions, and some international sustainable shipping partnerships have pledged to support and invest in cleaner ships. Additionally, support systems for whistleblowers allow them to share their stories in confidence, so that authorities can punish the operators of vessels that  are polluting at sea with large fines and probations. And groups like SkyTruth are out there fighting for a cleaner world.  You can access my findings in my series of blog posts here

Likely bilge dumping events identified by SkyTruth in 2020

In general, my time at SkyTruth taught me how to use powerful technology to solve complex issues and how to use data to tell stories. I was encouraged to ask as many questions as I answered, to differentiate between what was certain or just an assumption, to be fair in my reporting (using words such as “likely” or “suspected” instead of assuming blame) and to seek evidence-based truths. I was included on esoteric programming projects that I couldn’t quite understand and was pushed to grow from those challenges; I learned faster this way. I was given the autonomy to do my own investigative research, and was provided  a platform to report on — an overwhelming transition from positions I had previously held. The SkyTruth team had confidence in me, and valued my feedback. I will take my experiences from SkyTruth out into my next venture with the same enthusiasm to do work with an important mission. 

When I started writing my  series on global bilge dumping I was inspired by a quote SkyTruth’s Writer-Editor Amy Mathews introduced to me: “Don’t just share your data, share your awe,” which she attributed to former National Public Radio correspondent Christopher Joyce. True fulfillment comes from making a difference and being motivated by what matters most to us. Nonprofit organizations like SkyTruth have the ability to engage in both local and exotic pursuits, to consider personal stories, and tackle the challenges of society for reasons beyond mere profit. They think big — really big — and look into the future; this is awe-inspiring work. They pursue concerns we may not know we have, matters that elude us in our day-to-day lives, but that have true impact. Working for a cause you care about is fulfilling. You never have to doubt the importance of your work. It was humbling to be a changemaker in the weekday hours. 

As I wrap up my ten months at SkyTruth, every day I still feel a profound sense of how small I am. SkyTruth, through the constant engagement with global imagery, made me recognize the interconnectedness of the world and amplified the numerous opportunities to advocate for change. I’ve learned that the more informed you are, the more you can make good decisions about your life and future. I’ve learned to seek a deeper understanding of issues beyond what appears on the surface. And I’ve learned to question everything, observe the environment, appreciate it, and protect it. 

 

Photo: Tatianna at work during COVID-19 quarantine. Photo credit: Tatianna Evanisko

Alice Foster’s Internship Triggered New Excitement About Her Career Possibilities

Before her internship, Alice felt burnt out at school. After applying new skills and technologies to environmental projects at SkyTruth, she’s looking forward to her remaining classes and a fulfilling career.

As I wrap up my four-month internship at SkyTruth, I would like to share some highlights and takeaways from my experience. During my internship I explored the field of geospatial technology for the first time, which allowed me to learn new skills and gave me insight into my career goals. I learned about global environmental issues that I hadn’t known existed. And I got to work with a kind, dedicated, creative group of people. I contributed to SkyTruth’s mountaintop mining research and Project Inambari, which will create an early alert system for tropical forest mining. I also spent time identifying oil and gas well pads, collecting images of oil slicks, and creating annotated maps in QGIS, a geographic information system application that can be used to analyze and visualize geospatial data such as satellite imagery or a ship’s track across the ocean.

On just my first day of orientation at SkyTruth, the high level of support and guidance I received from the staff surprised me. My advisers Brendan Jarrell and Christian Thomas spent lots of time introducing me to concepts and technologies (like Google Earth Engine and QGIS) that I would use in my work. One of the first skills I learned was recognizing oil slicks on satellite imagery — most likely from vessels dumping oily bilge water at sea — and creating an annotated map to reveal the slicks to the public. Brendan patiently guided me through the steps to making a map twice. The team congratulated me when I found my first slick, even though I did not think it merited attention. This encouragement made me feel welcomed and excited about my work. 

The search for oil slicks allowed me to virtually explore oceans and coastlines across the globe. With time, it revealed to me more than how to use geospatial technology, but how little geography I knew. I would toggle past a country or island and wonder what it was like there, realizing I did not even know its name. And so I started exploring a geography trivia website in my free time to teach myself the countries of the world. I am now learning capital cities in Europe, which I tend to forget.

After getting practice with Google Earth Engine — a tool for analyzing and mapping satellite imagery and change around the world —  during my first couple of weeks at SkyTruth, I became involved in some mining-related projects. In one project, I adapted code from SkyTruth’s mountaintop mining research to incorporate satellite imagery from the European Space Agency’s Sentinel-2 satellite. This imagery provides us additional data, which could improve our ability to detect surface mining throughout Central Appalachia. Working with the code in Earth Engine allowed me to better understand SkyTruth’s process for identifying mines. First, we produce a greenest pixel composite image from a collection of images. Making a composite in Earth Engine means combining multiple overlapping images to create a single image. Images can be combined in different ways; in this case, the greenest pixel composite selects pixels with the highest Normalized Difference Vegetation Index (NDVI) values compared with corresponding pixels in the image collection. NDVI is an indicator of plant health in a given area. To provide a more concrete example, suppose we want to make a greenest pixel composite from three images, all showing a part of West Virginia at different times of summer. Say we look at one pixel in one of the images, which covers a small square of forest. We then compare this pixel with the pixels covering the same bit of forest in the other two images, and we choose the greenest of the three (or, the pixel with the highest NDVI value). If we repeat this process for every pixel in the image, we get one image with all the greenest pixels selected from the collection. 

A second script uses the greenest pixel composite to approximate the lowest NDVI value for each county, producing a threshold image. Again, say we have the greenest pixel composite of West Virginia that we just made. Now we look at forested areas within one county and find the pixels that are least green, or have the lowest NDVI values, and then take the average of these NDVI values. This is the threshold for that county; if a pixel is less green than the threshold, it is likely a mine. Our output image contains these values for every county. As a final step, we compare the greenest pixels with the NDVI thresholds to determine likely mine areas. 

Figure 1. Mining data overlying a Sentinel-2 greenest composite image. The image covers counties in West Virginia, Virginia, and Kentucky.

SkyTruth’s surface mining expert, Christian Thomas, also had me experiment with two different techniques for masking clouds in Sentinel-2 imagery. Clouds obstruct necessary data in images, so clearing them out improves analyses. The standard approach uses a built-in “cloud mask” band. The other approach is an adapted “FMasking” method. This takes advantage of the  arrangement of sensors on Sentinel-2 satellites, which creates a displacement effect in the imagery that is more pronounced for objects at altitude. The FMask uses this effect to distinguish low altitude clouds from human-made infrastructure on land. Though the two methods had similar results, the FMask seemed slightly more accurate.

Working on technical projects like this, I learned how much I enjoy using imagery and geospatial data. I had found analyzing data interesting in the past, but something about being able to visualize the information on a map was even more appealing. I loved how a satellite image could be reduced to numbers and assessed quantitatively, or understood visually, almost as a piece of art. 

In another project, I had the opportunity to develop my writing skills by contributing to an  application for the Artisanal Mining Grand Challenge, a global competition to provide solutions for small-scale, low-tech, and/or informal mining. Researching artisanal gold mining was illuminating, as I knew almost nothing about the subject beforehand. I learned that illegal gold mining in Venezuela and Peru has often involved brutal violence and exploitation. In recent decades, labor and sex trafficking have plagued remote mining regions like Madre de Dios. Small-scale mining practices are also particularly damaging to the biodiverse Amazon ecosystem. To extract a small amount of gold, miners must dig up massive amounts of sediment, denuding the landscape in the process. The use of mercury in artisanal gold mining is incredibly detrimental to water quality and human health.

I was also able to be involved in the technical side of this project, building a tool to detect mines in the Peruvian Amazon. I created a mask that removes water from satellite images so that water areas could not be mistaken for mine areas or vice versa. Mines are often near water or can look like water in imagery. To make the mask, I used the European Commission’s Joint Research Centre global surface water dataset. This dataset contains information about where and when surface water occurred around the world over the past thirty years. In Google Earth Engine, the data is stored in an image with bands representing different measures of surface water. I used the “occurrence,” “seasonality,” and “recurrence” bands to create the mask. “Occurrence” refers to how often water was present at a location; “seasonality” means the number of months during which water was present; and “recurrence” is the frequency with which water returned from one year to the next. I tried to find a combination of band values that would do the best job getting rid of water without masking mines or forest. For example, using an occurrence value of twenty, (that is, masking pixels where water was present twenty percent or more of the time), ended up masking mine areas as well. Christian also suggested using a buffer, which meant that pixels adjacent to a masked pixel also got masked. Since the mask often did not capture all of the pixels in a body of water, the buffer filled in the gaps. Masked pixels dotting a river became a continuous thread. The buffer also helped eliminate river banks, which look similar to mines. We applied the finished water mask to the area of interest in Madre de Dios, Peru.

Figure 2: Water mask in the Madre de Dios region of Peru. White pixels have value 1, while black pixels (water) have value 0. When the mask is applied to a satellite image, all pixels in the black areas appear transparent and are not included in analyses. When identifying potential mines in the image, the masked areas are ignored.

Researching issues related to artisanal gold mining left me unsure of how countermeasures can fully promote the welfare of mine workers and others involved in the long term. The problem of illegal gold mining seems entrenched in broader economic and social issues and therefore cannot be addressed simply by identifying and eradicating mines. Nevertheless, understanding the great damage that this type of mining can do to humans and their environment made clear to me the importance of the project. 

Not only did working at SkyTruth teach me a variety of technical and professional skills, it also helped reveal to me what I want to learn about and pursue in the future. In school last fall, I felt burnt out to the point that I just wanted to get through my remaining semesters and be done. Now I feel the excitement about academics I had as a freshman, motivated and informed by my experience at SkyTruth. With my interest in geology and climate issues renewed, I feel like there is barely enough time left to take all the classes I want to. I hope to improve on skills like writing and computer programming so that I can contribute my best work in the future. Being part of an amazing team has motivated me in that way. I also know that I would like to use the geospatial technologies and approaches I learned at SkyTruth moving forward. I feel excited about future career possibilities; before my internship, I felt confused.

I want to give a huge thank you to Bruce and Carolyn Thomas for hosting me in Shepherdstown. I want to thank Christian for introducing me to SkyTruth and for including me in his Dungeons and Dragons game! And I want to thank everyone on the SkyTruth team for their guidance and for being wonderful.

Figure 3: Team Hike, Harpers Ferry, West Virginia. Photo by Amy Emert.

Right-sizing Our Data Pipeline to Detect Polluters

How does SkyTruth’s new project Cerulean reduce the time and cost of processing enormous volumes of satellite information?

Project Cerulean is SkyTruth’s systematic endeavor to curb the widespread practice of bilge dumping, in which moving vessels empty their oily wastewater directly into the ocean. (We recently highlighted the scope and scale of the problem in this blog series) Our goal is to stop oil pollution at sea, and this particular project aims to do that by automating what has historically been a laborious manual process for our team. SkyTruthers have spent days scrolling through satellite radar imagery looking to identify the telltale black streaks of an oil slick stretching for dozens of kilometers on the sea’s surface. We are finally able to do this automatically thanks to recent developments in the field of machine learning (ML). Machine learning “teaches” computers to identify certain traits, such as bilge slicks. To do so efficiently, however, requires a lot of computation that costs both time and money. In this article, we explore a few tricks that allow us to reduce that load without sacrificing accuracy.

The sheer volume of data being collected by the thousands of satellites currently in orbit can be overwhelming, and over time, those numbers will only continue to increase. However, SkyTruth’s bilge dump detection relies primarily on one particular pair of satellites called Sentinel-1, whose data are made available to the public by the European Space Agency’s Copernicus program. Sentinel-1 satellites beam radar energy down at the surface and gather the signal that bounces back up, compiling that information into birds-eye images of Earth. To get a sense of how much data is being collected, Figure 1 shows a composite of all the footprints of these images from a single day of collecting. If you spent 60 seconds looking at each image, it would take you 21 hours to comb through them all. Fortunately, the repetitive nature of the task makes it ripe for automation via machine learning.

Figure 1. One day’s worth of radar imagery collected by Sentinel-1 satellites. Each blue quadrilateral represents the location of a single image. You can see the diagonal swath that the satellites cut across the equator. (Note the scenes near the poles are distorted on this map because it uses the Mercator projection.) Image compiled by Jona Raphael, SkyTruth.

Figure 2 illustrates a typical satellite radar image: just one of those blue polygons above. These images are so big — this one captures 50,000 square kilometers (more than 19,000 square miles) — that it can be tough to spot the thin oil slicks that appear as a slightly blacker patch on the dark sea surface. In this image, if you look closely, you can see a slick just south of the bright white landmass. (It’s a bit clearer in the zoomed in detail that follows.)

Figure 2. Satellite imagery, from January 1, 2019, capturing a difficult to see bilge dump just south of the tip of Papua New Guinea (Copernicus Sentinel data 2019).

Contrary to intuition, you are not seeing any clouds in this picture. Radar consists of electromagnetic radiation with very long wavelengths, so it travels from satellites through the atmosphere largely undisturbed until it hits a surface on the Earth. Once it hits, it is either absorbed by the surface or reflected back into space (much like bats using echolocation). The lightest section of the image in the top right corner is part of a mountainous island. This jagged terrain scatters the radar diffusely in all directions and reflects much of it back to the satellite. The rest of the image is much darker, and shows us the ocean where less of the radar energy bounces back to the satellite receiver. The muddled, medium-gray area along the bottom of the image shows us where strong gusty winds blowing across the ocean surface have made the water choppy and less mirror-like. Figure 3 shows us more clearly the oil slick just offshore. 

 

Figure 3: Detail from Sentinel-1 radar image shown above. The narrow oil slick identified by a dark gray streak along the bottom of this image is roughly 60 kilometers (40 miles) long, and only 15 kilometers (roughly nine miles) offshore. (Contains modified Copernicus Sentinel data 2019).

Although each image covers a large area, we need to process many images each day to monitor the entire Earth. How many? Here’s an approximation:

  • 510,000,000 square kilometers  = The total surface of the earth
  • 90,000,000 square kilometers  = Total area of images captured in one day (1,300 scenes)

This means that we expect the whole Earth to be imaged somewhere between every six to 12 days (because many of the images overlap each other), or roughly 10,000 images. 

Every year more satellite constellations are being launched, so if a new constellation were to capture the whole earth’s surface in a single day, then we would need to spend an order of magnitude more processing time to ingest it. We care about this because each image scanned will cost time, money, and computational power. To enable appropriate allocation of resources for automation, it’s critical to understand the scale of the data. For now, we can size our processing pipeline to the current number, but we must take measures to ensure the system is scalable for the increasing numbers of satellite images we anticipate in coming years.

So does that mean we need to look at 1,300 images every day? Thankfully not. We have a few techniques that we’ll use to make the computations manageable:

  • First off, we’ve found that radar satellite images near the poles are typically captured using a particular configuration called HH polarization — great for mapping sea ice, but not ideal for detecting oil slicks. And the presence of the sea ice itself makes oil slick detection difficult. If we remove those images from the 1,300, we have about 880 that suit our needs (using VV polarization).
  • Next, we won’t be looking for oil slicks on land, so we can further filter out any images that don’t have at least some ocean in them. That reduces our set of images to about 500. (Note: Sentinel-1 coverage of the open ocean is generally poor, as discussed in our previous blog post, but we anticipate future radar constellations will fill that gap.)
  • Those 500 images represent roughly 25,000,000 square kilometers of area, but we can reduce that even further by finding images on the shoreline, and eliminating any pixels in those images we know that belong to land. That drops the total area by another quarter to 15,000,000 square kilometers.

So at this point, we’ve reduced our load to about 17% of the data that our satellite source is actively sending each day. Figure 4 illustrates the effect that these filters have on our data load:

Figure 4. [Gallery] Filtering images. a) All Sentinel-1 images in one day. b) Filtered by polarization. c) Filtered by intersection with the ocean. Images compiled by Jona Raphael, SkyTruth.

Can we do better? Let’s hope so — remember that each of those 500 images is almost a gigabyte of data. That’s like filling up the memory of a brand new Apple laptop every day: an expensive proposition. Here are some ways that we make it easier to process that much information:

  • First, we don’t need to store all that data at SkyTruth. Just like you don’t need to store all of Wikipedia just to read one article, it is more efficient to load an article one at a time, read it, and then close the window to release it from memory before opening another one. That means if we’re clever, we’ll never have to load all 500 images at once.
  • Each image is originally created as 32-bit, but we can easily convert it to 16-bit, thereby halving the data size. You can think of the bit-depth as the number of digits after the decimal place: Instead of storing the value ‘1.2345678’, we would store ‘1.235’, which is almost the same value, but takes a lot less effort to store in active memory.
  • We can further reduce the amount of required memory by reducing the resolution of the image. We find it works just fine to reduce each dimension (height and width) by a factor of eight each, which means the number of pixels actually decreases by a factor of 64. This has the effect of averaging 64 small pixels into one large one.
  • Finally, we don’t need to process the whole image at once. Just like it was easier for you to spot the oil slick when you were looking at a smaller zoomed in portion of the satellite image above, we can divide up each picture into 32 smaller square ‘chips’.

Taken together, we can now work on single chips that are only 250 kilobytes  — instead of 1 gigabyte images — effectively a 99.975% reduction in memory load. That translates directly to speeding up the two core parts of machine learning: training the model and making predictions. Because training an ML model requires performing complex mathematics on thousands of examples, this speed up represents the difference between training for 10 minutes versus 20 hours.

But, wait! What about the step in which we threw away data by averaging 64 pixels into one large pixel? Is that the same as losing 63 pieces of information and saving only one? It is important to address this idea in the context of machine learning and what a machine is capable of learning. It hinges on the difference between information and data. Generally speaking, we want resolution to be as low as possible, while retaining the critical information in the image. That’s because the lower the resolution, the faster our model can be trained and make predictions. To figure out how much resolution is necessary, a convenient rule of thumb is to ask whether a human expert could still accurately make a proper assessment at the lower resolution. Let’s take a quick look at an example. Each of the following images in Figure 5 has one-fourth as many pixels as the previous:

Figure 5: [Gallery]  Reducing resolution by factor of 4: 512×512, 256×256, 128×128, 64×64, 32×32. (Contains modified Copernicus Sentinel data 2019).

If your objective is to find all the pixels that represent oil, you could easily do so in the first two images (zooming in is allowed). The next two are a bit harder, and the final one is impossible. Note that the fourth image is a factor of 64, or 98.5%, smaller than the first, but the oil is still visible. If we think critically about it, we realize that the number of pixels in a picture is irrelevant — what actually matters is that the pixels must be much smaller than the object you are identifying. In the first image, the oil slick is many pixels wide and so it has sufficient information for an ML model to learn. However, in the last image the pixels are so large that the tiny slick all but disappears in the averaging. If, instead, your objective is to find all the pixels that represent land, then the final image (32×32) is more than sufficient and could probably be reduced even further.

However, reducing resolution also comes with downsides. For instance, the reduction in visual information means there are fewer context clues, and therefore less certainty, about which pixels are oil and which are not. This results in more pixels being labeled with the wrong prediction, and a corresponding reduction in accuracy. Furthermore, even if it were possible to perfectly label all of the pixels that are oil, the pixels are so big that it’s difficult to get a good sense for the exact shape, outline, or path of the oil itself, which results in lower quality estimates derived from this work (for instance, estimating the volume of oil spilled).

If only we could use the resolution of the first image somehow… 

Good news — we can! It turns out that the first image above, the 512×512, already had its resolution reduced from the original satellite image by a factor of 64. To give you a sense, Figure 6 shows the original resolution satellite image side by side with the version we use in training for our ML model. The second image has lower resolution, but because the pixels are still much smaller than the features we care about, we are able to avoid most of the downsides described above. The oil spill is still unambiguously identifiable, so we are willing to trade the resolution for gains in training and prediction. This is a subjective tradeoff that each machine learning project must make based on the resources available

Figure 6:  Original satellite image at full resolution (left), compared to reduced resolution used for ML prediction (right). (Contains modified Copernicus Sentinel data 2019).

So what are the takeaways from this exploration? We first learned that it takes a lot of imagery to monitor the surface of the Earth. However, by thoughtfully excluding imagery that doesn’t capture the information we care about, it’s possible to reduce the volume of data we need to process significantly. Then, we discussed a few different tricks that make it much faster to train and predict on that pared down dataset. In particular, the most important technique is to reduce the resolution as much as possible, while maintaining a pixel size that is substantially smaller than any features we are attempting to identify. All told, there is a lot of data that still needs to be processed, but if we can achieve a total process-time for each satellite image that is under three minutes, then the whole pipeline can be run on a single computer. Right now, we are down to about 10 minutes per image, and still have a few tricks up our sleeve. So we’re hopeful that in the coming months we’ll hit our target and have a robust and scalable mechanism for identifying oil slicks automatically. 

Until then, we continue to tip our hats to the SkyTruthers that regularly identify and expose these egregious acts, and we look forward to the day that the illegal practice of bilge dumping comes to an end.

Bilge Dumping at Sea: What are SkyTruth and others doing about it?

With free public monitoring services and the help of watchdogs globally, bilge dumping can come to an end. SkyTruth’s new project Cerulean is one solution.

This is the last entry in a multi-part series revealing the significance of bilge dumping globally. You can read parts one, two, and three on SkyTruth’s blog.

Throughout this multi-part series on global bilge dumping, we have revealed how SkyTruth has identified oily pollution from ocean-going vessels, the range of harmful effects to coastal communities and marine ecosystems, and the challenges in prosecuting ocean offenders. We now address the future. What’s next? 

Recall that bilge dumping occurs when vessels bypass treating their oily wastewater and discharge it directly into the ocean. When we started this series, we reported on SkyTruth’s 2019 ocean monitoring efforts in which we recorded 163 oily slicks around the world. So far this year we have detected 98 new slicks. Now that we better understand what is occurring at sea, we are working hard to share that information with citizen watchdogs and others, enabling them to take action such as alerting authorities and publicly exposing wrongdoing in their coastal waters. With sustained pressure, we hope polluters will be forced to act responsibly.

Using radar satellite imagery, SkyTruth can observe and measure oil pollution in the ocean. Making a conservative assumption about an oil slick’s thickness, we can estimate the volume of oil, an indicator of the severity of an incident. The legal limit for discharging oily wastewater under international law outlined by the International Convention for the Prevention of Pollution from Ships (MARPOL) is 15 ppm for most cargo vessels and tankers. Trained observers on aircraft can see oil at concentrations of 50 parts per million (ppm) and above. It’s possible that under favorable sea-state conditions, radar satellite imagery is capable of revealing oil at lower concentrations, so we’ll be able to spot more potential violations. In a resolution to MARPOL specifying visibility limits of oil discharges, the Marine Environment Protection Committee “urgently requests authorities to consider the principle that a visible trace of oil is one element of proof that the 15 ppm discharge standard of Annex I of the MARPOL Convention may have been violated.” Similarly, under the U.S. Clean Water Act, oil discharges that cause a visible sheen are encouraged to be reported and are classified as potentially harmful to public health or welfare. 

Using advanced remote sensing techniques, we often can determine a likely source of pollution, whether from a vessel, a natural oil seep, or energy infrastructure like pipelines, drill rigs and production platforms. This kind of information can help oil-spill response efforts: If they know the location and source of oil pollution, responders are better able to take action and reduce the impacts of the pollution. In addition, if the pollution is from a vessel, near-real-time monitoring can help authorities catch a perpetrator who might still be nearby, or take administrative action when that vessel comes into port. Prosecuting criminal offenses for environmental crime requires robust evidence connecting a culprit to the crime. Satellite imagery can help: It collects evidence remotely, potentially providing an efficient way to prosecute ocean offenders.

To demonstrate this potential, in late 2019 SkyTruth launched project Cerulean to automate the detection of oil slicks on the world’s oceans and identify likely sources. Cerulean applies a machine learning technique called Deep Learning (a type of Artificial Intelligence) to make it possible to analyze thousands of satellite images every day to detect slicks and sources. We compare these slicks with public Automatic Identification System (AIS) vessel tracking data in order to identify the likely sources of this pollution (you can read more about our process here).

Image 1. Sentinel-1 radar satellite image of suspected bilge slick (black streak) in the Java Sea on February 15, 2020. AIS signals are shown for two potential sources. The likely source vessel is circled in red, with its AIS track (red) closely matching the slick in space and time. The other nearby vessel’s broadcasts (white) did not match the time and location of the slick.

Satellite data such as that provided by Cerulean could potentially help prosecutors convict oil polluters if it is approved for use in court. Remote sensing evidence is a relatively new tool in court cases. It must be authenticated and checked for integrity regarding the collection and storage of the data to ensure its validity. In the United States, cases using satellite evidence have not been widely published since many decisions about whether evidence is admissible can be made orally or pretrial. Because of this, we do not have a good estimate of how frequently satellite imagery is accepted as evidence. However, we do know that if the remote sensing technology is not new or novel it is more easily admissible. When trying to prove a fact, an expert witness with specialized knowledge can explain the reliability of the technology used and the data presented. In describing the detection of an oily slick, for example, an expert might need to explain a variety of factors that are considered in the analysis of ocean imagery, including weather conditions such as rain and wind speed, atmospheric and oceanographic features, and differentiating a vessel from ocean infrastructure in order to demonstrate the admissibility of image-based evidence

Satellite imagery has been used as evidence of other environmental crimes such as deforestation and illegal fishing. However, it is uncommon for imagery alone to provide conclusive evidence, or to prove guilt beyond a reasonable doubt in prosecution for environmental crime. Satellite imagery has provided corroborating evidence that has led to prosecutions of pollution at sea. However, we have found no cases where it provided conclusive (dispositive) evidence, meaning it alone is usually not enough to link a crime to a perpetrator, or even prove that a crime occurred at all. Currently, eyewitness reporting is more often used as evidence for prosecution of bilge dumping. Whistleblowers and vessel inspectors have been a common catalyst for prosecution. For example, between 1993 and 2017, 76% of prosecuted cases under the U.S. Act to Prevent Pollution from Ships came from whistleblowers. Overall, there are no restrictions on satellite imagery’s power to serve as evidence in court as long as the technology has been proven valid and trustworthy.

Europe currently has an ocean monitoring service called the CleanSeaNet that has led to several prosecutions using satellite imagery in combination with on-ground evidence, in which fines were imposed. The European Union Directive 2009/123/EC specifies types of prosecution for ship-source pollution. However many courts still rarely prosecute. Often they require official inspection on site where the pollution occurred and on the vessel in order to connect the evidence to the crime, which can be challenging when collecting on-ground evidence is time sensitive. (Oil can disperse within 12 hours, and ships can easily be long gone by then.) Additionally, the European Maritime Safety Agency (EMSA) states “(r)esolution of cases can take many years, and in some countries privacy issues prevent the full details being disclosed publicly.” 

Image 2. Suspected bilge slick in the Gulf of Guinea on February 23, 2020. The red AIS signals closely follow the linear slick to the likely source vessel (circled in red). Another nearby vessel’s broadcasts (white AIS signals) were also analyzed; however, this vessel was moving nearly perpendicular to the slick and is not a likely source.

But law enforcement agencies and citizen-activists aren’t the only stakeholders concerned about clean oceans. Companies have market-based incentives to be recognized as sustainable and socially responsible businesses, and they are liable to their stakeholders to uphold promises of corporate social responsibility. While the shipping industry produces ocean oil pollution, it is also the most energy efficient method for transporting goods. Such a large and fast-growing industry requires effective environmental regulation and enforcement. 

The International Maritime Organization’s (IMO) theme for 2020 provides a unique opportunity for the shipping industry to clean up its act. The theme is “Sustainable shipping for a sustainable planet.”  For example, part of this includes the IMO’s 2020 sulphur cap on ships’ fuel oil. In transitioning to lower sulphur fuel, some shipping companies must adjust their current operations and infrastructure, which requires financing. By choosing cleaner ships that meet current and future environmental standards, investors, banks and other stakeholders consider Environmental, Social, and Governance (ESG) criteria. ESG concerns have led to efforts such as the Sustainable Shipping Initiative (SSI), an international partnership that promotes sustainability between its members, from ship owners to insurers to NGOs. SSI’s “Save As You Sail” program partners ship owners with financial providers in order to update vessels with more efficient emission-reducing technology and save on fuel costs. 

Additionally, vessel companies can voluntarily use grading initiatives such as the Swedish Clean Shipping Index (CSI) to hold themselves accountable. CSI scores a vessel company and its operations from its supply chain to emissions to operational discharges, allowing investors, clean tech providers, and other stakeholders, such as freight forwarders, to partner with responsible vessels who can be trusted to comply with international standards. A similar program exists in North America called Green Marine. The Environmental Ship Index (ESI) is an international scoring option that provides vessel-specific scoring and offers incentives such as discounts on port dues and bonuses to clean ships. Its index currently evaluates over 7,000 ships globally. To help support responsible ships and shipping practices, investment frameworks such as The Poseidon Principles provide banks around the world with principles that align ship financing with climate interests. Several large European banks such as the European Investment Bank (EIB) are committed to providing green investments through Blue Economy Finance Principles. These principles were created by the EIB to uphold the United Nations Environment Programme’s Sustainable Development Goal “Life below water.” In 2018, two banks, EIB and ING, agreed to provide 300 million euros of financing to support green shipping in Europe. These initiatives provide evidence that sustainability and a blue economy aren’t mutually exclusive. 

Cerulean aims to support international clean sea initiatives as well as stakeholder interests by providing a platform for stakeholders to directly monitor oily discharges from vessels. Companies and coalitions like those listed above can use Cerulean to help ensure a greener supply chain. Scoring indexes can use Cerulean to make sure vessels hold true to their ratings, allowing investors and insurers to be more confident in their partnerships. Widespread transparency of activity in the oceans encourages vessel operators, owners, and crew to operate responsibly in order to continue to receive support from their clients and other stakeholders, as well as to avoid criminal penalties. 

Image 3. These bright lines, compiled from dozens of radar satellite images, reveal heavy vessel traffic through the Strait of Malacca, one of the world’s busiest shipping lanes, over the course of several months in 2019.

Our conversation with a resort director in Southeast Asia suggests Cerulean will support citizen activists as well. This resort director (first introduced in the second blog post in this series) reached out to SkyTruth to share some of his experiences with bilge dumping. He has regularly found oil and tar washing ashore on beaches near Singapore and the Strait of Malacca for several years. He believes that part of the reason ship operators act irresponsibly is that they don’t really understand that they can be monitored, and don’t realize that their AIS broadcasts can be collected by the public. As an environmental activist whose livelihood depends on a clean coast, he would like to make vessel operators more aware of the transparency of their behavior and is interested in doing more personal monitoring of the sea surrounding his property. He believes Cerulean can help him, stating  “(i)t is super compelling to pair satellite data with AIS.” Knowledge of potential oil slicks nearby, even at night, allows citizens to prepare for when oil hits the land and can alleviate some of the clean-up. “It would be useful to know if a slick is coming,” he said. “We could get people organized to stop it. If we had a warning — even 30 minutes, it would give us enough time to make a difference.”

While the first version of Cerulean won’t provide continuous monitoring of the ocean (because the satellites now in orbit can only cover a portion of the earth every day), it would allow insight into ocean pollution every several days and will build a one-of-a-kind record of incidents over time. One key objective of Cerulean is to establish a scalable pipeline that can process more and more images as new satellite constellations are launched and cover our oceans more continuously. 

Cerulean is currently under development, and we hope to have an operational product by 2021. Once up and running, Cerulean will be free and open to the public, providing a user-friendly interface that allows anyone, without any technical expertise, to monitor their areas of interest for oily slicks. Users can receive near real-time alerts for oil detected in their area of interest along with insights into who might be responsible for the pollution. Cerulean will identify features such as a slick’s location, date, time, and length, in addition to specific vessel characteristics such as a vessel’s ID number, destination, owner, operator, and flag country. We hope Cerulean will provide more insight into hotspots — where bilge dumping is particularly common — as well as track repeat offenders to enable more robust prosecutions. We anticipate Cerulean will be a useful tool for fleet owners, shipping companies, and international merchandise retailers that choose to be environmentally responsible: they’ll be able to use Cerulean to track the performance of their fleet and incentivize operators and crew to stay in compliance with ocean law. Additionally, law enforcement can use these near real-time alerts to catch perpetrators in port and in coastal waters. Finally, we hope it will be a tool that empowers citizen watchdogs, nongovernmental organizations (NGOs) and journalists  to keep an eye on what’s happening at sea, report wrongdoings, and rachet up pressure on governments and businesses to do a better job protecting our ocean. 

In the meantime, SkyTruth analysts will continue to monitor hotspots in the ocean and share our results on this blog. By continuing to report on pollution across the world, innovate user-friendly platforms, and spread awareness, we are working to stop oil pollution at sea and protect the world’s marine ecosystems and the coastal communities that depend on them.