When Remote Sensing Meets Artificial Intelligence & GIS

The convergence of artificial intelligence, remote sensing and location present massive opportunities to GIS professionals and together they can be utilised to enable effective and prompt decision making. 

Derived outputs obtained by combining these 3 fields can provide valuable insight, allowing patterns to be discerned where none was previously obvious, permitting those who can interpret this data and exploit this potential. It's "The Science of Where" in action. 
 

The use of remote sensing is important because it contains masses of valuable information if you know how to extract it. This type of imagery is captured on sensors mounted on aircraft, unmanned drones or satellites. Imagery of these types are increasingly used in many areas such as commercial industry, scientific research, emergency and resource management and security and reconnaissance.

This, coupled with the almost ubiquitous availability of drones and the low entry point they provide for obtaining high quality aerial photography, has led to many potential applications and uses that may traditionally have been impossible to achieve for organisations or agencies with smaller budgets. 
 

So, let’s step back and have a think about imagery, in its raw form imagery is just raw unstructured data. Binary files with rows and columns of pixels with numbers assigned that mean something to somebody according to a scale. Imagery contains data – machine learning allows us to extract information from that data. Combining the power of GIS and Machine learning, we can transform this unstructured data to geospatial information products. Products from which you can derive results and actionable information from. Applications such as precision agriculture, change detection, target detection and asset management are just 4 applications that can benefit from consuming these derived products. 
 

Another such application is detecting building footprints. Late last year my colleague Jonathan Sloan and I held a webinar on 'Taking a Geographic Approach to Machine Learning', a part of the Esri Ireland ‘Do One thing Well’ webinar series. In that webinar we utilised machine learning techniques and orthophotograph's to try and detect building footprints. 

The process we used was achieved using ArcGIS Pro 2.4.2 and the Out-of-the-box installations of Anaconda and Jupyter notebooks that come with it.
 

Our aim was to determine how effective an automated approach would be when compared against a known output. The imagery used was a small sample area of 3 Band, 16cm resolution Orthophotography covering a small area in Bangor, Northern Ireland captured in 2016.
 

For the input area we also obtained polygons for existing building footprints from Land and Property Services. This was our baseline dataset that we would use for training and for comparison against our machine learning output;
 

The first step in the image classification workflow was the creation of training samples. Training samples “teach” the deep learning model what the size, shape, and spectral signature of a building footprint may be. The more samples provided to the model, the more accurate the result will be. We used the LPS Building Footprint polygons to train our model and from this created a simple, 8-bit unsigned label dataset or classified raster image as shown below;
 

This is the dataset we can use to train our model, using this as the Classified Raster input to the ‘Export Training Data for Deep Learning’ Geoprocessing Tool along with our orthophotograph as the Input raster, we can set some basic parameters and then export image chips which can be used to train our deep learning model.

The output of the training is an .emd file which is an Esri Model Definition file. This file is the bridge between the GIS and the deep learning framework.
 

Now that we have a training dataset and have created our model, we can run that model against a different set of orthophotography, in our case we used 16cm 3 band imagery of Belfast captured in 2018. We do this by running the ‘Classify Pixels using Deep Learning’ Geoprocessing tool using our Belfast raster and our deep learning model definition created earlier as inputs.

The output of this process is another classified raster, one where the deep learning model has identified clusters of pixels which represent building footprints.
 

Using this classified raster, we can then run the ‘Raster to Polygon’ geoprocessing tool to convert our raster buildings to polygons. 
 

This creates polygons with many vertices and of irregular shape so at this point we may wish to clean these up somewhat by running the ‘Regularize Building Footprints’ geoprocessing tool which will remove those excess vertices and square off our building edges. The output of this tool are the green footprints as shown in the image below;
 

As you can see, the process has been largely successful in identifying buildings using an automated approach. Results could have been improved by further refining our training sets and our deep learning model however, for the purposes of our demo we deemed the results to be a success. 

I hope that this has demonstrated how Image classification tools can enable you to utilize data collected from sensors and automate the processing of this data with the powerful computational capabilities within ArcGIS to identify and quantify features. You can follow the exact steps we took by watching a recording of the webinar here.
 

The simple case study outlined in this post merely scratches the surface of what can be achieved by utilising machine learning and GIS. Imagery and remotely sensed data is a valuable resource for many organisations who have made substantial investment obtaining the data and, who are now deriving substantial value from that data by utilising innovative data analysis techniques. One such example presented at last year’s Esri User Conference, describes how an insurance company is using similar techniques to improve their response in times of disaster, you can watch their presentation here.

The field of machine learning is both broad and deep and is constantly evolving. Using ArcGIS and machine learning allows organisations to derive valuable new content. With ArcGIS being an open, interoperable platform it allows for the integration of complementary methods and techniques that empower ArcGIS users to solve complex, real-world problems in a fundamentally spatial way. By combining powerful built-in Image analysis tools with any machine learning package users can benefit from the spatial validation, geo-enrichment, and visualisation of results. 

Finally, I hope this has sparked an interest in learning how machine learning and AI could benefit you and your organisation. If you wish to know more about the exciting world of machine learning and artificial intelligence, Esri provide a wealth of resources to get you started and you could begin by taking a look here.