DETECT: The FfE Software for Automatic Object Detection

What is DETECT used for?

DETECT is our software solution designed for large-scale object detection in imaging data, including remote sensing, aerial images, and drone images.

In recent years, an increasing amount of high-resolution aerial imagery (captured from planes or satellites) has become publicly available in Germany and across Europe. The higher resolution enables AI-based approaches to automatically detect structures such as vehicles, buildings, and even smaller objects like solar installations and trees.

The data obtained covers a wide range of use cases, such as:

• Network expansion planning by distribution and transmission system operators
• Municipal energy planning by helping municipalities and public utilities to better understand their area and plan for the energy transition.
• Identifying prosumer behavior for market analysis, sales, and data-based strategies
• Research and scenario calculations

As an example of data extraction using DETECT, we have applied our software to publicly available orthoimages to detect objects for several use cases (detailed in the last section):

• Detection of existing solar panel installations
• Detection of trucks to assess logistics center capacity

Software Components

As illustrated in the visualization below, the DETECT Software comprises three major steps: Data Scraping, Object Detection, and Analysis Report Generation.

 

 

Data Scraping

Object Detection is performed on remote sensing data such as satellite imagery or aerial photography. So, the first step before conducting actual detection tasks is to acquire appropriate images. We obtain such data through so called Web Map Services (WMS) which can be used to request images for a specific region of interest. Oftentimes there are multiple data providers each offering imagery for a different region (e.g. German States) which can make analysis difficult if multiple data providers are needed. The FfE has already open sourced a part of the DETECT Software data acquisition step to seamlessly access data from multiple WMS services. It offers fast, parallel, automated image download for the region of interest and such facilitates automations and overall improves efficiency and performance. It also includes a masking feature which is used to only process relevant parts within the area of interest. The tool will intersect the mask with the region of interest and only download images which intersect with the mask. For example, when detecting rooftop PV systems, a building layer can be used as a mask so only images containing buildings are processed.

Detection

The Detection Module itself as the core of the DETECT Software applies AI-based Computer Vision models to the images obtained by the Data Scraping Module in order to detect the respective object instances on them.

These models are typically trained on image datasets that have been manually annotated beforehand with masks and/or bounding boxes indicating the position of the object instances present in the image (Figure 2). This way, the models can learn to detect the objects in images they have never seen before.

The AI models used for detection are different variants of convolutional neural networks (CNNs), with the exact architecture and implementation depending on the respective use case. The Detection Module allows to efficiently determine the best-suited model architecture for a new use case and, if necessary, to add a new architecture to the model zoo.

Integration of Open-Source-Ressources

AI-based Computer Vision is currently a very active field of research, with plenty of contributions from the open source community being available for public use. These range from Machine Learning frameworks such as PyTorch or TensorFlow over implementations of neural network architectures, open source annotated datasets up to fully trained neural network weights that have been optimized to either solve some specific task or to extract universally relevant features from images.

As training on large and diverse datasets is crucial for obtaining good detection results, we typically use a combination of various openly available datasets to train our models on. This provides the model with a good “basic knowledge” about the use case’s task (e.g. detecting PV modules on aerial images).
As a second step, we handcraft our own, use-case-specific dataset to make the model adapt to specific properties of the data source (e.g. open data aerial imagery provided by the German Länder) and specific requirements of the detection results (e.g. differentiating between conventional PV modules and full-black modules that entered the market during the last years). We then continue to train the pretrained model on our own datasets to achieve good generalization power as well as accurate results on the data provided by the Data Scraping Module.

Reporting

The raw outputs of our Detection Module are geo-locations and shapes of the detected objects. The Reporting Module provides various ways to aggregate and visualize this raw data to support decision processes and gain insights. Some of those are visualized in the following chapter on our various use cases.

Other possible final outputs include structured data in different formats (e.g. CSV, (Geo)JSON, Excel, …) as well as derived data that includes inputs from the FfE’s broad simulation model landscape and energy economics knowledge of FfE experts from various topic areas.

The Reporting Module is highly customizable and can be adapted to provide novel output formats according to the customer’s needs.

Some Applications

Solar Panel Detection

One important use case of DETECT is the identification of solar panels in aerial imagery. For this purpose, we trained a model using both existing datasets from various EU regions and self-annotated data for Germany. The model was then integrated into the DETECT software, enabling us to identify existing PV installations in any given area.

The resulting building-level data is relevant for a wide range of applications, such as the creation of so-called Digital Twins or the calculation of building-specific PV generation profiles in the context of grid planning. Additionally, we use this data as a foundation for energy consulting projects and integrate it into other FfE products and tools.

For example, within our FfE solutions, the potential for additional PV expansion is calculated on the single-building-level by integrating 3D building data. The PV modules already detected by DETECT are combined with precise roof geometries (including gables, windows, roof surface orientation, etc.) in a dataset. This dataset contains building-specific information on the unused potential for solar power generation on rooftops, as well as details on orientation, tilt angle, and potential generation load profiles of additional PV modules.

Further information about the dataset and ways to contact us can be found in our solution article.

Truck Detection

Another use case of the DETECT solution is the detection of trucks in industrial areas. The information on how many trucks are parked in a commercial area can help distinguish logistics centers from other industrial sites and estimate their scale. This is crucial for analyzing the impact of heavy-duty transport electrification on distribution grids. For instance, it allows an estimation of how many charging points would need to be newly installed in the event of full electrification of a site’s fleet, and what grid connection capacity would be required at each location.

For classification, industrial areas are first extracted using OpenStreetMap data and used as masks in DETECT so that only these areas are analyzed. After training a model to detect trucks and large vehicles, the model was loaded into the DETECT software to identify trucks in relevant areas, thereby providing a decision-making basis for classifying logistics centers.

For more information about the data set and how to contact us, please refer to our solution article.