FREM Article Series: Open Data

The FfE Open Data Portal (opendata.ffe.de) was launched as part of the eXtremOS and DemandRegio research projects. It now includes a wide variety of data sets from several FfE projects. Examples of the availble data include e.g., energy production and demand or statistical data on population and housing situation, as well as project results. The FfE Open Data Portal makes the generated data available in a human- and machine-readable form. By sharing free and licensed data, the FfE is making an important contribution to both the further development of energy system analysis and to public awareness. The publication of data forms the basis for good scientific work; uniform interfaces enable efficient data handling. “Open data” often implies the absence of any restrictions or specifications. “Real” open data, however, must meet some criteria to be designated as such. You will learn what these are and how the FfE implements open data in the last article of the series of articles concerning FREM.

Overview of the topics in the FREM Article Series

The Added Value of Open Data

In 2016, the Konrad Adenauer Foundation examined open data more closely, and compiled various general benefits of open data. In addition to more obvious aspects such as “building trust through transparency,” the survey also included statements such as “economic benefits,” “strengthening democracy,” and “better decision-making processes”. (FfE-translation of the original German quote)

Traditional energy systems with base-load and flexible power plants on one side, large and small consumers on the other, and interconnection via a manageable network infrastructure, are easier to depict in the data world. Modern energy systems with volatile, distributed generators, prosumers, flexibilities, and increasingly smart systems are much more complex and require a correspondingly more extensive mapping in the data.

Open Data can thus provide a potent basis in energy system analysis for the efficient mapping of increasing complexity.

“Open” Does not Mean License-free!

Essential for disseminating and processing open data is an appropriate license that clarifies to the users how they are allowed to work with the data. This is supported by the non-profit organization Creative Commons, which offers standardized license agreements in many languages. The Creative Commons licenses are known worldwide and form the basis for a common understanding of the rights to use open data. The “CC 0” (share, edit, commercial use, and no attribution required) and “CC BY” (like CC 0, but with attribution) licenses are the standards for open data – but it should be checked in each case which license is the most suitable.

Figure 1: The CC licenses, arranged by openness. Source: CC-BY 4.0.

Standardized Metadata Makes Practical Use Possible

Metadata describes the data itself. It tells users what kind of data they are dealing with and how it can be processed. This includes, for example, information about units, temporal and spatial resolution, or sources, as well as general descriptions of the data. As with licenses, standardized forms exist for specifying metadata. The FfE uses the standardized metadata of the Open Energy Plattform and extends it with FfE-specific attributes. Using the same metadata formats facilitates working with open data obtained from different sources.

Figure 2: Example of a FfE metadata record based on the OEP metadata.

Criteria for open data

When open data is made available, the FAIR principle provides guidance on whether all criteria relevant to open data are met. FAIR stands for “findable, accessible, interoperable & reusable”.


Findable	Data and metadata are human and machine-readable
Accessible	Data is available for the long term Standard protocols are used (e.g., REST Api)
Interoperable	Data are linkable to other datasets (for example, through (same) management keys and ontologies References to dependent data sets are linked
Reusable	Data sets are sufficiently described Data are citable and licensable

FfE Open Data

Under the URL opendata.ffe.de, the FfE offers a user interface for open data created as part of FfE projects. Data sets can be found quickly by searching and filtering. In addition to descriptions and metadata of the individual datasets, some articles go into detail about methodologies and backgrounds to the data provided and provide visualizations of the data.

To visualize the data, diagrams are generated directly from the FREM database via the integration of the Python framework Plotly. By retrieving geodata in GeoJSON format, also from the database via the Open Data interface, these can be displayed as maps by JavaScript libraries such as OpenLayers, Leaflet, or D3.

Figure 4: OpenLayers integration with interactive map display.

The open data of the FfE are provided via a postgREST interface in JSON format directly from the FREM database. The data can be queried and filtered via a URL. Examples for using the data with Python or Matlab can be viewed here: opendata.ffe.de/how-to.