Data Analytics for HPC

EOPEN—A Big Data Platform for Earth Observation Data

Earth observation (EO) is all about collecting information about the Earth by remote sensing. Specifically, a series of satellites are collecting scientific data such as images and geo-positioning information to survey the Earth. The vast amounts of data collected contribute to a number of public policies and economic sectors including environmental protection, precision farming, or the detection of oil spills, to name but a few. In Europe, Copernicus, a joint service between the European Comission and the European Space Agency, operates a series of satellites and missions. For example, the so-called Sentinel-3 mission is about measuring the sea surface topography in order to improve forecasting systems and climate monitoring. Although the data can be accessed openly via Internet, the data sets are spread across multiple locations in Europe. Although potential areas of application for Copernicus data are huge, these areas are not yet fully exploited, because processing of large data collections is challenging. Tremendous amounts of Earth observation data are becoming available on a daily basis through the Copernicus services. 55TBs of data are available through the Copernicus climate change service alone, and 1.4TBs of data are produced every day through the atmosphere monitoring service.

EOPEN aims to tackle the technical barriers arising from the massive streams of EO data to ensure scalability of the data harmonisation, standardisation, fusion and exchange methods, combining also non-EO data and metadata annotation. The overall objective of EOPEN is to provide a platform targeting non-expert EO data users (i.e., non-traditional user communities), experts, and the SME community that reveals and makes Copernicus data and services easy-to- use for Big Data applications by providing EO data analytics services, decision making, and infrastructure to support the Big Data processing life-cycle allowing the chaining of value adding activities across multiple platforms.

The platform will be evaluated based on the three distinct, and challenging case studies: flood risk monitoring, climate change monitoring, and food security.

HLRS provides its HPC and Big Data infrastructures, as well as contributing to ambitious data analytics tasks such as clustering of EO and non-EO data.

Project Details


01.11.2017 - 31.10.2020