Terra_Byte – Top computing power for researching global change

"The collaboration between DLR and the LRZ marks a milestone in the development of future-oriented research within Bavaria, a hub for science! This project illustrates the wealth of resources that the Munich research landscape has to offer in this area," said Bavaria's Minister of Science and Arts/Culture, Bernd Sibler, at the signing of the cooperation agreement between the partner institutions on 27 May 2019 in Garching. "The collaboration between DLR and the LRZ marks a milestone in the development of future studies within Bavaria, an established hub for science research. This project illustrates the wealth of resources that the Munich research landscape has to offer in this area."  With this cooperation, DLR and the LRZ are pooling their vast expertise in the fields of satellite-based Earth observation and supercomputing.

"To understand the processes of global change and their development we must be able to evaluate the data from our satellites as effectively as possible," stressed Hansjörg Dittus, DLR Executive Board Member for Space Research and Technology. "In future, the cooperation between DLR and the LRZ will make it possible to analyse vast quantities of data using the latest methods independently and highly efficiently, to aid in our understanding of global change and its consequences. Examples of this are increasing urbanisation, the expansion of agricultural land use across the globe at the expense of natural ecosystems and the rapid changes occurring in the Earth's polar regions and in the atmosphere, which will have an undisputed impact on humankind. We will contribute our innovations and technology from space research, as well as our own sensor data to the analysis."

Dieter Kranzlmüller, Director of the LRZ, says, "The collaboration between these two leading research institutions brings together two partners that complement each other perfectly and contribute their relevant expertise, resources and research topics. The Leibniz Computing Centre has proven experience as an innovative provider of IT services and a high-performance computing centre. It is also a reliable and capable partner for Bavarian universities and will, in future, cooperate with DLR and its institutes in Oberpfaffenhofen."

Huge volumes of Earth observation data

Every day, Earth observation satellites generate vast quantities of data at such high resolution that conventional evaluation methods have long been pushed to their limits. "Only the combination of the online availability of a wide range of historical and current data stocks with cutting-edge supercomputing systems will make it possible for our researchers to derive high-resolution global information that will enable us to make statements about the development and evolution of Earth. Artificial intelligence methods are playing an increasingly important role in fully automated analysis. This enables us to identify phenomena and developments in ways that would be difficult to detect using conventional methods," says Stefan Dech, Director of the DLR German Remote Sensing Data Center. This cooperation is key for the DLR institutes in Oberpfaffenhofen involved in research into satellite-based Earth observation. We can now carry out a range of global methodological and geoscientific analyses, which have been the sole preserve of example cases up until now, due to the sheer quantity of data and limited computing power. The technological data concept jointly developed by DLR and the LRZ is particularly important, as it will link the LRZ up with DLR's German Satellite Data Archive in Oberpfaffenhofen, and, in addition to making global data stocks available online, will link  historical data from our archive and DLR's own data," continues Dech.

A challenge for data analysis

To cite one example, the volume of data from the European Earth observation programme Copernicus has already exceeded 10 petabytes. One petabtye is equivalent to the content of around 223,000 DVDs –  which would weigh approximately 3.5 tonnes. By 2024, the Sentinel satellites of the Copernicus programme will have produced over 40 petabytes of data. These will be supplemented by even more petabytes worth of data from national Earth observation missions, such as DLR’s TerraSAR-X and TanDEM-X radar satellites and US Landsat data. However, it is not only the large amounts of data from the satellite missions that are currently presenting scientists with challenges, but also data on global change that are published on social networks. While these are valuable sources, challenges arise because these data are extremely disparate, their accuracy is uncertain and they are only available for a limited period of time.

DLR researchers are thus increasingly using artificial intelligence (AI) and machine learning methods to identify trends in global change and analyses of natural disasters and environmental contexts in global and regional time series spanning several decades. But these methods require that the necessary data be available online, on high-performance data analytics platforms (HPDAs). The technical objective of this collaboration is to set up such a platform, providing researchers with access to all of the necessary Earth observation data via DLR’s German Satellite Data Archive (D-SDA) in Oberpfaffenhofen and data distribution points of various providers of freely available satellite data.

DLR's German Remote Sensing Data Center (DFD) will coordinate the activities of the participating DLR institutes. In addition to the DFD, the Remote Sensing Technology Institute, the Institute for Atmospheric Physics and the Microwaves and Radar Institute in Oberpfaffenhofen are involved in the project. The Institute of Data Science in Jena and the Simulation and Software Technology Facility in Cologne are also involved in the implementation of the technology.

Cooperation on global change

As part of the collaboration, DLR will address issues relating to environmental development and global change, methodological and algorithmic process development in physical modelling and artificial intelligence, the management of long-term archives and the processing of large data volumes.

The LRZ focuses on the research and implementation of operational, scalable, secure and reliable IT services and technologies, the optimisation of processes and procedures, supercomputing and cloud computing, as well as the use of artificial intelligence and Big Data methods. The LRZ's existing IT systems (including the MUC-NG supercomputer) and its experience with energy-efficient supercomputing will also prove useful.