Big Data from Space – seeking solutions for the flood of data from space

“We are receiving very large quantities of data due to the increasing number of satellites, as well as the use of new, higher-resolution sensors, so evaluating these data is proving an ever-greater technological challenge,” says Hansjörg Dittus, DLR Executive Board Member for Space Research and Technology. “For this reason, DLR is increasingly investigating efficient methods and processes such as machine learning. This will enable the provision of informative analysis and recommendations for action on urbanisation, atmospheric change and global warming. DLR is also planning to create the information technology infrastructure required for this purpose – a high-performance data analytics platform.”

“Efficient access, for example via online platforms, is crucial for the analysis of these large quantities of data. The DLR Space Administration is therefore promoting the development of these kinds of options for access and the corresponding processing models. In addition to innovative scientific applications, this opens the way for the development of new business models in the growing downstream sector within Earth observation,” explains Walther Pelzer, DLR Executive Board Member responsible for the Space Administration.

At a reception in the Munich Residenz, Bavaria’s State Minister of Economic Affairs, Regional Development and Energy, Hubert Aiwanger, stressed that: “It is remarkable how much space technology defines our everyday lives – navigation, head-up displays and the entire field of assistance robotics are just three examples among many. Today, climate change and environmental protection are at the very top of the political agenda, and not only in Bavaria. Both are global issues, and both require state-of-the-art technology. Scientific study of the climate system, the atmosphere, tropical rainforests, the oceans and the polar regions are all fields in which space can provide vital data that enables a better understanding of ecosystems and affords a better chance of protecting them. Bavaria has a strong aerospace sector, with a total of 65,000 engineers, technicians and skilled workers. They generate an annual revenue of 10 billion euro.”

Artificial intelligence guiding data selection

New ideas and concepts are needed in order to be able to process data and turn it into information. Artificial intelligence plays a major role in this, as such processes are extremely powerful, especially where large amounts of data are involved. DLR scientist Xiaoxiang Zhu, based at the Technical University of Munich, is conducting research into the use of such methods. Together with her team, Zhu is developing exploratory algorithms from signal processing and artificial intelligence (AI), particularly machine learning, in order to significantly improve the acquisition of global geoinformation from satellite data and achieve breakthroughs in geosciences and environmental sciences. Novel data science algorithms allow scientists to go one step further with the merging of petabytes of data from complementary georelevant sources, ranging from Earth observation satellites to social media networks. Their findings have the potential to address previously insoluble challenges, such as recording and mapping global urbanisation – one of the most important megatrends in global change.

Yet the field of satellite remote sensing is not alone in grappling with this challenge. Investigating phenomena the other way round – looking from Earth into space – also generates enormous amounts of data. Telescopes such as the Square Kilometre Array (SKA) in South Africa and Australia provide large quantities of data, as do ESA’s space-based telescopes, for example Gaia and Euclid. The systematic analysis of archive data by self-learning AI programs is thus becoming increasingly important in astronomical research.

Intelligent search portals provide the desired information

Earth observation satellites and telescopes show that methods drawn from data science and artificial intelligence are indispensable. Suitable storage and intelligent access options are required in order to be able to handle and evaluate these large quantities of data. Access is the key factor for interested users. Today this mostly takes the form of an online portal such as CODE-DE (Copernicus Data and Exploitation Platform, Germany), which is run by the DLR Space Administration. This allows users to access all of the data and process them there directly. Scientists can thus utilise a convenient online working environment for studying satellite data, provided free of charge under the Copernicus programme. This saves local storage space and computing capacity for users.

The data portals themselves are also the subject of research. What makes the ideal platform, and which search options should be made available to the user? Following the example of internet search machines, images of distant planets or Earth’s surface are easy to find by entering information such as “Which celestial bodies are considered likely to have water?” or “Which areas of Germany are particularly affected by drought in summer?” Modern portals should provide the correct datasets in response to such queries. The long-term availability of this global information is also of great importance. Similar to a library, it must be ensured that satellite data are preserved and can be found and retrieved over a long period of time. This is the purpose of facilities such as the German Satellite Data Archive (D-SDA) at the DLR Earth Observation Center in Oberpfaffenhofen.

Time series – simultaneously a curse and a blessing