20 Years Remote Sensing Technology Institute

Our mission is to extract geophysical variables and relevant geoinformation from remote sensing data. We strive to continuously improve the quality and availability of the derived information in order to open up new applications. To this end, we develop lab-based, physical, mathematical and information technology methods and algorithms, validate them in scientific application projects and implement them in operational processing systems for integration into ground segments (e.g. of the DFD and industry). With the knowledge thus gained, we also support the development of new sensor and mission concepts.

We see ourselves as an intermediary between Earth observation sensor systems and the users of geoinformation derived from remote sensing data, thus contributing to the advancement of Earth observation.

Our current research and development focuses on

• SAR (Synthetic Aperture Radar)
• imaging optical remote sensing
• passive and active atmospheric spectrometry
• data science and artificial intelligence

and so, with the exception of the new area of Data Science and Artificial Intelligence, essentially the same three remote sensing technology lines with which the institute started 20 years ago. But one step at a time ...

No Y2K bug: The IMF was founded just after the Millennium Fireworks on January 1, 2000, out of

• the "Algorithms and Processors" department and other scientists of the former DFD
• the majority of the Institute of Optoelectronics and
• the "Marine Ecosystems" department of the Institute for Space Sensor Technology

Together with the German Remote Sensing Data Center (Deutsches Fernerkundungsdatenzentrum – DFD) and a joint controlling unit, the institute formed the Applied Remote Sensing Cluster, which has been operating under the name Earth Observation Center (EOC) since 2010. The collegial cooperation of the IMF and the DFD in the EOC has proven to be a model of success over the past 20 years; in practically all major projects, applications and missions, we combine our respective expertise in an often "unbeatable" package.

The IMF was planned as an institute for algorithm development long before algorithms had entered the public discussion. It was to develop methods for extracting information from the sensor data of different remote sensing technologies, e.g. to generate high-quality imagery from raw data or to retrieve geophysical and geobiochemical parameters. Thus, the IMF positioned itself in the above-mentioned system chain between the institutes that developed optical and radar sensors, and e.g. the DFD, which devised high-quality application products and conducted geoscientific research with the information gained. So much for theory.

Of course, the question arose at the outset as to where the institute's development opportunities lay, sandwiched in this position. It was also foreseeable that there would be certain pressure from both sides with sensor development moving towards data evaluation, and geoscientific research having to develop strong methodological elements. It could well have been that the IMF would become marginalized in the DLR institute landscape after a few years. Nevertheless, three aspects have enabled us to develop very well: We have expanded our breadth of topics (i.e. orthogonal to the sandwich), positioned ourselves with unique selling points, and finally, the boom in Earth observation (above all through the European Copernicus program) has opened up ample room for methodological research and initiated a shift in emphasis towards modern evaluation algorithms – the very topic of the IMF. Some of our key scientific indicators provide convincing evidence of the success and development of the institute over the past two decades:

• When the Institute was founded, two of its staff were professors, i.e. held the “Promotionsrecht”; today there are 13 (appointed, guest, honorary and adjoint teaching professors) at the IMF and our two associated TUM chairs.
• The number of doctoral students increased from about 15 to currently 60.
• At the beginning, the IMF produced less than 30 papers per year. With more than 200 peer-reviewed publications per year, the IMF now ranks among the top three DLR institutes.
• IMF scientists win many coveted scientific awards.
• Today, the IMF and the two chairs teach in more than 20 different courses at several German and international universities, are represented in many advisory, steering and expert committees, and are actively involved as editors of professional journals.

When it was founded, the IMF set itself the following thematic priorities: Remote sensing of the atmosphere, SAR processors, digital elevation models from SAR and photogrammetry, data fusion, data mining and oceanography. These were represented by the former heads of department Dr. Tom Rother, Prof. Manfred Schroeder, Dr. Peter Haschberger and Dr. Andreas Neumann (deceased in 2012). Even today, 20 years and three institute evaluations later, these topics are still reflected in our five departments:

• Atmospheric Processors (Prof. Thomas Trautmann)
• SAR Signal Processing (Prof. Michael Eineder)
• Photogrammetry and Image Analysis (Prof. Peter Reinartz)
• EO Data Science
• Experimental Methods (Dr. Peter Haschberger)

Since then, a lot has been added in terms of content, such as our involvement in DLR's transportation program, the large Copernicus atmospheric line, SAR interferometry for ground subsidence monitoring, the IMF-initiated research area of SAR Imaging Geodesy, maritime security with our new subsidiary in Bremen, hyperspectral remote sensing with our own instruments, and the boom topic of data science, AI and machine learning in Earth observation ("AI4EO"), for which the foundation had already been laid by data fusion and data mining.

The IMF "inherited" two laboratories from the former Institute of Optoelectronics, one for spectroscopy of gases and a second one for characterization of imaging spectrometers (today in the function of the ESA Calibration Homebase (CHB) for the hyperspectral instrument APEX). Initially seemingly foreign objects, both laboratories have proven to be important elements of the institute's portfolio, especially with regard to sensor-related data analysis. The institute generates spectroscopic reference data from trace gas spectra measured in the lab, and characterizes and calibrates, among other things, our own aircraft and field instruments, setting quality standards in both areas.

Major advances in the IMF's development were often triggered by German and European missions in which the Institute was able to contribute processing algorithms and software – i.e. processors – for the ground segment. We have developed the operational processors for all German civilian SAR missions, including SRTM, TerraSAR-X and TanDEM-X, as well as for the Spanish PAZ, the DESIS and EnMAP hyperspectral missions and the Franco-German methane mission MERLIN. We have become an indispensable player in the development of processors for the European atmospheric emissions GOME, SCIAMACHY, GOME-2A -B -C, Aeolus, TROPOMI/Sentinel-5p, Sentinel-4 and Sentinel-5 and are now responsible for the majority of the trace gas product portfolio. In addition, we were and still are involved in many other missions, either in the concept phase or by developing processors and components. Radiation transport codes, inversion algorithms, atmospheric correction, high-precision geolocation, SAR interferometry or stereo processing, which have also been successfully licensed to industry, are just a few examples.

However, this dependence on new missions always represented a risk that we could hardly influence ourselves. For example, there has not been a launch of a German Earth observation satellite for over ten years. The planned EnMAP and MERLIN missions are delayed by many years. We have therefore increasingly adopted a twin-track approach: On one hand, we are building up sustainable and long-term knowledge in the form of software systems that are continuously improved and used as operational processors in ground segments at the DFD, EUMETSAT, ESA or in industry. In parallel, we are conducting research on new evaluation algorithms and more explorative topics, which often take several years to become operational. Both lines cross-fertilise and complement each other; they can – within limits and reasonable time scales – be mutulally ramped up or scaled down depending on the demand and funding situation.

"More than the past, I am interested in the future, because I intend to live in it". This quote from Albert Einstein came to my mind even before I had written the first lines of this birthday post. So now a few words about what is currently fun and will determine our future.

For some years now, Earth observation with satellite remote sensing has been experiencing unprecedented dynamics and significance. With its powerful fleet of Sentinel satellites, the European Copernicus program has created observation capacities that far surpass everything that has existed up to now – and with a long-term perspective that extends beyond 2040. This program has made Europe the largest producer of Earth observation data and information.  The free-and-open data policy and the Copernicus services have led to a broad acceptance of remote sensing information beyond the scientific community in government, politics and business. NewSpace companies are positioning themselves on the market with creative new satellite concepts, internet giants such as Google and Amazon are entering the field with their cloud capacities and start-ups with AI algorithms – Earth observation is becoming more colorful, more diverse and faster.

This is an exciting time of change, and we are helping to shape it. Just a few examples: Ground segments are increasingly cloud-based and distributed worldwide; with the DESIS processor (in the Amazon cloud) we have already taken this step, our processors for Sentinel-5p are also cloud-enabled, and SAR processors will follow. Evaluation algorithms use AI and machine learning, process huge amounts of data and require large computing capacities; with the massive expansion of our data science and applied AI research at the IMF and the TUM, we were pioneers in this field and today hold a leading position. Open Data and Open Software has become a natural expectation of the user community, but also an accelerator for scientific research; we make huge, valuable, reference data sets and algorithms freely available.

All this sounds like exclusively technological goals. However, we never forget that with our efforts to derive ever better and thus ever more relevant information and knowledge from remote sensing data, we want to contribute to the solution of urgent problems, wherever geoinformation is needed. These can be the oft-quoted Grand Challenges, as they are derived from global change or named in the sustainable development goals of the United Nations, but also very practical and planning issues, such as mobility, maritime safety and security, air quality or natural geological hazards.