Sentinel-5P satellite data reprocessing campaign completed
The Copernicus Sentinel-5 Precursor (S5P) satellite mission recently completed its first reprocessing campaign. Its purpose was to assure that the entire S5P product portfolio was generated using the latest versions of the requisite science algorithms. In the course of the campaign some 950,000 products and 1.6 petabytes of data were computed, including those for the atmospheric trace gases methane, nitrogen dioxide, carbon monoxide, ozone and sulphur dioxide. These particular data products are all highly relevant for monitoring air pollution and for climate research on the global scale.
The reprocessing covered the period from end of April 2018 (when routine operation of the S5P satellite began) to the end of July 2022. The result is for the first time a coherent data set for the total product spectrum for the entire course of the S5P mission. It now shows the same high data quality as the continuously generated standard products that are being generated daily by the S5P ground segment team at the Earth Observation Center (EOC) of the German Aerospace Center (DLR).
The success of the reprocessing campaign was made possible by the intensive collaboration of all European partners participating in the S5P mission. The science data is available to the public and can be downloaded using the link on the right to the Copernicus Data Space Ecosystem (CDSE).
DLR’s Earth Observation Center was responsible for reprocessing the data on behalf of ESA and in this work could benefit from its decades of experience in earth observation and the use of the latest technologies.
To reprocess the S5P data, the newly developed cloud-native open-source software system prosEO was utilized. This system can process earth observation data to meet a variety of requirements, from systematic processing during day-to-day operational runs to customized on-demand processing to reprocessing extensive data inventories, as in the current project. The prosEO software is being developed by DLR in collaboration with Dr. Bassler & Co. Managementberatung (BCM) and Prophos Informatik and is accessible at https://github.com/dlr-eoc/prosEO.
The reprocessing uses the cloud infrastructure of IONOS, a cloud service provider selected by DLR in a public call for tenders. Exploiting IONOS’ flexible Infrastructure as a Service (IaaS), an optimal VM (Virtual Machine) configuration with 9 CPUs and 96 GiB RAM could be found. Up to 500 VMs of this size were used in parallel during the campaign.
The prosEO software and the parallelization of data calculation on the cloud scale turned out to be extremely robust despite the high calculation load with a wide diversity of application areas for the various data products.
The DLR project team is grateful for the contributions provided by all the partners participating in this project.
The operation of the Sentinel-5 Precursor Ground Segment is financed by the European Commission as part of the Copernicus program and managed by ESA.
GOME-2 and OMI are precursor instruments of S5P. OMI has been in orbit since 2004 and GOME-2B since 2012. Many products resulting from the S5P missions are based on the experience gained with these instruments. Based on OFFL (offline) total ozone data, algorithms for tropical tropospheric ozone were also developed for these instruments and improved and adapted for S5P.
With the operational OFFL data (see the figure above) we see an increase in the differences between S5P-OMI and S5P-G2B. This indicates that the S5P data for the tropospheric columns drift (or better: increase with time). The increase ended in July 2021 with the first Level 1 update.
Links
