Information about Calibration and Processor changes

1. L1C/L2A update and re-definition of scene azimuth angle (see Data Product Specifications DESIS-DLR-ICD-002 Issue 1.3 

2. Changes of processor exit codes

3. Internal changes in SW structure with 2 new python libraries

Regular update of calibration tables

Extension of JobOrder File (adding processing parameter for manual boresight angle definition)

Helper programs for geometric Multi-Scene-Refinements MSR 

L1A / L1C Processor Component – Leap Seconds Fix

• This fix handles the UTC to GPS time as well as GPS to UTC time calculations

L2A Processor Component – Bug Fix

• This bug fix in the algorithm improves the scene AOT (visibility) estimation using dark pixels (Red/NIR wavelengths) affecting mainly water-land scenes (very seldom cases).

Calibration Updated to version V02.06. This SW release contains the following changes (please note that release V02.11 has been never installed at TBE and therefore the changes below are with respect to V02.10)

L1B Processor Component

• Updated of the algorithm to apply the striping correction table to separate the case low temperature gradient from high temperature gradient

L2A Processor Component

• Improvement in the AOT retrieval over water with a smoother transition between land and water
• Change BOA reflectance from unsigned to signed integer to allow negative reflectance
• Set haze sigma threshold and moved haze detection band to around 420 nm avoiding around 430 nm wavelengths.
• Avoid DEM information in the atmospheric correction on cloud pixels
• Avoid over correction over water inside the water vapor absorption bands
• Fixed return value for high zenith angle scenes
• Correct AOT map scaling factor
• Adapt scale path radiance for scenes without DDV pixels
• Changed DEM creation routines for large stdout messages
• Solved wrong error code for high zenith angles
• Improved system calls to avoid problems when stdout is too large
• Increase clouds threshold from 20% to 30% due to the changes in the calibration tables.

Clibration Tables

• Update calibration tables form version v2.04 / v2.05 to v2.06 with an increased number of calibration periods. The difference between the calibrations v2.05 and v2.6 is minimal, except in some particular cases. A look at the 2.5 nm shows some differences.
• Period 0 (until September 2019) in some scenes with big temperature gradient the noise in the etaloning spectral region is smaller because CTB_V02.05 did not have the distinction between the temperature gradients in that period (the calibration for that time period was exactly the same as in CTB_V02.04).
• Period 2 (after May 2020) in some scenes there is an improvement by reducing fluctuations at the lowest wavelengths again in the scenes with high temperature gradient (see specially scenes 892_2, 906_2). In a couple of cases there is also a jump in the whole wavelength range (see scenes 905_2 and 980_9) that is only seen in L2A spectra. This change is due to L2A in V02.12 and has nothing to do with calibration (see slides 10, 11 and 12 to see the effect at L1C level)

In summary, the changes introduced by CTB_V02.06 are very small w.r.t. V02.05.

• Additional support programs (not part of the processing chain, but as helper programs)
• The program DESIS_HSI_PROCESSOR/bin/fixcorruption tries to recover damaged data-takes automatically   

Calibration Updated to version V02.05. This SW release contains the following changes:

L1B Processor Component

• Updated of the algorithm to apply the striping correction table to separate the case low temperature gradient from high temperature gradient

L2A Processor Component

• Improvement in the AOT retrieval over water with a smoother transition between land and water
• Change BOA reflectance from unsigned to signed integer to allow negative reflectance
• Set haze sigma threshold and moved haze detection band to around 420 nm avoiding around 430 nm wavelengths.
• Avoid DEM information in the atmospheric correction on cloud pixels
• Avoid over correction over water inside the water vapor absorption bands
• Fixed return value for high zenith angle scenes
• Correct AOT map scaling factor
• Adapt scale path radiance for scenes without DDV pixels
• Changed DEM creation routines for large stdout messages
• Solved wrong error code for high zenith angles

Calibration Tables

• Update calibration tables form version v2.04 to v2.05 with an increased number of calibration periods
• Additional support programs (not part of the processing chain, but as helper programs)
• The program DESIS_HSI_PROCESSOR/bin/fixcorruption tries to recover damaged data-takes automatically

Calibration Updated to version V02.04. This SW release contains the following changes:

• Metadata xml scheme range for mean AOT value in L2A processing (Bug Fix)
• Corner coordinates of scenes crossing the anti-meridian can be wrong in special cases (Bug Fix)
• Update smile correction in L1B due to temperature gradients in the instrument
• Update of calibration tables
• Correction of the “128 pixel block swaps”
• Quality Image in L2A now with correct size

• Quicklook update introduced in V02.07 discarded

This SW release contains add-ons of the processor and no bug fixes. The following features are added.

• Adding an ENVI header file as a new product component for L1B, L1C and L2A
• Changing QL generation in all products including L1A
• Adding a Recovery tool to handle incomplete data takes 

• Minor bug fix introduced in Version 02.05 at a very special anti-meridian situation 

• Solving problem of images crossing anti-meridian

• Solving problem of partial or blank water image
• Calibration Updated to version V02.03.

• Calibration Updated to version V02.02.
• Change of DEM Database from ASTGTM2 to SRTM-C1ARC due to high noise values in ASTGTM2

Calibration Updated to version V02.01.

This SW release should be regarded as an intermediate version. The next version will include the non-linearity correction based on laboratory measurements provided the product quality is better than before.

• Resolving issue of quality masks generated by L1B and L1C processor
• Adding de-striping algorithm in L1B processing
• Adding new calibration tables for de-striping in CTB_STR
• Minor bug fixes 

This SW release should be regarded as an intermediate version. The next version will include the non-linearity correction based on laboratory measurements provided the product quality is better than before.

• Resolving issue of NaN value in the Water Vapour metadata value
• Resolving format of Ozone metadata (mismatch integer – float)
• Resolving underestimation of cloud coverage

Minor bug fixes

Calibration Updated to version V02.00

• SW Binning implemented, which requests changes in the JobOrder file of L1B and requires al-ways raw data acquired with full spectral resolution. Please have also look to [AD03]
• Change of naming convention of tif images from the extension “geotiff” to “tif”. Please have also a look to [AD01].

Minor bug fixes

Calibration Updated to version V01.01

• A minor bug in the Quicklook DEM is removed; Background value was set to 0, which is normally a valid height.
• TBE (GeoCue) shall test the processors with any of the available real data; shortcomings shall be reported to DLR using the specified interface described in [AD04]

Calibration Updated to version V01.00

• Based on real DESIS data (about 140 data takes up to now) all processors have been tested, mi-nor bugs removed and processing parameters adjusted.
• Calibration tables are re-processed and integrated into the processor system
• TBE (GeoCue) shall test the processors with any of the available real data; shortcomings shall be reported to DLR using the specified interface described in [AD04]

Calibration updated to version V00.87

• New File Format support implemented (especially L1A processor)
• Changes in the Product Metadata
• All processors updated due to metadata change
• L1B processor: DC algorithm update and quicklook alpha channel corrected

Calibration updated to version V00.84

• Inconsistencies in geometric processing (pat file) solved
• TC-TBE-ALL-0010 containing 35 data takes (each data take with exactly one Earth tile)
• Quality algorithms extended and updated.