Data Quality as a Development Aspect of Save AI
Poster P01-01 at the HelmholtzAI 2024, Düsseldorf, Germany
by Lars Steffens & Oscar H. Ramiréz-Agudelo
![](/de/ki/aktuelles/veranstaltungen/data-quality-of-safe-ai/ingedients-for-safe-ai/@@images/image-1600-f57f83e2da3fdba03aad9796d4857c68.jpeg)
Introduction
One of the key pillars of AI is the data used to set up the AI systems (Fig. 1). Therefore, the quality of the data used to feed the AI model has a decisive impact on the quality of the AI system and thus on its safety. Degradation of the data quality (DQ) might have a high impact safeguarding critical systems.
Example: If training data for an AI system for automated driving does not contain a road sign or only in poor image quality or the image is incorrectly labeled, the recognition of road signs will not work properly and thus endanger traffic.
This example addresses the DQ characteristics completeness, (semantic) accuracy and understandability.
Therefore, well-established concepts of DQ management are beneficial to ensure AI safety.
Methodology
DQ can be defined as the “degree to which the characteristics of data satisfy stated and implied needs when used under specified conditions” [1].
As shown in Fig. 2, to ensure DQ, the use case must be well defined to formulate precise DQ requirements. These can be assessed by a suitable DQ model consisting of DQ characteristics and DQ measures as defined, for example, in the forthcoming ISO/IEC 5259 [2].
![](/de/ki/aktuelles/veranstaltungen/data-quality-of-safe-ai/components-of-data-quality-management/@@images/image-1600-d433f24a6a9f46e36ff7f14bfcedc27e.jpeg)
Roadmap
These well-established DQ concepts can serve as the basis for a roadmap for incorporating DQ aspects into the development process of safety-critical technologies involving AI, especially when discriminating the different stages of the process.
Using a common approach, these stages of development can be defined by the Technical Readiness Level (TRL) system [3]. This system makes it possible to differentiate the use of existing techniques in different phases of the development process, in particular to systematically refine the use case, requirements and the DQ model (Fig. 3).
Example: The data for a laboratory prototype (TRL 4) has different requirements than those for a field prototype (TRL 6-7) or those for a fully operational system (TRL 9).
This refinement also applies to the process of data selection and optimization. In certain development scenarios, synthetic data can be used for lower TRLs allowing better and controllable data quality initially, so that the requirements for integrating real data can be defined later. The requirements can be kept simpler and thus examined in detail for lower TRLs and refined on based on the results obtained for higher TRLs, which also allows mapping relevant DQ characteristics to safety-relevant AI quality characteristics.
![](/de/ki/aktuelles/veranstaltungen/data-quality-of-safe-ai/refinement-of-data-quality-with-trl/@@images/image-1600-574ce69d6c12292488020f4fdee7c73a.jpeg)
Appendix
References
[1] ISO/IEC 25012:2008, Data quality model
[2] ISO/IEC DIS 5259-2, Data quality measures
[3] ISO 16290:2013, Definition of the Technology Readiness Levels (TRLs) and their criteria of assessment