CIMON-2 makes its debut on the ISS
Your consent to the storage of data ('cookies') is required for the playback of this video on Youtube.com. You can view and change your current data storage settings at any time under privacy.
ESA/DLR/NASA
- A technology experiment on human-machine interaction in space implemented by the DLR Space Administration, Airbus and IBM has successfully taken its next step with ESA astronaut Luca Parmitano on the International Space Station.
- Scientific research into the impact of stress and isolation on long-term missions.
- Focus: Artificial intelligence, human-machine interaction
CIMON-2, an updated version of the Crew Interactive MObile companioN astronaut assistant developed and constructed in Germany, has successfully proven its capabilities on board the International Space Station (ISS). The spherical, free-flying technology demonstrator equipped with Artificial Intelligence (AI), demonstrated its capabilities by interacting with the ESA astronaut Luca Parmitano. On 5 December 2019, CIMON-2 was launched from Cape Canaveral, Florida, on the CRS-19 supply mission to the ISS. It is planned that it will remain there for up to three years. Nearly two months after the successful deployment of CIMON-2 in February, the project team has now completed its initial analysis.
Among other things, CIMON-2's autonomous flight capabilities and its voice control system, used for navigating and assigning tasks were tested. The demonstrator successfully conducted an approach to a specific point within the European Columbus Module on the ISS for the first time. By performing absolute navigation within the space, CIMON-2 showed that it was able to reach a defined place using verbal commands, regardless of its location when they were issued. During the initial operation of the new hardware and software, Parmitano asked CIMON-2 to fly to the Biological Experiment Laboratory (Biolab) within the research module.
Another task was the acquisition of images and videos within the Columbus Module on command and then showing them to the astronaut. CIMON-2 will be able to use these capabilities to support scientific experiments on the ISS.
The current version of the technology demonstrator has more sensitive microphones and a more advanced sense of orientation than its predecessor (CIMON). The AI capabilities and stability of the complex software applications have also been significantly improved for CIMON-2. In addition, the autonomy of the battery-powered assistant has been increased by approximately 30 percent. With CIMON-2, an astronaut can activate a linguistic emotion analysis on request, which enables the AI assistant to respond to their co-worker in an empathic way.
Another of the project's goals is to research the potential for reducing stress by using an intelligent assistant such as CIMON. As a partner and companion, CIMON would be able to support astronauts with their high workload of experiments, maintenance and repair tasks, and thus reduce their exposure to stress. CIMON is laying the groundwork for a potential social assistance system in space, which could reduce stress due to isolation or group dynamics on long-term missions. In addition, it could possibly contribute towards alleviating such problems on Earth. The CIMON team, comprising researchers from the German Aerospace Center (Deutsches Zentrum für Luft- und Raumfahrt; DLR), Airbus, IBM, Ludwig Maximilians University Munich and the ESA User Support Centre in Lucerne (Switzerland), has been delighted with the performance of CIMON-2 so far, as the new, improved hardware and complex software have proven highly effective. "This new success for the CIMON project represents another pioneering achievement for the use of AI in crewed space exploration," says Christian Karrasch, CIMON Project Manager at the DLR Space Administration in Bonn.
CIMON-2 was launched to the ISS from the Kennedy Space Center in Cape Canaveral, Florida on 5 December 2019. It is planned to remain there for up to three years, with further experiments scheduled to take place. Its predecessor, CIMON-1, returned to Earth in August 2019 after a total of 14 months on the ISS.
The CIMON 'family'
The development and construction of the interactive astronaut assistant CIMON were commissioned by the DLR Space Administration with funding from the German Federal Ministry for Economic Affairs and Energy. The work was performed by Airbus in Friedrichshafen and Bremen. Voice-enabled artificial intelligence is provided by Watson AI technology from the IBM cloud. The human aspects of the assistance system have been jointly developed and supervised by scientists from the medical centre at Ludwig Maximilians University in Munich. The ESA BIOTESC User Support Centre at the Lucerne University of Applied Sciences and Arts in Switzerland has ensured that CIMON also works perfectly in the Columbus Module, and is following the work that the astronauts perform with CIMON from Earth. The prototype for the technology demonstration was on board the ISS from 2 July 2018 to 27 August 2019 and had its world debut on 15 November 2018, when it assisted ESA astronaut Alexander Gerst for 90 minutes. It is no mere coincidence that CIMON’s name has echoes of 'Professor Simon Wright', the robotic assistant – or 'flying brain' – in the Japanese science fiction series 'Captain Future'. After the successful mission of CIMON-1, the first European autonomous robot to be used in crewed space exploration was named a German cultural asset and returned to Earth. CIMON-2 was implemented in less than a year by the approximately 20 researchers that make up the CIMON 'family'.
CIMON – the concept
Developed and built in Germany, CIMON is a technology experiment to support astronauts and increase the efficiency of their work. CIMON is able to show and explain information, instructions for scientific experiments and for repairs. The voice-controlled access to documents and media is an advantage, as the astronauts can keep both hands free. It can also be used as a mobile camera to save astronaut crew time. CIMON can perform routine tasks, such as documenting experiments, searching for objects and taking inventory. CIMON is also able to see, hear, understand and speak. Its 'eyes' include a stereo camera used for orientation, a high-resolution camera for facial recognition and two lateral cameras for imaging and video documentation. Ultrasonic sensors measure distances for collision detection. Its 'ears' are eight microphones used to detect the direction of sound sources plus a directional microphone for speech recognition. Its 'mouth' is a loudspeaker through which it can talk and play music. At the heart of the AI for language understanding is the IBM Watson AI technology from the IBM Cloud. CIMON is not equipped with self-learning capabilities and requires active human instruction. The AI used for autonomous navigation was contributed by Airbus and is designed for movement planning and object recognition. Twelve internal fans allow CIMON to move and rotate freely in all directions. This means it can turn toward the astronaut when addressed. It can also nod or shake its head and follow its human co-worker either autonomously or on command.