December 5, 2019 | Human–machine interaction and artificial intelligence in space
CIMON-2 is on its way to the ISS
Launch of SpaceX-19 with CIMON-2
On 5 December 2019 at 18:29 Central European Time (12:29 local time), US SpaceX-19 took off from Cape Canaveral in Florida (USA). On board was the astronaut assistant CIMON-2.
The first mobile astronaut assistant in space, CIMON, has a sibling: CIMON 2. It is shown here during tests in the Columbus mock-up at the European Astronaut Centre in Cologne. CIMON 2 was launched to the International Space Station on 4 December 2019 on board a SpaceX-19 rocket from Cape Canaveral in Florida.
CIMON-2 and three of its 'parents' (from left): DLR project manager Christian Karrasch, medical doctor and scientific director Judith Buchheim, Clinic of the Ludwig Maximilian University Munich and Till Eisenberg, CIMON project manager at Airbus.
Like its predecessor, the technology experiment, developed and built in Germany, is designed to interact with astronauts in the Columbus laboratory.
CIMON-2 has a better 'sense of orientation' and is more 'empathic'.
DLR, Airbus and IBM continue their partnership.
Focus: Space, artificial intelligence
A new Crew Interactive MObile companioN (CIMON) is on its way the International Space Station (ISS). On 5 December 2019 at 18:29 CET (12:29 local time) the US SpaceX CRS-19 mission lifted off from the spaceport at Cape Canaveral, Florida. On board was CIMON-2, an astronaut assistant developed and built in Germany that has been modified and equipped for new tasks. Like its predecessor, CIMON-2 will be used in the European Columbus research module. CIMON is a spherical, free-flying technology demonstrator equipped with artificial intelligence (AI) and designed for human–machine interaction.
"CIMON-1 – our prototype – landed back on Earth on 27 August 2019 after 14 months on the ISS, and has now arrived at Airbus in Friedrichshafen," says Christian Karrasch, CIMON Project Manager at the German Aerospace Center (Deutsches Zentrum für Luft- und Raumfahrt; DLR) Space Administration in Bonn. The DLR Space Administration commissioned the technology experiment from Airbus in Friedrichshafen and Bremen with funding from the German Federal Ministry of Economic Affairs and Energy (BMWi). CIMON's AI is based on IBM's Watson technology. Medical experts from Ludwig Maximilian University in Munich are responsible for scientific matters. CIMON-1 went into operation on 15 November 2018 as the world’s first AI system on the ISS, assisting the German ESA astronaut Alexander Gerst.
"With CIMON-2, we are looking to build on the successful CIMON demonstration," says Karrasch. At its premiere, CIMON demonstrated that an AI-based mobile application could be very beneficial on the ISS while 'working' with Alexander Gerst for 90 minutes. "It is planned that CIMON-2 will stay on the ISS for up to three years and support the crew," explains Till Eisenberg, CIMON Project Manager at Airbus. "CIMON-2 has more sensitive microphones and a more advanced sense of orientation. The AI capabilities and stability of the complex software applications have also been substantially improved." Another important step in CIMON's evolution is its extended lifespan. "During this operational period we will be thinking of further steps, such as uploading the AI to an ISS cloud." This would be a milestone in the gradual evolution of a fully autonomous assistance system. "On a journey to the Moon or Mars, the crew would be able to rely on an AI-based assistance service, without a permanent data link to Earth," says Karrasch. "One specific application for Earth, for instance, would be helping people to perform complex tasks in areas with poor infrastructure."
IBM is responsible for implementing the artificial intelligence for CIMON. "When it was first deployed on the ISS, CIMON proved that it could understand not only the content within its given context, but also the intention behind it," says Matthias Biniok, IBM Lead Watson Architect for Germany. "CIMON-2 goes one step further. With the help of the IBM Watson Tone Analyzer from the IBM cloud in Frankfurt, it is now able to evaluate the emotions of the astronauts and respond to the situation in an appropriate way if this is desired by the astronauts or if its emotional analysis capabilities are being tested as part of an experiment. This allows CIMON-2 to transition from a scientific assistant into an empathetic companion, as required."
The CIMON 'family'
The development and construction of the interactive astronaut assistant CIMON were commissioned by the DLR Space Administration, with funding from the Federal Ministry for Economic Affairs and Energy, and carried out by Airbus in Friedrichshafen and Bremen. Voice-enabled artificial intelligence is provided by Watson AI technology from the IBM cloud. The human aspects of the assistance system were jointly developed and supervised by scientists at the medical centre at Ludwig Maximilian University (LMU) in Munich. A team of around 50 people from DLR, Airbus, IBM and LMU have been working on implementing CIMON since August 2016. The prototype for the technology experiment was on board the ISS from 2 July 2018 to 27 August 2019 and had its world premiere on 15 November 2018, when it assisted ESA astronaut Alexander Gerst for 90 minutes. CIMON's name is, in part, a reference to 'Professor Simon Wright', the robotic assistant – or 'flying brain' – in the Japanese science fiction series 'Captain Future'.
CIMON – The idea
Developed and built in Germany, CIMON is a technology experiment to support astronauts and increase the efficiency of their work. CIMON is able to show and explain information, instructions for scientific experiments and repairs. The voice-controlled access to documents and media is an advantage, as the astronauts can keep both hands free. It can also be used as a mobile camera to save astronaut crew time. CIMON can perform routine tasks, in particular, such as the documentation of experiments, the search for objects and for taking inventory. CIMON is also able to see, hear, understand and speak. Its eyes are actually two cameras that it uses for facial recognition, as well as five other cameras for orientation and video documentation. Ultrasound sensors measure distances to recognise potential collisions. Its ears consist of eight microphones to identify directions, and an additional directional microphone to improve voice comprehension. Its mouth is a loudspeaker used to speak or to play music. At the heart of the AI for language understanding is the IBM Watson AI technology from the IBM Cloud. CIMON was not equipped with self-learning capabilities and requires active human instruction. The AI used for autonomous navigation was contributed by Airbus and is designed for movement planning and object recognition. Twelve internal fans allow CIMON to move and rotate freely in all directions. This means it can turn toward the astronaut when addressed. It can also nod or shake its head and follow through space either autonomously or on command.
