Surface Avatar – an astronaut on board the ISS controls a robot team on Earth

"For us, it is very important to focus on the aspect of human-robot collaboration in order to provide the astronauts with optimal support during crewed spaceflight. To this end, we developed the technologies for collaborative robots some years ago, and these are now widely used in terrestrial applications. With the latest breakthroughs in AI, robots are becoming so versatile and intelligent that they can easily be used by non-roboticists," explains Alin Albu-Schäffer, Director of the DLR Institute of Robotics and Mechatronics.

The robotics team is pursuing two goals with the experiments. On the one hand, they want to demonstrate how different robots can work together to complete complex tasks for space missions. On the other hand, the team is investigating the variations in how the robots can be telecommanded in order to use them as intelligent co-workers at any time exactly as the situation requires.

Work from manual to fully automated

For the current experiment, a Martian landscape was constructed at the German Space Operations Center in Oberpfaffenhofen. In this scenario, three robots were to carry out initial work on the planet's surface in advance of the arrival of humans. Frank Rubio commanded the robots from the Columbus module of the ISS and was able to carry out all the tasks during the two-hour test period. With the help of DLR's humanoid robot Rollin' Justin, he unloaded the lander and installed a seismic sensor. The astronaut used ESA's Interact Rover to monitor the terrain and had the DLR Landing and Mobility Test Facility (LAMA) to support the scientific activities.

Rubio tested a controller with scalable autonomy, which is a completely new technology. In other words, he could determine the extent to which a robot should perform an action autonomously. At the push of a button, he could have a robot perform a task completely autonomously . But the astronaut could also take over the robot as an avatar and carry out individual steps as if with his own hand. For this purpose, he had the Robot Command Terminal (RCT) at his disposal, which combined three control elements. On a screen, he could see what the individual robot saw at any time, control its movements with a joystick and feel what the robot 'felt' thanks to an interactive device with force feedback. For example, when Frank Rubio as Rollin' Justin placed a seismometer on the planet’s surface, the astronaut felt the resistance of the instrument in his hand.

Cooperation on the ground and in space

The RCT is intuitive to use, so Rubio quickly got used to the tele-operation. As a result, he was able to carry out most of the tasks without requiring any assistance. The robotics team monitored the experiment from the Mars laboratory in Oberpfaffenhofen and was in radio contact with the astronaut.

"We are excited to be a step closer to giving astronauts and experts on Earth a wide range of possibilities to command and manage teams of different robots in space. We will be able to use our robots on the surface as physical avatars and intelligent co-workers to execute ever more complex tasks", says Principal Investigator Neal Lii from the DLR Institute of Robotics and Mechatronics.

"This human-robot collaboration paves the way for future missions and permanent outposts on the Moon and beyond," adds ESA Project Leader Thomas Krüger from the ESA Human Robot Interaction Laboratory.

To enable the project team to concentrate on the technical aspects of their work, they were supported by their colleagues at the Columbus Control Centre. The operations team took care of the practical requirements and made sure that Surface Avatar and the other activities on board the ISS were coordinated.

In the future – the Moon and Mars

After the successful technology demonstration, a detailed evaluation will now follow, as well as the preparation of the next simulations. Surface Avatar envisages at least three experiments at intervals of approximately six months. These will become increasingly extensive and complex. The researchers from DLR and ESA will specifically develop the capabilities and telecommanding of the robots for this purpose. The robot team will also be joined by DLR's 'Bert', which can explore rough terrain with its dog-like body and four legs. "The aim is to realise and demonstrate the use of intelligent robots working together with astronauts and to continue to improve the interfaces in order to use the technologies in future crewed missions to the Moon and later to Mars," Albu-Schäffer summarises.

Robots can be used wherever it is too dangerous for humans. Exploration missions in space are therefore inconceivable without robotic support. For future missions, the interaction between humans and robots will play an even more important role. From the Surface Avatar experiments, the researchers are gaining fundamental data on this – for example, how latency affects control, how microgravity influences an astronaut's perception and what challenges arise during operation. For the upcoming Lunar Gateway and other missions to the Moon or Mars, Surface Avatar will thus decisively advance telerobotics.