Our ability to dexterously manipulate objects with our hands is an essential element of our daily lives. In order to seamlessly integrate robotic systems into our everyday life, they will require similar capabilities. Enabling robots to manipulate objects with precision requires a set of algorithmic skills, equivalent to the cognitive abilities of humans. To achieve this goal we are developing a dexterous manipulation framework, which enables the accurate control of grasped objects with robotic hands. Fundamentally, accomplishing a demanding manipulation task requires knowledge about the state of the grasp. Our grasp state estimation method integrates information from tactile sensing, proprioception and vision into a common formulation. Utilizing the estimated grasp state, the developed model-based controller realizes the compliant positioning of the object inside of the hand.

The complexity of the elastic neck makes it a good candidate for applying machine learning methods, where modeling the robot behavior can be hard. Methods such as ensembling for predicting the pose and handling failures, together with reinforcement learning for learning to control the neck from data are explored.

The capability maps provide a graphic representation of every reachable point in the discretized workspace and the expected level of dexterity in every reachable point. The different levels of dexterity are color-coded related to a capability index given to each reachable point. The capability index is obtained from sampling the possible end-effector orientations at each point in the reachable workspace. The resulting percentage of occupancy defines the capability index and the final color of the discretized point (according to a colormap). Therefore, the green regions observed in the image represent a group of points with several possible end-effector orientations for the same point and thus a region with a better capability index than the regions at edges of the map.