Problem formulation

Imitation learning for dexterous robotic hands, such as the Shadow Hand, faces significant challenges in transferring skills learned in simulation to real-world applications. Discrepancies between simulated and real environments can lead to performance degradation when deploying models trained in simulation. Developing a robust sim-to-real framework, supported by a tailored simulation environment, is essential for improving the transferability of learned manipulation skills to the real Shadow Hand.

Task definition

This thesis will focus on creating a sim-to-real imitation learning framework for the Shadow Hand, utilizing a simulation environment based on Nividia Isaac Sim designed to closely replicate real-world conditions. The research will involve designing and implementing a simulation setup that mimics the physical properties and constraints of the real Shadow Hand, followed by training imitation learning models within this environment. The effectiveness of the sim-to-real transfer will be evaluated by comparing the performance of the Shadow Hand in both simulated and real-world manipulation tasks, with a focus on accuracy, adaptability, and robustness.

You shall offer

• Solid knowledge base and experience in deep learning, and robotics.

• Coding skills in Python and C++.

• Experience with ROS.

• Experience with Simulation is beneficial.

We will offer

• The most state-of-the-art technologies in deep learning and computer vision.

• Working in a lab with a Germany-wide unique Shadow Teleoperation System.

• Tight support from supervisors, including a workshop on scientific writing.