Problem formulation

In recent years, due to the growing demand for robots capable of handling objects with varying shapes, sizes, and material properties in unstructured environments, one state-of-the-art learning-based robot control algorithm, namely diffusion policy, which leverages the diffusion process to generate multimodal robot behaviors that handle various scenarios and tasks, become interesting in the research. Hence, we aim to examine their generalization abilities in terms of grasp learning.

Task definition

This thesis involves developing and implementing diffusion policies for robotic grasping in cluttered environments, where the robot must efficiently and reliably grasp target objects while navigating and avoiding collisions with surrounding objects. The diffusion process will be used to generate multimodal grasping strategies that adapt to the complexities of cluttered settings, predicting the effects of the robot's actions on both the target and obstructing objects. Our focus will be on realtime adaptation, leveraging both simulated and real-world data, to train and refine these policies through reinforcement learning, ultimately evaluating success based on grasp success rate, collision avoidance, and adaptability to diverse scenarios.

You shall offer

• Solid knowledge and experience in computer vision, deep learning.

• Coding skills in Python and Linux.

• Experience in simulation is a plus.

We will offer

• Real Robot for experimentation.

• GPU clusters for training your policies.

• State-of-the-art technological support and professional supervision

References

• Chi, Cheng, et al. "Diffusion policy: Visuomotor policy learning via action diffusion." arXiv preprint arXiv:2303.04137 (2023).

• Ze, Yanjie, et al. "3d diffusion policy." arXiv preprint arXiv:2403.03954 (2024).

• Urain, Julen, et al. "Se (3)-diffusionfields: Learning smooth cost functions for joint grasp and motion optimization through diffusion." 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023.