Object Sorting Using Eye-in-Hand Manipulator

Last updated on April 2022.

---

Introduction

This project was a course project of ECE4310 Programming for Robotics, Spring 2022, at CUHK-Shenzhen. In this project, I programmed a 6-DoF serial manipulator (Interbotix WidowX 250) with eye-in-hand camera configuration to perform an intelligent sorting task: automatically places the different colored blocks into the corresponding colored buckets.

Method

In this project, I used the ROS MoveIt! toolkit to implement most of the functions, including motion planning and obstacle avoidance. I used the OpenCV library to implement a HSV mask based method to detect and distinguish the differently colored targets.

I used linear regression for single pose eye-in-hand calibration (the robot is programmed to recognize objects using the camera only in a preset pose. Therefore, only the preset pose need to be calibrated).

\[\begin{bmatrix}x_{w}\\y_{w}\end{bmatrix} = \boldsymbol{k}^{\top} \begin{bmatrix}x_{c}\\y_{c}\end{bmatrix} + \boldsymbol{b}\]

Where $[x_{w}, y_{w}, 0]^{\top}$ is target’s position in world frame, and $[x_{c}, y_{c}]^{\top}$ is target’s position in camera image frame. In this project, coefficients $\boldsymbol{k}$ and $\boldsymbol{b}$ were computed by using least squares on multiple pairs of sampled target positions.

For more info

You may check the project report for more technical details.

Demo

Here is a video demo of this project.