UC Berkeley

This dissertation is concerned with the motion control of robot manipulators by utilizing vision and inertial sensors. It is separated into two parts, the first part focuses on vision based online trajectory generation for contour following, and the second part is about inertial and vision sensor based end-effector sensing and control for robot manipulators.

Vision camera is a useful robotic sensor since it mimics human vision and allows for non-contact measurements of the environment. Although there is a considerable amount of research on how to include the vision information in the feedback loop(visual servo), the first part of this dissertation will mainly focus on online reference trajectory generation based on vision sensor information. It is well known that most commercially available cameras have a relatively low sampling rate and unavoidable measurement delay due to exposure time, image transportation and processing time etc. A systematic scheme is proposed to overcome these problems in the first part of this dissertation. The proposed scheme is different from pure visual servo since the joint loop is closed by encoder signals in the inner feedback loop and the vision loop in the outer loop to generate reference trajectory. Experimental results of following a variety of contours at different speeds are presented to validate the effectiveness of the proposed trajectory generation scheme.

In the applications of industrial robot manipulators, it is often desirable to obtain accurate end-effector position and velocity information. Estimation based on motor-side encoders alone is often inaccurate due to joint flexibilities and kinematic errors of robot links. A vision based approach may also be insufficient due to its low sampling rate and image processing and transportation delay. With additional acceleration measurements, however, a multi-rate kinematic Kalman filter (KKF) with large measurement delay can be formulated to estimate the end-effector motion accurately without encoder signals. The estimation results based on the proposed scheme are utilized as feedback signals for real-time tracking control. The effectiveness of the proposed scheme is demonstrated by experiments on a single joint setup and a two link manipulator.