People Education Research Industrial Agenda  
Overview MSc info MSc program MSc topics ET TN WBMT  

ME1140: 3D Robot Vision
Responsible Instructor: P.P. Jonker
Instructor: B. Calli, R.G. Prevel
Contact Hours / Week x/x/x/x: 0/2/0/0
Education Period: 2
Start Education: 2
Exam Period: 2, 3
Course Language: English
Course Contents: Vision is an essential tool for robot sensing especially if these robots have to operate in an uncontrolled environment. Vision is used by robots to locate and identify objects in 3D or to build a map of their environment for localization and navigation purposes. The design of such system is a complex task that relies on complex techniques. The goal of this course is to give and highlight on related problems and popular algorithms and methods to solve them. The course focuses on 3D computer vision techniques applied to robotics with a highlight on stereovision approaches. Lectures are organized as follows:
* Camera models and projective geometry
* Stereovision
* Stereo matching
* 3D interpretation
* Object recognition
* Motion
* Visual Servoing
Study Goals: Provide student with an overview of the state of the art algorithms for 3D Robot Vision. After successfully finishing the course student is able to read, discuss, summarize and comment on scientific journal and conference papers in this area.
At the end of the each topic, students shall be able to:
* Camera models and projective geometry
- Have an insight on digital camera architecture and sensors
- Know the basics of optics and how to choose a lens for a specific purpose
- Understand the basic principles of projective geometry, homographies and projection transformations in space
- Understand how perspective projection is used for camera modeling
- Understand lens distortions in the camera model
- Understand and apply the basic principles of camera calibration
* Stereovision
- Understand concepts and limitations in stereo vision system; know how to estimate depth using triangulation.
- Understand importance of stereo matching, difference between dense stereo matching and sparse stereo matching.
- Understand parameters of cameras and epipolar geometry
- Calculate essential matrix and fundamental matrix; Perform image rectification.
- Apply different 3-D reconstruction methods according to priori knowledge.
- Implement mathematic function for depth estimation and the pseudo image rectification algorithm.
* Matching
- Understand basic idea behind matching. What to match: points, lines or regions.
- Describe and apply basic image filtering techniques
- Provide an overview of similarity measurements (Correlation, Mahalonobis distance, Hamming distance) and use them in a matching algorithm.
- Comprehend steps of dense stereo matching algorithms, apply and design a template matching algorithm
- Compare different keypoint detectors, extract affine and illumination invariant features and design a feature matching algorithm.
- Describe problems of stereo matching (e.g dealing with illumination change, improving accuracy and selectivity)
* 3D interpretation
- Understand different 3D information extraction methods via various sensors (laser, sonar, lidar)
- Understand the basic components of 3D information and how to use them
- Understand local surface properties and extract them from different data types
- Compare different local surface extraction methods
- Comprehend the steps of homogeneous region segmentation for grouping similar regions.
- Understand and utilize basic tools like region growing, mean-shift, Hough transform and Random Sample Consensus (RANSAC)
- Understand how to infer (primitive) geometric shapes (e.g. planes, cylinders) by fitting models on segmented 3D data.
* Object recognition
- Understand basic idea of model based object recognition
- Compare and apply different global image descriptors (color, texture, shape)
- Calculate affine invariant local keypoint descriptors
- Combine global and local 2D descriptors.
- Design function for object recognition under real world constraints
- Understand the basic idea of local 3D keypoint extraction
- Calculate local 3D descriptors via Local Surface Properties (LSP)
- Understand the basic idea of global 3D descriptors
- Understand how to combine global and local 3D descriptors
* Motion
- Understand concepts and problems of motion analysis.
- Comprehend two different methods of motion analysis.
- Construct the basic equation of motion field and perform optical flow algorithm.
- Track objects using Kalman Filter algorithm.
- Implement Lucas-Kanade optical flow algorithm.
* Visual servoing
- Understand basic concept of visual servoing
- Understand and apply image based visual servoing approach
- Understand and apply position based visual servoing approach
- Provide an overview of different Hybrid Approaches
- Simulate different visual servoing algorthms using Matlab Visual Servoing Toolbox
Education Method: Lectures (2 hours per week)
PC practical (2 hours per week)
Literature and Study Materials: 1. Book: “Introductory Techniques for 3-D computer vision”, Emanuele Trucco and Alessandro Verri ( Main course book)
2. Book: “Multiple view geometry”, Richard Hartley and Andrew Zisserman,
3. Book: “The Geometry of Multiple Images”, Olivier Faugeras and Quang-Tuang Long
Assessment: Exams during the course period
Each subject is examined during the course (7x)
Last modified: 25 November 2013, 16:30 UTC
Search   Site map