Welcome to EE6901 3D Vision web page!

Announcement:

  1. There is a RAP (research among peers among my research students) seminar/group-meeting every now and then on topics in 3D Vision. It might cover topics related to our course. Will let you know the details when available. Or if the class size is small, we will have seminar paper readings led by individual students, tailored to the research interest of that student whenever possible.
  2. This module has a pre-requisite EE5731R. Unless your research is related to Computer Vision in general or you have equivalent exposure to a broad range of computer vision topics, you will not be given waiver to read this module without the necessary pre-requisite.
  3. This module is now run once every alternate year. The next run is in AY 2015/16.

 

Sample Exam Paper     Solution     Sample Exam Paper 2     Solution (Q4a,b,c no longer in syllabus)    Sample Exam Paper 3   Solution

Objective, Syllabus, Assessment Method, Textbooks & General Books 

Note for MSc students and research students not in the Computer Vision area: This is a research oriented (and a tough) module, and you should take up this module ONLY IF the course helps you in your R&D efforts. This course should not be taken merely to get a letter grade. I strongly recommend MSc students and research students not in the Computer Vision area to meet me before you decide to continue with this module. If you wish to get a broad exposure to topics and techniques in computer vision (at the graduate level), EE5907R and EE5731R are better modules. EE5731R serves as pre-requisite to this course in the sense that it gives a broad-based introduction to computer vision topics; however, the topics introduced in this course can be understood on its own if you already have a general exposure to computer vision. Send an email to me if you are not sure. Deadlines for this course are strict, and no excuses (business trips etc.) will be entertained. Failing to submit on time will lead to a failure grade straight away.

 

The class will alternate between lectures teaching the basic mathematical and algorithmic techniques of these methods, and discussion of vision research papers that apply these techniques. Also, you should expect research oriented continuous assessments and self-readings of state-of-the-art technical papers in the field; you should know that there is no clear cut answers for many problems we deal with in this course, and not all materials covered in the required reading can be taught. Some of the mathematical background of these papers are not required, but recommended. Most of the material is open ended and so will be the questions in the assessment. We will test your research oriented mind, not how much you memorize.

My Lecture Schedules:

Topics

Lecture slides

Required Reading

1

Introduction   Mathematics + Geometry of Projection

This is included for reference only: J. Mundy and A. Zisserman, Appendix: Projective Geometry for Machine Vision, in Geometric Invariance for Computer Vision, edited by J. Mundy and A. Zisserman, MIT Press, 1992.

Recognition by components (Biederman, 1987)

2

Problem of Depth Perception I Problem of Depth Perception II

 

3

Stereo I

"Graph Cut Algorithms for Binocular Stereo with Occlusions'', In: Mathematical Models in Computer Vision: The Handbook, Springer-Verlag (2005). ( PDF ) (Omit Section 3 on Voxel labeling)

 

"Fast Approximate Energy Minimization via Graph Cuts'', PAMI 23(11) p1222-1239, 2001. ( PDF )

A more comprehensive explanation of graph cut: M.Eng thesis of Oliver Demetz

 

Suggested Reading from the book of Hartley and Zisserman  

 

An efficient solution to the five-point relative pose problem, PAMI, 26(6):756-770, 2004. (PDF)

 

4

Motion Analysis I   Motion Analysis II

Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods, IJCV 2005. (PDF)

Secrets of Optical flow: Estimation and Their Principles, CVPR 2010 (PDF)

 

5

Multiple View Geometry

Suggested Reading from the book of Hartley and Zisserman    

 

6

Camera Calibration

http://cmp.felk.cvut.cz/minimal/ (various calibration problems & codes for self exploration, not mandatory)

7

Sparse representation, low rank matrix recovery

From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images, SIAM Review, vol. 51, no. 1, pp. 34-81, 2009. (PDF). You can also read the e-book available from our NUS library: Elad, M, Sparse and redundant representations: from theory to applications in signal and image processing, Springer, 2010.

TILT: Transform Invariant Low-rank Textures. ACCV 2010. (PDF)

This is not required but you might need it to understand the ALM technique mentioned in the TILT paper: Z. Lin, M. Chen, L. Wu, and Y. Ma, “The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices (PDF)

8

Motion Segmentation

A multibody factorization method for independently moving objects. IJCV, 29(3): 159-179, 1998. (PDF)

A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate , European Conference on Computer Vision, 2006. [pdf]

Sparse subspace clustering, CVPR 2009. (PDF)

9

Optional topics (if time permits):

Recognition

Human Visual Pathway

Geometric hashing paper by Lamdan et al.

10

Miscellaneous

Background reading required for this class (you are likely to encounter these techniques in many computer vision problems):

o       David G. Lowe, "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, 60, 2 (2004), pp. 91-110.

o       Jianbo Shi and Jitendra Malik. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence , 22(8):888-905, August 2000.

o       Luxburg. A Tutorial on Spectral Clustering. https://arxiv.org/abs/0711.0189

 

Supplementary material:

Beside the lecture slide material given above, you are also expected to read the textbook. In addition, the following handout contains other required readings not found in the textbook.

Biological perception: Introduction to Visual Science   Problem of Depth Perception Motion and Depth perception

Human Visual Pathway: Part I   Part II   Eye, Brain, and Vision by David Hubel

Background mathematics: Maths Refresher       Singular Value Decomposition More properties of SVD       Projective Geometry Professor Gilbert Strang's Lecture on Linear Algebra (MIT webcast of Videos)

Matlab example of solving a LS system using SVD, normal equation, QR decomposition. Note the vastly different answers!

 

Java demo of Epipolar Geometry, etc

 

Supplementary Reading List:

Contain a list of state-of-the-art papers, for PhD student working in this area to master aspects of this subject

 

Matlab resources:

 

  Introductory Tutorials

 

 

Review Questions (Note: these questions serve to help you in reviewing course material and to test your understanding. They are not compulsory.)

#1: General                                                     #1: Solution

#2: Stereo                                                      #2: Solution

#3: Motion                                                       #3: Solution

#4: Multiple View                                             Not available

 

 

Assignments:

My assignment consists of 2 take-home exams. Solutions must be submitted in hardcopy by the deadlines stated on the take-home exam papers. Late submissions will not be entertained (unless an extension is granted by me to the whole class). Take-home exam1 Take-home exam1 solution

Take-home exam2 video you can use in CA2, Q1

 

Seminar presentation: Depending on class size, we will vary the format of seminar presentations. It could be that in some of our classes, we will discuss research papers led by individual students, tailored to the research interest of that student whenever possible. Prior to each of these classes, students must turn in a half-page critique and questions for discussion in class.   Or if there are a lot of students enrolled for the class, it could be that students will give group presentations , in which they synthesize material from a number of papers.  We will settle on the exact format of these presentations in class.  Class Participation:  Everyone should read papers before class and contribute to discussion of them. These seminar components will go towards some percentage of your CA grades. 

Presenters must prepare the presentation material very carefully and meticulously, in order to make genuine contribution to the class. As a teacher, you must read much more carefully than a student. You will be assessed based on your understanding of the material and how well you convey the idea! You might want to propose your own extensions to topics covered in class such as Multi-body factorization, SFM/epipolar geometry for non-conventional cameras such as catadioptric and non-central camera, better optimization techniques such as Graph cut based techniques, semi-definite programming, variational approach to solving various vision problems such as optical flow, segmentation, reconstruction.


Samples of my research:

Much of these work are initiated when I was a graduate student (1992-1996) with my advisor, Yiannis Aloimonos at Maryland. See his Socratic Dialogue which describes the research at the difficulty level of a Scientific American article. Since then, my research programme revolves around various motion-related vision tasks such as motion segmentation, scene reconstruction from multiple views, tracking, action classification. See my recent research


About my other interest


Links of Interest


Major CV Journals:

  • IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI)
  • Computer Vision and Image Understanding (CVIU)
  • International Journal of Computer Vision (IJCV)
  • Image and Vision Computing (IVC)
  • IEEE Transactions on Image Processing (TIP)
  • Pattern Recognition (PR)
  • Machine Vision and Applications (MVA)
  • Major CV Conferences:

  • International Conference on Computer Vision (ICCV)
  • Computer Vision and Pattern Recognition (CVPR)
  • European Conference on Computer Vision (ECCV)
  • International Conference of Image Processing (ICIP)
  • International Conference of Pattern Recognition (ICPR)
  • Calendar of Computer Image Analysis, Computer Vision Conferences

  • Useful links to other resources

  • Geometry in Action
  • Applied Statistics

    Feature Extraction

    • VLFeat (SIFT, MSER, plus fast kmeans, hierarchical kmeans ?)
    • SIFT Demo by David Lowe (closed-source)
    • SURF (Speeded Up Robust Features, original implementation, closed-source)
    • OpenSURF (An opensource implementation of SURF)

    Machine Learning Algorithms

    • LIBSVM (A Library for Support Vector Machines)
    • SVMlight (Another popular implementation of SVM)
    • shogun (A large-scale ML toolbox, specialized in SVM. It provides unified interface for several popular SVM implementations, and features supports for Multiple Kernel Learning. )