Welcome to my EE4212 Computer Vision web page!

 

 

Announcement:

1. This supercedes the earlier announcement about classroom arrangement. Follow the announcement on LumiNUS posted on 11 Jan 2021. Specifically, we have divided the class into 2 halves based on alphabetical order as per the class list in LumiNUS on 11 Jan 2021 (for details, see the announcement on LumiNUS). The 1st half will attend the odd-week lectures (1,3,5) in the face-to-face mode (i.e. come to classroom), whereas the 2nd half will do so for the even-week lectures (2,4,6). In the alternating weeks for the respective 2 halves, they will attend via zoom (Join through the LumiNUS conferencing portal). Note that if you are only auditing as guest students, you should only attend via zoom.

2. NB: for everyone who prefers to e-learn instead of coming to the f-f lessons, please note that the webcast lectures are recorded from previous cohort (2016), but the content is largely the same so it is perfectly ok to listen to the webcast version. The main difference is in the way we will be administering the quizzes in the first half of the course (week 2, 4, 6). Instead of doing it during the lessons, you will do them at home and then submit through LumiNUS. More details to be announced later.

3. Please also note that we are going to miss a lesson due to the Chinese New Year holiday. The make-up lesson will be on 26 Feb (Fri) during the recess week, same time, same venue.

Sample Exam Paper 2002 Sample Exam Solution 2002 NB: Q1 & 2 are no longer in the syllabus of EE4212.

Sample Exam Paper 2004 Sample Exam Solution 2004 NB: Q1 & 2 are no longer in the syllabus of EE4212.

Sample Exam Paper 2008 Sample Exam Solution 2008

Sample Exam Paper 2010 Sample Exam Solution 2010

Objective, Syllabus, Assessment Method, Textbooks & General Books Textbook available in NUS Co-Op

Learning Strategies

My part of the course on 3D Vision is somewhat unconventional in terms of both teaching and assessment. You will be taught concepts from various cross-disciplinary perspectives: engineering, computational, biological and evolutionary. For instance, we will consider how compound-eyed insects can fly with precision through cluttered environments, avoid obstacles and land on demand on surfaces oriented in various ways. Despite the cross-disciplinary perspectives, the core of 3D Vision relies on a lot of mathematics and is regarded by most engineers (and past students of this module) as a very difficult area. To understand the current research in 3D vision, concepts in advanced linear algebra and projective geometry are required. In terms of assessment, I place a great deal of emphasis on getting you to be actively involved in problem solving at an advanced level. Thus I give substantial take-home examinations (see “Assessment further down) where some of the questions require students to read up current literature and to do more thinking and writing than is possible in an in-class exam. You are also welcome to discuss the questions with me. It is tough, but I believe it will achieve more in-depth learning and you will get a foretaste of research. If you really want to understand and explore the complexity of a visual system, this course is for you!

Graduate Assistant for this course  -Mr Jiang Zihang: jzihang@u.nus.edu ;

(You can email him if you have problems regarding anything about the course). Venue: Vision & Machine Learning Lab; Block E4 #08-24.

If you have any other problems or comments, e.g. difficulties in following course material, interest in pursuing Ph.D. research in 3D vision, do contact me by e-mail (eleclf@nus.edu.sg) and set up an appointment.

 

 

Tutorial Questions (These questions serve to help you in reviewing course material and to test your understanding. They will be discussed in class and your participation is expected.)

#1: General #1: solution

#2: Stereo #2: solution

#3: Motion #3: solution

 

My Lecture Slides: Note that these slides are no substitutes for attending the lectures; students who skip lessons will find that the slides don’t make much sense as they are only a summary of the lessons.

Topics 1: Introduction Mathematics + Geometry of Projection

Topics 2: Problem of Depth Perception Part I Problem of Depth Perception Part II

Topics 3: Stereo Analysis

Topics 4: Motion Analysis I Motion Analysis II

Supplementary material:

Beside the lecture slide material given above, you are also expected to read the textbook. In addition, the following handout contains other required readings not found in the textbook.

Introduction: Introduction to Visual Science Problem of Depth Perception

Structure from Motion: Motion and Depth perception

A Matlab example of solving a LS system using SVD, normal equation, QR decomposition. Notice the vastly different answers! This example also serves to introduce you the powerful functions available in Matlab and would be useful in your take-home exam. For further introduction on matlab, see these tutorial files [Basic Operations | Programming | Working with Images ]. This ppt (courtesy of the course GA) will give you some basic knowledge about working with images in MATLAB and also contains several examples illustrating good MATLAB style Basic Operations on Images in Matlab

Maths Refresher Singular Value Decomposition Projective Geometry Professor Gilbert Strang's Lecture on Linear Algebra (MIT webcast of Videos)

 

An excellent though much more expensive reference for 3D computer vision is Richard Hartley and Andrew Zisserman's book Multiple View Geometry in Computer Vision, Cambridge University Press, June 2000. Of particular interest is the sample chapter available online: Epipolar Geometry and the Fundamental Matrix (beware that the material is discussed at a much higher level than Trucco’s, your course textbook, and is therefore particularly suitable for those who want to pursue research in this area)

Assignments: Take-home assignment 1 Take-home assignment 1 solution

The CA of this course consists of two take-home assignments. Solutions must be submitted in hardcopy by the deadlines stated on the take-home assignments. It is a strict deadline; late submissions will not be entertained (unless an extension is granted by me to the whole class). Do not plagiarise others' solutions. Students found copying or allowing others to copy their works will have their marks voided. More importantly, by copying, the student is short-changing himself or herself. While there might be short-term gain, an important learning component is missed. No student can get away by doing so, as the Final Exam contains questions that will certainly expose the student's deficiency.

The files you need for question 1d: pts.txt and pts_prime.txt, and for question 2: time-lapse video

The images you can use for question 4 in CA1 are: inria1.tif , inria2.tif , frc1.tif , frc2.tif, building1.jpg , building2.jpg. You are also given the following Matlab files for this question: displayEpipolarF.m , epipoles.m. These two files are zipped together with some tutorials on matlab here. Lastly, the display package is here.

Options: Those who want to actually implement some 3D Vision applications will be waived on some of the questions in my CA Assignments. Please contact me if you have any 3D vision projects in mind and need assistance in implementation. In going for this option, you should carefully consider the relevance of your project to the lectures and the difficulty involved. With my approval, the project can be related to the student's own FYP research. Look at the vast online resource listed below; they would be good places to look for potential projects or research ideas. Or you might want to consider the following suggestions from me: 1) to combine computer vision and computer graphics technologies to enhance interaction (e.g. the kind of visual effects you see in the film The Matrix). 3D Video and View morphing; 2) to apply computer vision technology to assist blind and visually impaired persons. Optical flow estimation for obstacle avoidance. (Unfortunately, the challenge involved in both these topics (especially the first one) is considerable so it is not for everyone.)

 


Samples of my research:

Much of these work are initiated when I was a graduate student (1992-1996) with my advisor, Yiannis Aloimonos at Maryland. See his Socratic Dialogue which describes the research at the difficulty level of a Scientific American article. Since then, my research programme revolves around various motion-related vision tasks such as motion segmentation, scene reconstruction from multiple views, tracking, action classification. See my recent research


About my other interest


Links of Interest


Major CV Journals:

  • IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI)
  • Computer Vision and Image Understanding (CVIU)
  • International Journal of Computer Vision (IJCV)
  • Image and Vision Computing (IVC)
  • IEEE Transactions on Image Processing (TIP)
  • Pattern Recognition (PR)
  • Machine Vision and Applications (MVA)
  • Major CV Conferences:

  • International Conference on Computer Vision (ICCV)
  • Computer Vision and Pattern Recognition (CVPR)
  • European Conference on Computer Vision (ECCV)
  • International Conference of Image Processing (ICIP)
  • International Conference of Pattern Recognition (ICPR)
  • Calendar of Computer Image Analysis, Computer Vision Conferences

  • Useful links to other resources

  • Geometry in Action
  • Applied Statistics

    Feature Extraction

    • VLFeat (SIFT, MSER, plus fast kmeans, hierarchical kmeans ?)
    • SIFT Demo by David Lowe (closed-source)
    • SURF (Speeded Up Robust Features, original implementation, closed-source)
    • OpenSURF (An opensource implementation of SURF)

    Machine Learning Algorithms

    • LIBSVM (A Library for Support Vector Machines)
    • SVMlight (Another popular implementation of SVM)
    • shogun (A large-scale ML toolbox, specialized in SVM. It provides unified interface for several popular SVM implementations, and features supports for Multiple Kernel Learning. )