CSCI3330: Fundamentals of Applied Computer Vision (Spring 2024)

Course Description

Computer vision is the enterprise of building machines that can sense and extract information from 3D scenes. It is one of the fastest growing and exciting disciplines in today’s academia and industry, with a wide variety of applications in robotics, video surveillance, navigation, consumer electronics, human-computer interaction, medical imaging, remote sensing, space exploration, and so on. This course is designed to open the doors for students who are interested in learning about the fundamental principles and important applications of computer vision. The following topics will be covered: cameras and image formation, image filtering, edge and corner detection, image features and matching, image alignment, geometric camera models, binocular stereo, optical flow, radiometry, photometric stereo, structured light, and other applications in machine vision.

Course Information

Lectures:

  • Mondays 4:30pm-6:15pm LHC 104

  • Tuesdays 4:30pm-5:15pm LSB LT5

Tutorials:

  • Tuesdays 2:30pm-3:15pm ERB 401

  • Tuesdays 5:30pm-6:15pm LSB LT5

Instructor: Prof. Jinwei Gu

TA(s): TBA

Course notes and assignments are available at Blackboard http://blackboard.cuhk.edu.hk

Reference Books

  • (SZ) Computer Vision: Algorithms and Applications, by Richard Szeliski. Free online

  • (FP) Computer Vision: A Modern Approach, by David Forsyth and Jean Ponce. Free online

  • (BH) Robot Vision, by Berthold K.P. Horn.

  • First Principles of Computer Vision, by Shree Nayar. Free online

Tentative Schedule

Week Date Lecture Reading HW & Exam
1 Jan 8 Course Introduction and Overview SZ Ch.1, Ch.2.3;
FP Ch.1.1; BH Ch.1, Ch.2
Jan 9 Camera Model, Image Formation
2 Jan 15 Image Sensing SZ Ch.3.1, Ch.3.2;
FP Ch.4.1, Ch.4.2, Ch.22
Jan 16 Image Filtering (1)
3 Jan 22 Image Filtering (2) SZ Ch.3.3, Ch.3.4;
FP Ch.4.3, Ch.4.4;
BH Ch.6.7
HW1 out
Jan 23 Camera ISP Pipeline
4 Jan 29 Noise and HDR Imaging SZ Ch.7.1; FP Ch.5
Jan 30 Edge and Corner Detection
5 Feb 5 Line and Boundary Detection SZ Ch.7.1; FP Ch.5
Feb 6 Image Features and Matching
Spring Festival Holidays
6 Feb 19 Image Stitching SZ Ch.7.4, Ch.7.5;
FP Ch.9.1, Ch.9.3,
Ch.10.1, Ch.10.2
HW2 out
Feb 20 Image Segmentation
7 Feb 26 Face Detection and Recognition SZ Ch.2.1, Ch.11.1;
FP Ch.1.1, Ch.1.2, Ch.1.3
Feb 27 Review Session
Reading Week
8 Mar 11 Midterm SZ Ch.8.1, Ch.8.2; FP Ch.7.1 Midterm
Mar 12 Geometric Camera Model
9 Mar 18 Epipolar Geometry SZ Ch.12.1, Ch.12.3, Ch.12.5;
FP Ch.7.2, Ch.7.4, Ch.7.5;
BH Ch.13
Mar 19 Binocular Stereo HW3 Out
10 Mar 25 Motion and Optical Flow SZ Ch.9.1, Ch.9.3; BH Ch.12
Mar 26 Structure from Motion
11 Apr 1 No class (Holiday) SZ Ch.2.2; FP Ch.2.1, Ch.2.2;
BH Ch.9
Apr 2 Radiometry
12 Apr 8 Photometric Stereo SZ Ch.13.1; FP Ch.2.4;
BH Ch.10, Ch.11
HW4 out
Apr 9 Structured Light for 3D Capture
13 Apr 15 Neural Networks for Computer Vision
Apr 16 Review Session

Tutorial Sessions

In addition to regular lectures, this class will also have weekly tutorial sessions. The tutorial sessions will be used mostly to review and clarify the programming assignments, introduce supplement materials of the lectures, and answer questions from students.

Course Grades and Assignment Policy

The final grade is based on four programming assignments, one mid-term exam, and one final exam. Each assignment is 15%, the midterm exam is 20%, and the final exam is 20%. For each assignment, there are also optional bonus points (up to 20% of its total grade).

All assignments are due on 11:59pm (HK Time) on the due date. In total there are 4 late days to handle unexpected circumstances (e.g., sickness, personal crisis, family problems). If you use up the 4 late days, we will allow late submissions for up to 24 hours, with a 20% point penalty. No late submissions after 24 hours are allowed, and zero marks will be given in that case. All the assignments will be submitted via Blackboard. More detailed instructions for submission will be provided along with each assignment.

Students are welcome to discuss their partial solutions and questions with course staff members during the tutorial sessions, in office hours, or via Blackboard forum. Students are also permitted to discuss common concerns with classmates, but these discussions must be kept at a general level, without exposing their solutions or source code.

Finally, please do not publish any questions or solutions of the assignments and the exams, e.g., Github or a publicly accessible web page. This is a violation of the basic Rights, Rules, Responsibilities of members of the University community.

Academic Honesty

Plagiarism, including copying any parts of code or solutions from your classmates, or releasing your code for others to copy, or copying during the exams, is strictly prohibited and will be treated very seriously. If found responsible, the typical penalty is an F as a course grade plus whatever penalty that the university imposes. If you have any doubts, please read the academic honesty guidelines from the university(https://www.cuhk.edu.hk/policy/academichonesty/) and the academic honesty guidelines from the Faulty of Engineering (https://www.erg.cuhk.edu.hk/erg/sites/default/files/Guidelines_to_Academic_Honesty.pdf), or ask the instructor.

Acknowledgement

Some of the materials in the lecture slides of this course is built based on similar courses taught by the following professors:

  • Prof. Shree Nayar (Columbia)

  • Prof. Matthew O'Toole (CMU)

  • Prof. Noah Snavely (Cornell)