CSCI3330: Fundamentals of Applied Computer Vision (Spring 2024)

Course Description

Computer vision is the enterprise of building machines that can sense and extract information from 3D scenes. It is one of the fastest growing and exciting disciplines in today’s academia and industry, with a wide variety of applications in robotics, video surveillance, navigation, consumer electronics, human-computer interaction, medical imaging, remote sensing, space exploration, and so on. This course is designed to open the doors for students who are interested in learning about the fundamental principles and important applications of computer vision. The following topics will be covered: cameras and image formation, image filtering, edge and corner detection, image features and matching, image alignment, geometric camera models, binocular stereo, optical flow, radiometry, photometric stereo, structured light, and other applications in machine vision.

Course Information

Lectures:

  • Mondays 4:30pm-6:15pm LHC 104

  • Tuesdays 4:30pm-5:15pm LSB LT5

Tutorials:

  • Tuesdays 2:30pm-3:15pm ERB 401

  • Tuesdays 5:30pm-6:15pm LSB LT5

Course Staff

Instructor TA1 TA2 TA3
Contact Prof.Jinwei GU Lingen Li Yongrui MA Yitong JIANG
Office Hour Mon 2:30pm-4:30pm Fri 2:00pm-4:00pm Wed 9:00am-11:00am Mon 9:00am-11:00am
Office SHB 1007 SHB 1013 SHB 1013 SHB 904

Course notes and assignments are available at Blackboard http://blackboard.cuhk.edu.hk

Reference Books

  • (SZ) Computer Vision: Algorithms and Applications, by Richard Szeliski. Free online

  • (FP) Computer Vision: A Modern Approach, by David Forsyth and Jean Ponce. Free online

  • (BH) Robot Vision, by Berthold K.P. Horn.

  • First Principles of Computer Vision, by Shree Nayar. Free online

Tentative Schedule

Week Date Lecture Reading HW & Exam
1 Jan 8 Course Introduction and Overview SZ Ch.1, Ch.2.3;
FP Ch.1.1; BH Ch.1, Ch.2
Jan 9 Camera Model, Image Formation
2 Jan 15 Image Sensing SZ Ch.3.1, Ch.3.2;
FP Ch.4.1, Ch.4.2, Ch.22
Jan 16 Image Filtering (1) HW1 out
3 Jan 22 Image Filtering (2) SZ Ch.3.3, Ch.3.4;
FP Ch.4.3, Ch.4.4;
BH Ch.6.7
Jan 23 Camera ISP Pipeline
4 Jan 29 Noise and HDR Imaging SZ Ch.7.1; FP Ch.5
Jan 30 Edge and Corner Detection
5 Feb 5 Line and Boundary Detection SZ Ch.7.1; FP Ch.5 (HW1 due Feb 4). HW2 out
Feb 6 Image Features and Matching
Spring Festival Holidays
6 Feb 19 Image Alignment and Stitching SZ Ch.7.4, Ch.7.5;
FP Ch.9.1, Ch.9.3,
Ch.10.1, Ch.10.2
Feb 20 Image Segmentation
7 Feb 26 Image Classification SZ Ch.2.1, Ch.11.1;
FP Ch.1.1, Ch.1.2, Ch.1.3
(HW2 due Feb 25). HW3 out
Feb 27 Midterm Review Session
Reading Week
8 Mar 11 Midterm SZ Ch.8.1, Ch.8.2; FP Ch.7.1 Midterm Exam
Mar 12 Geometric Camera Model
9 Mar 18 Epipolar Geometry SZ Ch.12.1, Ch.12.3, Ch.12.5;
FP Ch.7.2, Ch.7.4, Ch.7.5;
BH Ch.13
(HW3 due Mar 17). HW4 out
Mar 19 Binocular Stereo
10 Mar 25 Motion and Optical Flow SZ Ch.9.1, Ch.9.3; BH Ch.12
Mar 26 Structure from Motion
11 Apr 1 No class (Holiday) SZ Ch.2.2; FP Ch.2.1, Ch.2.2;
BH Ch.9
(HW4 due Mar 31). HW5 out
Apr 2 Radiometry
12 Apr 8 Photometric Stereo SZ Ch.13.1; FP Ch.2.4;
BH Ch.10, Ch.11
Apr 9 Structured Light for 3D Capture
13 Apr 15 Neural Networks for Computer Vision (HW5 due Apr 22)
Apr 16 Final Review Session
14 Final Exam (TBD)

Tutorial Sessions

In addition to regular lectures, this class will also have weekly tutorial sessions. The tutorial sessions will be used mostly to review and clarify the programming assignments, introduce supplement materials of the lectures, and answer questions from students.

Course Grades and Assignment Policy

The final grade is based on five assignments, one mid-term exam, and one final exam. Each assignment is 12%, the midterm exam is 15%, and the final exam is 25%. For some assignments, there are also optional bonus points (up to 20% of its total grade). Each assignment will have the written part and the programming part (with Python Notebook).

All assignments are due on 11:59pm (HK Time) on the due date. No late submission is allowed, and zero mark will be given in that case. All the assignments will be submitted via Blackboard. More detailed instructions for submission will be provided along with each assignment.

Students are welcome to discuss their partial solutions and questions with course staff members during the tutorial sessions, in office hours, or via Blackboard forum. Students are also permitted to discuss common concerns with classmates, but these discussions must be kept at a general level, without exposing their solutions or source code.

Finally, please do not publish any questions or solutions of the assignments and the exams, e.g., Github or a publicly accessible web page. This is a violation of the basic Rights, Rules, Responsibilities of members of the University community.

Academic Honesty

Plagiarism, including copying any parts of code or solutions from your classmates, or releasing your code for others to copy, or copying during the exams, is strictly prohibited and will be treated very seriously. If found responsible, the typical penalty is an F as a course grade plus whatever penalty that the university imposes. If you have any doubts, please read the academic honesty guidelines from the university(https://www.cuhk.edu.hk/policy/academichonesty/) and the academic honesty guidelines from the Faulty of Engineering (https://www.erg.cuhk.edu.hk/erg/sites/default/files/Guidelines_to_Academic_Honesty.pdf), or ask the instructor.

Acknowledgement

Some of the materials in the lecture slides of this course is built based on similar courses taught by the following professors:

  • Prof. Shree Nayar (Columbia)

  • Prof. Matthew O'Toole (CMU)

  • Prof. Noah Snavely (Cornell)

  • Prof. James Tompkin (Brown)

  • Prof. Michael Brown (York)

  • Prof. Gordon Wezstein (Stanford)