ECE 5554: Computer Vision, Fall 2015

ECE 4984: Introduction to Computer Vision, Fall 2015

Electrical and Computer Engineering Department
Virginia Tech

Tuesdays and Thursdays (TR)
From 3:30 PM to 4:45 PM
In Goodwin Hall (GOODW) 135

Devi Parikh
Office: 440 Whittemore Hall
Office hours: Fridays 3:00 pm to 4:00 pm (starting 09/11/2015)

Jiasen Lu
Office: 264 Whittemore Hall
Office Hours: Mondays 10:00 am to 11:30 am
                      Wednesdays 1:30 pm to 3:00 pm

Announcements      Course overview      Requirements      Schedule       Class Projects       References


Welcome to ECE 5554!

Course overview:

"To see is to know what is where by looking"- David Marr

Over the past few decades, machines have come a long way in their ability to "see". Some examples are autonomous navigators like Mars Rover and Google car, medical imaging technologies, pedestrian detectors in vehicles, image search engines, face detectors in cameras and Facebook, aids for the visually impaired, control-free video games, and industrial automation systems.

In this introductory Computer Vision course, we will learn how to "teach machines to see". We will explore several fundamental concepts including image formation, feature detection, segmentation, multiple view geometry, recognition, and video processing. We will use these concepts to build applications that aid machines to see the world around them.


Problem Sets (55% of final grade): You will be given 6 problem sets, one approximately every two weeks. These will involve a combination of conceptual questions and programming problems. The programming problems will provide hands-on experience working with techniques covered in or related to the lectures. All code and written responses must be completed individually and submitted to Scholar. Most problem sets will take significant time to complete. Please start early. Problem Set 0 (PS0) will be worth 5% the final grade, and the remaining 5 problem sets will be worth 10% each.

Project (25% of final grade): Your project can be about applying any of the techniques we studied in class to real world problems. You can also extend a technique, or empirically analyze it. Comparisons between two approaches are also welcome. It is wonderful if you design and evaluate a novel approach to an important existing or new vision problem. Be creative! You must work in teams of two or more. The instructor may make exceptions. If you want to work on a project alone, come talk to the instructor first. The following are deliverables for your project. Only one team-member should submit each deliverable. That is, only one submission (for each deliverable) per team please.

Due Dates: All problem sets/reports are to be submitted by the due date noted on the assignment. Deadlines are firm. Anything from 1 minute to 24 hours is one day late.

Late Day policy: Throughout the term you have an allowance of four free late days for your submissions, meaning you can accrue up to four days in late submissions with no penalty. For example, you could turn in one assignment four days late, or two assignments/project proposal/project report webpage each one day late. Once you have used all your free late days, a late submission will not be accepted. Please plan ahead so you can spend your late days wisely. In particular, note that we expect you will find the earlier assignments easier than those later in the course. We will count a full additional day as having passed for submissions 1 minute to 24 hours late. Submissions that are one day late after all the late days are used up will receive only up to 50% credit. Submissions that are two days late after all the late days are used up will receive only up to 25% credit. After that, 0 credit.

Exams (15% of final grade): There will be an in-class final exam. The exam is closed everything, but one 8.5 x 11 cheat sheet is allowed (two sides).

Participation and attendance (5% of final grade): Participation in class and regular attendance is expected. If for whatever reason you are absent, it is your responsibility to find out what you missed that day. Note that attendance and participation do factor into the final grade.

Textbook: Computer Vision: Algorithms and Applications, by Rick Szeliski. An electronic copy is available free online here. Some background reading on object recognition is from Kristen Grauman and Bastian Leibe's short book on Visual Object Recognition.

Scholar: The scholar webpage for this course is here (

Schedule (tentative):

Day/Date Topic Readings and links Lectures Assignments, Exams, Deadlines
Tue 8/25/2015 Course Intro Sec 1.1-1.3 Intro [ppt] PS0 out

Thu 8/27/2015 Features and filters Sec 3.1.1-2, 3.2 Linear filters [ppt]
Mon 08/31/2015

PS0 due
Tue 9/1/2015
Sec 3.2.3, 4.2
Seam carving paper
Seam carving video
Gradients [ppt]

Thu 9/3/2015
Sec 3.3.2-4 Edges and binary image analysis [ppt] PS1 out
Tue 9/8/2015
Sec 10.5
Texture Synthesis
Texture [ppt]
Thu 9/10/2015
Sec 2.3.2
Foundations of Color, B. Wandell
Lotto Lab illusions
Color [ppt]

Tue 9/15/2015 Grouping and fitting Sec 5.2-5.4
k-means demo
Segmentation and clustering [ppt]
Wed 9/16/2015

PS1 due
Thu 9/17/2015
Sec 4.3.2
Hough Transform demo
Excerpt from Ballard & Brown
Hough transform [ppt]
Tue 9/22/2015
Sec 5.1.1 Deformable contours [ppt] PS2 out
Thu 9/24/2015
Sec 2.1.1, 2.1.2, 6.1.1 Alignment and 2D image transformations [ppt]

Tue 9/29/2015 Multiple views and motion Sec 3.6.1, 6.1.4 Homography and image warping [ppt]
Notes on homography matrix
Project proposals due
Thu 10/1/2015
Sec 4.1 Local invariant features 1 [ppt]
Mon 10/5/2015

PS2 due
Tue 10/6/2015
Sec 4.1 Local invariant features 2 [ppt] PS3 out
Thu 10/8/2015
Sec 11.1.1, 11.2-11.5 Image formation [ppt]
Tue 10/13/2015
Sec 11.1.1, 11.2-11.5
Epipolar geometry demo
Audio camera, O'Donovan et al.
Virtual viewpoint video, Zitnick et al.
Epipolar geometry and stereo [ppt]
Thu 10/15/2015
Tomasi Kanade 1992
Affine structure fom motion
Epipolar geometry and projective structure from motion

Structure from motion [ppt]

Mon 10/19/2015

PS3 due
Tue 10/20/2015 Recognition Grauman & Leibe Ch 1-4 (3 is review)
Grauman & Leibe Ch 5,6
Szeliski 14.3
Video Google demo by Sivic et al., paper
Indexing local features and instance recognition [ppt]
Thu 10/22/2015
Grauman & Leibe Ch 7,8.1,9.1,11.1
Szeliski 14.1
Intro to category recognition [ppt]
Tue 10/27/2015
Grauman & Leibe Ch 7,8.1,9.1,11.1
Szeliski 14.1
Viola-Jones face detection paper (for additional reference)
Face detection [ppt]
Thu 10/29/2015
Grauman & Leibe Ch 11.3 11.4
Szeliski 14.4
Discriminative classifiers for image recognition [ppt]
Tue 11/3/2015
Grauman & Leibe Ch 11.3 11.4
Szeliski 14.4 
Parts-based models [ppt] PS4 out
Thu 11/5/2015
Face recognition: A Literature Survey (pp 1-26)
Face recognition [ppt]

Tue 11/10/2015 Video processing 8.4,12.6.4 Motion and optical flow [ppt]
Thu 11/12/2015

No class
Mon 11/16/2015

PS4 due
Tue 11/17/2015
Davis & Bobick paper: The Representation and Recognition of Action Using Temporal Templates
Stauffer & Grimson paper: Adaptive Background Mixture Models for Real-Time Tracking.
Background subtraction, action recognition [ppt]
Thu 11/19/2015
5.1.2, 4.1.4 Tracking [ppt] PS5 out
Sat 11/21/2015

Project report due Project report due
Tue 11/24/2015

No class (Thanksgiving)
Thu 11/26/2015

No class (Thanksgiving)

Tue 12/1/2015

Guest lectures:
Convolutional Neural Networks (CNNs) [ppt]: Michael Cogswell
Learning and Using Common Sense [ppt]:
Xiao Lin

Wed 12/2/2015

PS5 due
Thu 12/3/2015

Poster Presentations
2:30 pm - 5:30 pm
GoodWin Hall Atrium
List of class projects
Tue 12/8/2015

Visual Question Answering (VQA) + SPOT Evaluations  
Thu 12/10/2015

Reading day (no class)
Tue 12/15/2015

Final Exam 10:05AM to 12:05PM  Final Exam

Class Projects:


This course closely follows the following course:

(Just a few) Other similar courses (by no means an exhaustive list):

Other code and data:

Tutorials, workshops, summer schools: