ECE 5554: Computer Vision, Fall 2013

Electrical and Computer Engineering Department
Virginia Tech

Tuesdays and Thursdays (TR)
From 3:30 PM to 4:45 PM
In McBryde Hall (MCB) 307

Devi Parikh
Office: 440 Whittemore Hall
Office hours: Fridays from 2:30 PM to 3:30 PM

Neelima Chavali
Office Hours: Mondays and Wednesdays from 11:00 AM to 12:00 @ WHIT 215

Announcements      Course overview      Requirements      Schedule      References


*** Best Project Prizes ***

Computer Vision Applications for UAV Control, Mapping and Road Recognition (Matt Frauenthal and Scott Radford)
Optical Character Recognition for Math Equations (Allan Kirchhoff)

Welcome to ECE 5554!

Course overview:

"To see is to know what is where by looking"- David Marr

Over the past few decades, machines have come a long way in their ability to "see". Some examples are autonomous navigators like Mars Rover and Google car, medical imaging technologies, pedestrian detectors in vehicles, image search engines, face detectors in cameras and Facebook, aids for the visually impaired, control-free video games, and industrial automation systems.

In this introductory Computer Vision course, we will learn how to "teach machines to see". We will explore several fundamental concepts including image formation, feature detection, segmentation, multiple view geometry, recognition, and video processing. We will use these concepts to build applications that aid machines to see the world around them.


Problem Sets (55% of final grade): You will be given 6 problem sets, one approximately every two weeks. These will involve a combination of conceptual questions and programming problems. The programming problems will provide hands-on experience working with techniques covered in or related to the lectures. All code and written responses must be completed individually and submitted to Scholar. Most problem sets will take significant time to complete. Please start early. Problem Set 0 (PS0) will be worth 5% the final grade, and the remaining 5 problem sets will be worth 10% each.

Project (25% of final grade): Your project can be about applying any of the techniques we studied in class to real world problems. You can also extend a technique, or empirically analyze it. Comparisons between two approaches are also welcome. It is wonderful if you design and evaluate a novel approach to an important existing or new vision problem. Be creative! You are welcome to work with a partner. If you need help with ideas for your project please come talk to the instructor and/or TA. The following are deliverables for your project:

Due Dates: All problem sets/reports are to be submitted by the due date noted on the assignment. Deadlines are firm. Anything from 1 minute to 24 hours is one day late

Late Day policy: Throughout the term you have an allowance of four free late days for your submissions, meaning you can accrue up to four days in late submissions with no penalty. For example, you could turn in one assignment four days late, or two assignments/project proposal/project report each one day late. Once you have used all your free late days, a late submission will not be accepted. Please plan ahead so you can spend your late days wisely. In particular, note that we expect you will find the earlier assignments easier than those later in the course. We will count a full additional day as having passed for submissions 1 minute to 24 hours late. No submissions will be accepted after all the late days are used up.

Exams (15% of final grade): There will be an in-class final exam.

Participation and attendance (5% of final grade): Participation in class and regular attendance is expected. If for whatever reason you are absent, it is your responsibility to find out what you missed that day. Note that attendance and participation do factor into the final grade.

Textbook: Computer Vision: Algorithms and Applications, by Rick Szeliski. An electronic copy is available free online here. Some background reading on object recognition is from Kristen Grauman and Bastian Leibe's short book on Visual Object Recognition.

Scholar: The scholar webpage for this course is here.

Schedule (tentative):

Day/Date Topic Readings and links Lectures Assignments, Exams, Deadlines
Tue 8/27/2013 Course Intro Sec 1.1-1.3 Intro [ppt] PS0 out
Thu 8/29/2013 Features and filters Sec 3.1.1-2, 3.2 Linear filters [ppt]
Mon 9/2/2013 PS0 due
Tue 9/3/2013 Sec 3.2.3, 4.2
Seam carving paper
Seam carving video
Gradients [ppt]
Thu 9/5/2013 Sec 3.3.2-4 Edges and binary image analysis [ppt] PS1 out. File check script
Tue 9/10/2013 Sec 10.5
Texture Synthesis
Texture [ppt]
Thu 9/12/2013 Sec 2.3.2
Foundations of Color, B. Wandell
Lotto Lab illusions
Color [ppt]
Tue 9/17/2013 Grouping and fitting Sec 5.2-5.4
k-means demo
Segmentation and clustering [ppt]
Wed 9/18/2013 PS1 due
Thu 9/19/2013 Sec 4.3.2
Hough Transform demo
Excerpt from Ballard & Brown
Hough transform [ppt]
Tue 9/24/2013 Sec 5.1.1 Deformable contours [ppt] PS2 out. File check script
Thu 9/26/2013 Sec 2.1.1, 2.1.2, 6.1.1 Alignment and 2D image transformations [ppt]
Tue 10/1/2013 Multiple views and motion Sec 3.6.1, 6.1.4 Homography and image warping [ppt]. Notes on homography matrix Project proposals due
Thu 10/3/2013 Sec 4.1 Local invariant features 1 [ppt]
Mon 10/7/2013 PS2 due
Tue 10/8/2013 Sec 4.1 Local invariant features 2 [ppt] PS3 out
Thu 10/10/2013 Sec 11.1.1, 11.2-11.5 Image formation [ppt]
Tue 10/15/2013 Sec 11.1.1, 11.2-11.5
Epipolar geometry demo
Audio camera, O'Donovan et al.
Virtual viewpoint video, Zitnick et al.
Epipolar geometry [ppt]
Thu 10/17/2013 Tomasi Kanade 1992
Affine structure fom motion
Epipolar geometry and projective structure from motion

Stereo and Structure from motion [ppt]
Mon 10/21/2013 PS3 due
Tue 10/22/2013 Recognition Grauman & Leibe Ch 1-4 (3 is review)
Grauman & Leibe Ch 5,6
Szeliski 14.3
Video Google demo by Sivic et al., paper
Indexing local features and instance recognition [ppt]
Thu 10/24/2013 Grauman & Leibe Ch 7,8.1,9.1,11.1
Szeliski 14.1
Intro to category recognition [ppt]
Tue 10/29/2013 Grauman & Leibe Ch 7,8.1,9.1,11.1
Szeliski 14.1
Viola-Jones face detection paper (for additional reference)
Face detection [ppt]
Thu 10/31/2013 Grauman & Leibe Ch 11.3 11.4
Szeliski 14.4
Discriminative classifiers for image recognition [ppt]
Tue 11/05/2013 Grauman & Leibe Ch 11.3 11.4
Szeliski 14.4 
Parts-based models [ppt] PS4 out
Thu 11/07/2013 Eigenfaces
Face recognition: A Literature Survey (pp 1-26)
Face recognition [ppt]
Tue 11/12/2013 Video processing 8.4,12.6.4 Motion and optical flow [ppt]
Thu 11/14/2013 8.4,12.6.4
Davis & Bobick paper: The Representation and Recognition of Action Using Temporal Templates
Stauffer & Grimson paper: Adaptive Background Mixture Models for Real-Time Tracking.
Bacground substraction, action recognition [ppt]
Mon 11/18/2013 PS4 due
Tue 11/19/2013 5.1.2, 4.1.4 Tracking [ppt] PS5 out
Thu 11/21/2013 Summary and discussion [ppt]
Sat 11/23/2013 Project report due Project report due
Tue 11/26/2013 No class (Thanksgiving)
Thu 11/28/2013 No class (Thanksgiving)
Tue 12/3/2013 Guest Lectures
Wed 12/04/2013 PS5 due
Thu 12/5/2013 Guest Lectures
Tue 12/10/2013 Poster Presentations @ WHIT 236 from 3PM to 6PM Poster Presentations @ WHIT 236 from 3PM to 6PM
Thu 12/12/2013 Reading day (no class)
Fri 12/13/2013 Final Exam 10:05AM to 12:05PM  Final Exam


This course closely follows the following course:

(Just a few) Other similar courses (by no means an exhaustive list):

Other code and data:

Tutorials, workshops, summer schools: