ECE 5554: Computer Vision, Fall 2015

ECE 4984: Introduction to Computer Vision, Fall 2015

Instructor: 
Devi Parikh
Email: parikh@vt.edu
Office: 440 Whittemore Hall
Office hours: Fridays 3:00 pm to 4:00 pm (starting 09/11/2015)

TA: 
Jiasen Lu
Email: jiasenlu@vt.edu
Office: 264 Whittemore Hall
Office Hours: Mondays 10:00 am to 11:30 am
                      Wednesdays 1:30 pm to 3:00 pm

Announcements      Course overview      Requirements      Schedule       Class Projects       References

Announcements:

Welcome to ECE 5554!

Course overview:

"To see is to know what is where by looking"- David Marr

Over the past few decades, machines have come a long way in their ability to "see". Some examples are autonomous navigators like Mars Rover and Google car, medical imaging technologies, pedestrian detectors in vehicles, image search engines, face detectors in cameras and Facebook, aids for the visually impaired, control-free video games, and industrial automation systems.

In this introductory Computer Vision course, we will learn how to "teach machines to see". We will explore several fundamental concepts including image formation, feature detection, segmentation, multiple view geometry, recognition, and video processing. We will use these concepts to build applications that aid machines to see the world around them.

Requirements:

Problem Sets (55% of final grade): You will be given 6 problem sets, one approximately every two weeks. These will involve a combination of conceptual questions and programming problems. The programming problems will provide hands-on experience working with techniques covered in or related to the lectures. All code and written responses must be completed individually and submitted to Scholar. Most problem sets will take significant time to complete. Please start early. Problem Set 0 (PS0) will be worth 5% the final grade, and the remaining 5 problem sets will be worth 10% each.

Project (25% of final grade): Your project can be about applying any of the techniques we studied in class to real world problems. You can also extend a technique, or empirically analyze it. Comparisons between two approaches are also welcome. It is wonderful if you design and evaluate a novel approach to an important existing or new vision problem. Be creative! You must work in teams of two or more. The instructor may make exceptions. If you want to work on a project alone, come talk to the instructor first. The following are deliverables for your project. Only one team-member should submit each deliverable. That is, only one submission (for each deliverable) per team please.

• Proposal (10% of project grade): A 1-page description that describes the following:
• Problem statement: Clearly state the goal of your project. When someone uses your system, what is the expected input to the system, and what is the desired output?
• Approach: Describe the technical approach you plan to employ. 
• Experiments and results: Describe the experimental setup you will follow, which datasets you will use, which existing code you will exploit, what you will implement yourself, and what you would define as a success for the project. If you plan on collecting your own data, describe what data collection protocol you will follow. Provide a list of experiments you will perform. Describe what you expect the experiments to reveal, or what is uncertain about the potential outcomes.
• Project Report Webpage (50% of project grade): We expect you to have completed your project by this time. The only thing we expect between this and the final poster presentation is incorporating feedback from the instructor and/or TA. The report should be in the form of a webpage. It should be a nice, professional looking, visual, self-contained webpage describing your project. We will link to all project pages from the class webpage. Any student in the class should be able to clearly understand your report such that he/she can implement it.Below is an outline of what we expect the webpage to cover:
  
• Abstract: One or two sentences on the motivation behind the problem you are solving. One or two sentences describing the approach you took. One or two sentences on the main result you obtained.
• Teaser figure: A figure that conveys the main idea behind the project or the main application being addressed.
  
• Introduction: Motivation behind the problem you are solving, what applications it has, any brief background on the particular domain you are working in (if not regular RBG photographs), etc. If you are using a new way to solve an existing problem, briefly mention and describe the existing approaches and tell us how your approach is new.
• Approach: Describe very clearly and systematically your approach to solve the problem. Tell us exactly what existing implementations you used to build your system. Tell us what obstacles you faced and how you addressed them. Justify any design choices or judgment calls you made in your approach.
• Experiments and results: Provide details about the experimental set up (number of images/videos, number of datasets you experimented with, train/test split if you used machine learning algorithms, etc.). Describe the evaluation metrics you used to evaluate how well your approach is working. Include clear figures and tables, as well as illustrative qualitative examples if appropriate. Be sure to include obvious baselines to see if your approach is doing better than a naive approach (e.g. for classification accuracy, how well would a classifier do that made random decisions?). Also discuss any parameters of your algorithms, and tell us how you set the values of those parameters. You can also show us how the performance varies as you change those parameter values. Be sure to discuss any trends you see in your results, and explain why these trends make sense. Are the results as expected? Why?
• Qualitative results: Show several visual examples of inputs/outputs of your system (success cases and failures) that help us better understand your approach.
  
• Conclusion and future work: Conclusion would likely make the same points as the abstract. Discuss any future ideas you have to make your approach better.
• References: List out all the references you have used for your work.
• Examples:
• See this for a webpage template.
• See this for an example of a nice, professional looking page.
• See this for an example of how to lay out the various details of your project. You may need to provide more details than this, beause you will not be submitting an associated paper to accompany the webpage. So the page should be self-contained.
  
• Poster Presentation (40% of project grade): A poster presentation that describes the same points as the proposal and the project report webpage. A sample poster template can be downloaded from here. Describe the results you have obtained. This may involve some additional experiments, etc. based on the instructor's and/or TA's feedback on your report webpage. Live demos when applicable are highly encouraged. Prepare a 1 minute schpeel that presents the main summary of your work. The instructor and TA will ask further questions as necessary.

Due Dates: All problem sets/reports are to be submitted by the due date noted on the assignment. Deadlines are firm. Anything from 1 minute to 24 hours is one day late.

Late Day policy: Throughout the term you have an allowance of four free late days for your submissions, meaning you can accrue up to four days in late submissions with no penalty. For example, you could turn in one assignment four days late, or two assignments/project proposal/project report webpage each one day late. Once you have used all your free late days, a late submission will not be accepted. Please plan ahead so you can spend your late days wisely. In particular, note that we expect you will find the earlier assignments easier than those later in the course. We will count a full additional day as having passed for submissions 1 minute to 24 hours late. Submissions that are one day late after all the late days are used up will receive only up to 50% credit. Submissions that are two days late after all the late days are used up will receive only up to 25% credit. After that, 0 credit.

Exams (15% of final grade): There will be an in-class final exam. The exam is closed everything, but one 8.5 x 11 cheat sheet is allowed (two sides).

Participation and attendance (5% of final grade): Participation in class and regular attendance is expected. If for whatever reason you are absent, it is your responsibility to find out what you missed that day. Note that attendance and participation do factor into the final grade.

Textbook: Computer Vision: Algorithms and Applications, by Rick Szeliski. An electronic copy is available free online here. Some background reading on object recognition is from Kristen Grauman and Bastian Leibe's short book on Visual Object Recognition.

Scholar: The scholar webpage for this course is here (https://scholar.vt.edu/portal/site/vision_fall2015).

• All problem sets, proposals and links to project report pages must be submitted in the Assignments section. No hard copies are accepted.
• Any questions relating to the course must be first raised in the Forums section, before reaching out to the Instructor/TA.
• Lectures and problem sets will be available on this webpage (not on scholar).
  

Schedule (tentative):

Class Projects:

• Counter-Evidence Discovery for Object Detection
• Comparative Study of Different Techniques for Facial Expression Recognition
• Tracking of Fiducial Markers for Bat Flight Motion Capture Experiments
• People Counter
• Controlling and navigating a quadrotor indoors using Vanishing Point Algorithm.
• Face Recognition via Convolutional Neural Network
• MotionTrack
• Robotic Vision Toolkit for Industrial Applications
• Overlapping Division in Object Tracking
• Detection of airborne objects via Hough transform
• Vehicle-Pedestrian Monitoring
• Exploring Nearest Neighbor Approach on VQA
• Hand Tool Recognition and Classification
• Tap-to-Jump Game Solver using Computer Vision
• Counting Everyday Objects in Everyday scenes
• Real Time Finger Recoginition
• Cataract Surgery Tool Detection
• Linear Infrastructure Detection for Real-Time UAV Control
• Robotic Path Following
• Discovering Image Clusters and Deep Representations
• Runway Boundary detection for safe Landing Assistance
• Mirror Worlds
• Facial Recognition Project
• Markov Chained Active Contour Model for semi-automatic segmentation of images
• Object Prediction using Image Context
• Shadow Detection and Removal from RGB Images
• Estimating GPS coordinates from an image
• Role of Attention for Visual Question Answering
• Shadow Detection and Removal
• Actively-learning Real-time Hand Gesture Recognition System for Personalized Static Gestures
• Visual Question Answering with Various Feature Combinations
• Identifying and counting cells
• Visual Interface Project
• Road Lanes Detection

References:

This course closely follows the following course:

• CS 376: Computer Vision at UT Austin taught by Kristen Grauman

(Just a few) Other similar courses (by no means an exhaustive list):

• CSE P 576: Computer Vision at University of Washington taught by Larry Zitnick
• CS 543 / ECE 549 Computer Vision at UIUC taught by Derek Hoiem (Video lectures)
• CS 143 Introduction to Computer Vision at Brown University taught by James Hays

Other code and data:

• Getting started with Matlab
• An introduction to Numpy and Scipy
• Python Numpy Tutorial
• Matplotlib tutorial
• Visual Object Recognition synthesis lecture by Grauman and Leibe (short book on object recognition methods)
• Compiled list of recognition datasets
  
• OpenCV (open source computer vision library)
• SimpleCV (open source computer vision platform using Python)
  
• Weka (Java data mining software)
• Netlab (Matlab toolbox for data analysis techniques, written by Ian Nabney and Christopher Bishop)
• CV Online
• Annotated Computer Vision Bibliography
• Oxford group interest point software
• Andrea Vedaldi's VLFeat code, including SIFT, MSER, hierarchical k-means.
• INRIA LEAR team's software, including interest points, shape features
• FLANN - Fast Library for Approximate Nearest Neighbors.  Marius Muja et al. 
• Google Goggles
• Kooaba
• LSH homepage
• Code for downloading Flickr images, by James Hays
• UW Community Photo Collections homepage
• INRIA Holiday images dataset
• NUS-WIDE tagged image dataset of 269K images
• MIRFlickr dataset
• LIBPMK feature extraction code, includes dense sampling
• LIBSVM library for support vector machines
• PASCAL VOC Visual Object Classes Challenge
  
• Fast SLIC superpixels
  
• Greg Mori's superpixel code
• Berkeley Segmentation Dataset and code
• Pedro Felzenszwalb's graph-based segmentation code
• Mean-shift: a Robust Approach Towards Feature Space Analysis [pdf]  [ code, Matlab interface by Shai Bagon]
• David Blei's Topic modeling code
• Berkeley 3D object dataset (kinect)
• Labelme Database 
• Stanford Event Dataset
  
• SUN Scene and object dataset
• ImageNet dataset of 15K objects and ImageNet challenge
• Animals with Attributes dataset
• aYahoo and aPascal attributes datasets
• Attribute discovery dataset of shopping categories
• Public Figures Face database with attributes
• Relative attributes data
• WhittleSearch relative attributes data
• SUN Scenes attribute dataset
• Cross-category object recognition (CORE) dataset
• Leeds Butterfly Dataset
• FaceTracer database from Columbia
• Caltech-UCSD Birds dataset
• Database of human attributes
  
• Face detection code in OpenCV
• Gallagher's Person Dataset
• Face data from Buffy episode, from Oxford Visual Geometry Group
• CALVIN upper-body detector code
• UMass Labeled Faces in the Wild
• Ivan Laptev's Space-Time Interest Points code
• Hollywood activity dataset
• Stanford 40 Actions still image dataset
• Stanford People Playing Musical Instrument dataset
  
• UCF activity datasets
• PASCAL VOC action recognition taster challenge
• TRECVID video retrieval challenge
• UMich Collective Activity dataset
• Egovision workshop at CVPR 2012
• Amazon Mechanical Turk
• Using Mechanical Turk with LabelMe
• Point Cloud Library
• Robot Operating System
• KITTI Benchmark
  

Tutorials, workshops, summer schools:

• Frontiers in Computer Vision Workshop
• Scene Understanding Symposium SUnS
• Como Workshop on Category-level Object Recognition(2008)
• IMA Visual Learning and Recognition Workshop(2006)
• MSRI Visual Recognition Workshop(2006)
• ENS/INRIA Visual Recognition and Machine Learning Summer School
• International Computer Vision Summer School
• Vision and Sports Summer School
• Computer vision conferences
• ICCV 2005 / CVPR 2007 / ICCV 2009 Short Course on Recognition
• AAAI 2008 Tutorial on Recognition
• ICML 2008 Tutorial on Recognition
• Greg Mori and Ivan Laptev's tutorial on action recognition at ECCV 2010
• CVPR 2009 Workshop on Visual Place Categorization