Introduction to Machine Learning and Perception

Virginia Tech, Electrical and Computer Engineering
Spring 2013: ECE 4984 / ECE 5984

Course Information


Have you ever wondered how Siri understands voice commands? How Netflix recommends movies to watch? How Kinnect recognizes full-body gestures? How Watson defeated Jeopardy champions? The answer is Machine Learning -- the study of algorithms that learn from large quantities of data, identify patterns and make predictions on new instances.

Instructor
Dhruv Batra
Fri 1-2pm, Whittemore 468
TA
Sherin Aly
Mon 1:15-2:15pm, Durham 377

Class meets

Mon, Wed 2:30 - 3:45, MCB 307
Homeworks
Q&A and Discussions
https://scholar.vt.edu/portal/site/ece4984ece5984
ECE4984_ECE5984 Forum


Notable Mentions

[Updated Aug 1, 2013] Here are some phenomenal achievements by the students who took this class:

  • Marc Lichtman's class project accepted for publication at MILCOM.

    Marc Lichtman, William Headley, Jeffrey Reed.
    Automatic Modulation Classification under IQ Imbalance using Supervised Learning.
    In Proceedings of MILCOM 2013 (Track 1: Waveforms and Signal Processing).

  • Delasa Aghamirzaie's class project accepted for publication at ICCCB.

    D.Aghamirzaie, D. Batra, E. Collakova, L. Heath, R. Grene.
    Modeling and Identifying Regulatory Modules in (Glycine max) Soybean
    Time Series Gene Expression Data using Bayesian Networks.
    International Conference on Computational Cell Biology, 2013.

  • Gordon Christie and Ujwal Krothapalli created a demo video for their class project.



  • Student poster/demo session featured in ECE News.
  • If you took this class and have something to share, please email me!


Best Project Prize Winners


  • Winning Team:

    Ujwal Krothapalli and Gordon Christie.
    Gesture Activated Interactive Assistant

  • Runners-up:

    SaiDhiraj Amuru and Vireshwar Kumar.
    Enabling Hands to Speak American Sign Language Detection.

    Talia Weiss
    Predicting Ratings in Fanfiction.


Schedule

Week/Day
Date Topic
1 W
Jan
23
Overview of Machine Learning & Perception
HW0 is out
2 M
Jan
28
Statistical Learning
  • Probability Review
  • Statistical Estimatation (MLE,Map,Bayesian)

2 W
Jan
30
  • MAP + Bayesian Estimation
  • Gaussians
  • Regression
HW1 is out
3 M
Feb
4
Supervised Learning
  • Supervised learning setup. Linear Models
    • Regression
  • Error Decomposition
    • Bias-Variance Tradeoff
    • Cross-validation

3 W
Feb
6
  • Error Decomposition (Bias-Variance Tradeoff)
  • Cross-validation
  • Crash Course on Optimization
HW1 due Feb 8
4 M
Feb
11
  • (Finish) Crash Course of Optimization
  • Classification: Naive Bayes
HW2 is out
4 W
Feb
13
  • Classification: (Finish) Naive Bayes
  • Classification: Logistic Regression

5 M
Feb
18
  • Project Details
  • Classification: (Finish) Logistic Regression
  • NB & LR connections

5 W
Feb
20
  • Support Vector Machines
    • Linear SVMs
    • Hard & soft-margin SVMs
HW2 due Feb 22
6 M
Feb
25
  • Support Vector Machines
    • Kernel Trick
    • SVM dual
    • Multi-class SVM

6 W
Feb
27
  • Decision Trees
    • Mutual Information

7 M
Mar
4
In class Mid-Term
Mid-Term
7 W
Mar
6
  • Neural Networks
    • Perceptron
    • Multilayer Perceptron
    • Backprop
Project Proposal Due
8 W
Mar
11
Spring break. No class.

8 M
Mar
13
Spring break. No class.

9 M
Mar
18
  • (Finish) Neural Networks
    • Backprop
  • Instance-based Learning
    • 1/k Nearest Neighbour
    • Kernel regression/classification

9 W
Mar
20
  • (Finish) Nearest Neighbour
  • Ensemble Methods
    • Bagging
    • Boosting

10 M
Mar
25
  • (Finish) Ensemble Methods
    • Bagging
    • Boosting
HW3 is out

10 W
Mar
27
Unsupervised & Semi-Supervised Learning
  • Clustering
    • K-Means
    • Gaussian Mixture Models

11 M

Apr
1
Unsupervised & Semi-Supervised Learning (Continued)
    • Gaussian Mixture Models

11 W
Apr
3 Unsupervised & Semi-supervised Learning (Continued)
    • EM

12 M
Apr
8
  • Factor Analysis
    • PCA
    • SVD
    • Dimensionality Reduction
HW3 due
12 W
Apr
10
  • (Finish) PCA
  • LDA
  • (Intro) Probablistic Graphical Models
project midterm report due
13 M
Apr
15
In-class Project Presentations
13 W
Apr
17
In-class Project Presentations
14 M
Apr
22
(Finish) In-class Project Presentations
  • Bayes Nets: Representation
HW4 is out
14 W
Apr
24
  • Bayes Nets: Representation
    • conditional independence
    • d-separation

15 M
Apr
29
  • (Finish) Bayes Nets: Representation
  • Bayes Nets
    • learning parameters via MLE
  • Bayes Nets: Inference
    • Variable elimination

15 W
Mar
1
  • Hidden Markov Models (HMMs)
    • Representation
    • Learning from fully observed data: N-grams
    • Inference with VE: forward-backward algorithm
HW4 due
16 M
Mar
6
  • (Finish) Markov Nets / Markov Random Fields
    • Representation
    • Inference
Overview of Advanced Topics
  • PAC-Learning
    • VC dimension
  • Structured-SVMs
  • Reinforcement Learning
    • Markov Decision Processes
  • Online Learning
  • Active Learning
  • Semi-Supervised Learning
  • Applications

16 W
Mar
8
Final Poster/Demo Presentations 2:00-4:30pm at 236 Whittemore

17 M
Mar
13
Final Exam 2:05-4:05pm in 307 MCB


Grading

  • 40% Homework
  • 25% Project
  • 10% Mid Term
  • 20% Final
  • 5% Class Participation

Late policy for deliverables

  • No penalties for medical reasons (bring doctor's note) or emergencies
  • Every student has 4 free late days (4 x 24hours-chunks) for this course; these cannot be used for midterm or final
  • After all late days are used up, half credit after two days, zero credit after 2 days.

Project

Team project: 1-2 people. Description and grading policy (proposal + presentation, progress report, final report + presentation).

Textbooks

None required.

  • Reference Books:
  • [Free PDF from author's webpage] 
  • Bayesian reasoning and Machine Learning, David Barber 
[On Library Reserve]
Machine Learning: A Probabilistic Perspective, Kevin Murphy

  • [On Library Reserve]
  • Pattern Recognition and Machine Learning, Chris Bishop

Prerequisites

  • Probability and Statistics
    • Distributions, densities, marginalization, Moments, typical distributions, regression
  • Linear Algebra
    • Matrix multiplication, eigenvalues, positive semi-definiteness.
  • Algorithms
    • Dynamic programming, basic data structures, complexity
  • Programming
    • Matlab for HWs. Your language of choice for project
    • No coding / compilation support
  • Ability to deal with “abstract mathematical concepts”

Related Classes / Online Resources

  • 10-701 Machine Learning, Carnegie Mellon University
  • CIS 520 Machine Learning, UPenn
  • CS 229 Machine Learning, Stanford
  • Machine Learning, Coursera

Webpage CSS courtesy Bootstrap and Polo Chau.

© 2013 Virginia Tech