Introduction to Machine Learning and Perception
Virginia Tech, Electrical and Computer EngineeringFall 2013: ECE 4984 / ECE 5984
Course Information
Have you ever wondered how Siri understands voice commands? How Netflix recommends movies to watch? How Kinnect recognizes full-body gestures? How Watson defeated Jeopardy champions? The answer is Machine Learning -- the study of algorithms that learn from large quantities of data, identify patterns and make predictions on new instances.
- Instructor
- Dhruv Batra
- Office Hour
- Fri 1-2pm, Whittemore 468
- TA
- Qing Sun (sunqing -at- vt)
- Office Hour
- Wed 6-8pm, Whittemore 415
- Class meets
- Tue, Thr 5:00 - 6:15, RAND 216
- Homeworks
- Q&A and Discussions
- https://scholar.vt.edu/portal/site/f13-ece4984-ece5984
- ECE4984_ECE5984 Forum
- Class Kaggle Links
-
http://inclass.kaggle.com/c/vt-ece-machine-learning-perception-project-1
http://inclass.kaggle.com/c/vt-ece-machine-learning-perception-hw-2
http://inclass.kaggle.com/c/vt-ece-machine-learning-perception-hw-3
Best Project Prize Winners
- Winning Team 5984:
Apoorva Garg and Deba Pratim Saha.
Gesture Recognition for Leap Motion. - Winning Team 4984:
Ben Weinstein-Raun and Sean Thweatt.
Application of Neural Networks to Prediction of Vegetation Index. - Runners-up:
Sudeep Bhattarai.
Comparing the Performance of k-NN and Correlation Coefficient Methods in Iris Recognition.
Michael Drescher.
Classification of Email Source using Support Vector Machines.
Schedule (Tentative)
- General Setup: learning from data
- Your First Classifier: 1/k Nearest Neighbour
- (Finish) 1/k Nearest Neighbour
- Kernel regression/classification
- Probability Review
- Statistical Estimatation (MLE,MAP,Bayesian)
- Linear Models
- Regression
- Gaussians
- Project Details
- (Finish) Regression
- Error Decomposition
- Bias-Variance Tradeoff
- Cross-validation
- Classification: Naive Bayes
- (Finish) Naive Bayes
- Classification: Logistic Regression
- (Finish) Logistic Regression
- NB & LR connections
- Support Vector Machines
- Support Vector Machines
- Linear SVMs
- Hard & soft-margin SVMs
- Support Vector Machines
- Lagrangian Duality
- SVM dual
- (Finish) Support Vector Machines
- Kernel Trick
- Multi-class SVM
- Neural Networks
- Perceptron
- Multilayer Perceptron
- Backprop
- (Finish) Neural Networks
- Backprop
- Autoencoders
- Convolutional Nets
- Decision Trees
- Mutual Information
- Bagging
- Boosting
- Clustering
- K-Means
- Gaussian Mixture Models
- EM for GMMs
- EM for general unsupervised learning
- (Finish) EM for general unsupervised learning
- Factor Analysis
- PCA
- SVD
- Dimensionality Reduction
- Probabilistic Graphical Models
- Bayes Nets
- Hidden Markov Models (HMMs)
- Online Learning
- Active Learning
- Semi-Supervised Learning
- Reinforcement Learning
- Learning Theory / PAC-Learning
- 40% Homework
- 25% Project
- 10% Mid Term
- 20% Final
- 5% Class Participation
- No penalties for medical reasons (bring doctor's note) or emergencies
- Every student has 4 free late days (4 x 24hour-chunks) for this course; these cannot be used for midterm or final
- After all late days are used up, half credit after one day, zero credit after two days.
- Probability and Statistics
- Distributions, densities, marginalization, Moments, typical distributions
- Linear Algebra
- Matrix multiplication, eigenvalues, positive semi-definiteness.
- Algorithms
- Dynamic programming, basic data structures, complexity
- Programming
- Matlab for HWs. Your language of choice for project
- No coding / compilation support
- Ability to deal with “abstract mathematical concepts”
- 10-701 Machine Learning, Carnegie Mellon University
- CIS 520 Machine Learning, UPenn
- CS 229 Machine Learning, Stanford
- Machine Learning, Coursera
| Week/Day | Date | Topic | Notes |
|---|---|---|---|
| 1 T | Aug 27 | Overview of Machine Learning & Perception | HW0 is out. Readings: Barber Chap 1, 13.1. [Optional] Video: Sam Roweis -- What is Machine Learning? |
| 1 R | Aug 29 | Supervised Learning
|
Reading: Barber Chap 14. |
| 2 T | Sep 3 |
|
HW0 is due the day before. |
| 2 R | Sep 5 | Supervised Learning |
Readings: Barber 8.1, 8.2. [Optional] Videos: Probability Primer. |
| 3 T | Sep 10 |
|
Readings: Barber 8.6, 8.7. [Optional] Video: Daphne Koller -- Coursera: Probabilistic Graphical Models, MLE Lecture, MAP Lecture. [Optional] Video: Michael Jordon -- Bayesian or Frequentist: Which Are You? |
| 3 R | Sep 12 |
|
HW1 is out. Readings: Barber 8.4, 17.1, 17.2. |
| 4 T | Sep 17 |
|
|
| 4 R | Sep 19 |
|
|
| 5 T | Sep 24 |
|
Readings: Barber 10.1-3. [Optional] Video: Andrew Ng -- Naive Bayes |
| 5 R | Sep 26 |
|
HW1 due the day before. Readings: Barber 17.4. [Optional] Video: Andrew Ng -- Logistic Regression. |
| 6 T | Oct 1 |
|
Reading: Tom Mitchell -- Book Chapter: Naive Bayes and Logistic Regression |
| 6 R | Oct 3 |
|
Project Proposals due the day before.
Readings: Barber 17.5. [Optional] Video: Andrew Ng -- KKT Conditions and SVM Duality. |
| 7 T | Oct 8 |
|
[Optional] Video: Stephen Boyd -- Lagrangian Duality. |
| 7 R | Oct 10 |
|
HW2 is due on Oct 11. |
| 8 T | Oct 15 | In class Mid-Term | |
| 8 R | Oct 17 |
|
Reading: Murphy 16.5. [Optional] Reading: Hastie, Tibshirani, Friedman -- Chap 11. [Optional] Video: Andrew Ng -- Coursera: Machine Learning, Neural Networks lecture, Backpropagation lecture. |
| 9 T | Oct 22 |
|
HW3 out. |
| 9 R | Oct 24 |
|
Reading: Murphy 16.1-16.2. |
| 10 T | Oct 29 | In-class Project Presentations | |
| 10 R | Oct 31 | In-class Project Presentations | |
| 11 T | Nov 5 | In-class Project Presentations | HW3 is due the day before. HW4 out. |
| 11 R | Nov 7 |
Ensemble Methods
|
Reading: Murphy 16.4. [Optional] Video: Robert Schapire -- Boosting |
| 12 T | Nov 12 |
Unsupervised Learning |
Readings: Barber 20.1-20.3. [Optional] Video: Andrew Ng -- Clustering, GMM |
| 12 R | Nov 14 |
|
|
| 13 T | Nov 19 |
|
Readings: Barber 15.1-15.4. [Optional] Video: Andrew Ng -- PCA |
| 13 R | Nov 21 |
Overview of Advanced Topics
|
HW4 is due on 22nd. [Optional] Reading: Pedro Domingo -- A Few Useful Things to Know about Machine Learning [Optional] Video: Andrew Ng -- Advice for Applying Machine Learning |
| 14 T | Nov 26 | Thanksgiving Break. | |
| 14 R | Nov 28 | Thanksgiving Break. | |
| 16 T | Dec 3 | No Class. | |
| 16 R | Dec 5 | No Class. | |
| Dec 10 | Project Poster+Demo Presentation: 12:30-3pm | 236 Whittemore; ECE Integrated Design Lab. | |
| Dec 16 | Final Exam 7-9pm |
Grading
Late policy for deliverables
Project
Team project: 1-2 people. Description and grading policy (proposal, midsem presentation, final poster + demo presentation).Textbooks
None required. Reference Books:[Free PDF from author's webpage]
Bayesian Reasoning and Machine Learning, David Barber
[On Library Reserve]
Machine Learning: A Probabilistic Perspective, Kevin Murphy
Prerequisites
Related Classes / Online Resources
© 2013 Virginia Tech
Webpage CSS courtesy Bootstrap and Polo Chau.