Course Overview

This course provides an introduction to machine learning with a special focus on engineering applications. The course starts with a mathematical background required for machine learning and covers approaches for supervised learning (linear models, kernel methods, decision trees, neural networks) and unsupervised learning (clustering, dimensionality reduction), as well as theoretical foundations of machine learning (learning theory, optimization). Evaluation will consist of mathematical problem sets and programming projects targeting real-world engineering applications.

Prerequisites

This course is intended for graduate students and qualified undergraduate students with a strong mathematical and programming background. Undergraduate level training or coursework in algorithms, linear algebra, calculus, probability, and statistics is suggested. A background in programming will also be necessary for the problem sets; students are expected to be familiar with python or learn it during the course. At CMU, this course is most similar to MLD's 10-701, though this course is meant specifically for students in engineering.

Textbooks

There will be no required textbooks, though we suggest the following to help you to study (all available online):

Piazza

We will use Piazza for class discussions. Please go to this Piazza website to join the course forum (note: you must use a cmu.edu email account to join the forum). We strongly encourage students to post on this forum rather than emailing the course staff directly (this will be more efficient for both students and staff). Students should use Piazza to:

The course Academic Integrity Policy must be followed on the message boards at all times. Do not post or request homework solutions! Also, please be polite.

Grading Policy

Grades will be based on the following components:

Staff Contact

TAs:

  Jacob Hoffman (Pitt)
jhoffma1@andrew.cmu.edu
  Samarth Gupta (Pitt)
samarthg@andrew.cmu.edu
  Ritwick Chaudhry (Pitt)
rchaudhr@andrew.cmu.edu
  Shreyas Chaudhari (Pitt)
schaudh2@andrew.cmu.edu
  Soham Deshmukh (Pitt)
sdeshmuk@andrew.cmu.edu
  Mike Weber (SV)
mweber2@andrew.cmu.edu
  TJ Kim(SV)
nrangara@andrew.cmu.edu
  Sweta Hari Kumar (SV)
sharikum@andrew.cmu.edu

Collaboration Policy

Group studying and collaborating on problem sets are encouraged, as working together is a great way to understand new material. Students are free to discuss the homework problems with anyone under the following conditions: Students are encouraged to read CMU's Policy on Cheating and Plagiarism.

Acknowledgments

This course is based in part on material developed by Fei Sha, Ameet Talwalkar, Matt Gormley, and Emily Fox. We also thank Anit Sahu and Joao Saude for their help with course development.


Tentative Schedule

Date Lecture Readings Announcements
Monday, 13th Jan Lecture 1 : Intro & Math Quiz [Slides]
  • KM, Ch. 1.

Wed, 15th Jan Lecture 2 : Probability and Linear Algebra Review, MLE/MAP [Slides]
Fri, 17th Jan Recitation

HW 1 released HW1 Jupyter Notebook

Mon, 20th Jan No class (MLK Day)

Wed, 22nd Jan Lecture 3 : Linear Regression, part I [Slides]

Mon, 27th Jan Lecture 4 : Linear Regression, part II [Slides]

HW 1 due

Wed, 29th Jan Lecture 5 : Overfitting, Bias/variance tradeoff, Evaluation [Slides]

HW 2 released

Mon, 3rd Feb Lecture 6 : Naive Bayes / Logistic Regression, part I [Slides]

Wed, 5th Feb Lecture 7 : Naive Bayes / Logistic Regression, part II
[Slides]

Mon, 10th Feb Lecture 8 : Multi-class Classification/Perceptron
[Slides]

HW 2 due

Wed, 12th Feb Lecture 9 : SVM, part I [Slides]

HW 3 released

Mon, 17th Feb Lecture 10 : SVM, part II [Slides]

Wed, 19th Feb Lecture 11 : SVM, part III [Slides]

Fri, 21st Feb In Recitation Exam Review

HW 3 Due

Mon, 24th Feb Lecture 12 : Nearest Neighbors [Slides]

Wed, 26th Feb Mid-term exam

Mon, 2nd Mar Lecture 14 : Decision Trees [Slides]
  • CIML,
  • KM, Ch. 16.2
  • ESL, Ch. 9.2
  • Ch. 1.3

Wed, 4th Mar Lecture 15 : Boosting, Random Forest [Slides]
  • KM, Ch. 16.4

Fri, 6th Mar

HW4 Released

Mon, 9th Mar No Class; Spring Break

Wed, 11th Mar No Class; Spring Break

Mon, 16th Mar Optional Mini-Lecture [Slides]

Wed, 18th Mar Lecture 16 : Neural Networks, Part I [Slides]

HW 5 release date delayed.(COVID19)

Mon, 23rd Mar Lecture 17 : Neural Networks, Part II [Slides]

HW 4 due
HW 5 Released
HW 5 Files

Wed, 25th Mar Lecture 18 : Neural Networks, Part III [Slides]

Mon, 30th Mar Lecture 19 : Clustering, Part I [Slides]

Wed, 1st Apr Lecture 20 : Clustering, Part II
[Slides]
  • ESL, Ch. 14.3.1-14.3.9

HW 6 released Data Files

Fri, 3rd Apr Recitation : Clustering,GMMs

HW 5 Due

Wed, 6th Apr Lecture 21 : EM
[Slides]

Wed, 8th Apr Lecture 22 : Dimensionality Reduction [Slides]

Sun, 12th Apr

HW6 due

Mon, 13th Apr Lecture 23 : Pytorch [ Slides]

Wed, 15th Apr Lecture 24 : Online Learning [Slides]

HW 7 released.

Mon, 20th Apr Lecture 25 : Reinforcement Learning [Slides]

Wed, 22nd Apr Guest Lecture | Special Topics

Fri, 24th Apr Recitation

HW 7 due

Mon, 27th Apr Lecture 26 : Last Lecture | Final Exam Review [Slides]

Wed, 29th April Final Exam