Course Overview

This course provides an introduction to machine learning with a special focus on engineering applications. The course starts with a mathematical background required for machine learning and covers approaches for supervised learning (linear models, kernel methods, decision trees, neural networks) and unsupervised learning (clustering, dimensionality reduction), as well as theoretical foundations of machine learning (learning theory, optimization). Evaluation will consist of mathematical problem sets and programming projects targeting real-world engineering applications.

Prerequisites

This course is intended for graduate students and qualified undergraduate students with a strong mathematical and programming background. Undergraduate level training or coursework in algorithms, linear algebra, calculus, probability, and statistics is suggested. A background in programming will also be necessary for the problem sets; students are expected to be familiar with python or learn it during the course. At CMU, this course is most similar to MLD's 10-701, though this course is meant specifically for students in engineering.

Textbooks

There will be no required textbooks, though we suggest the following to help you to study (all available online):

Piazza

We will use Piazza for class discussions. Please go to this Piazza website to join the course forum (note: you must use a cmu.edu email account to join the forum). We strongly encourage students to post on this forum rather than emailing the course staff directly (this will be more efficient for both students and staff). Students should use Piazza to:

The course Academic Integrity Policy must be followed on the message boards at all times. Do not post or request homework solutions! Also, please be polite.

Grading Policy

Grades will be based on the following components:

Collaboration Policy

Group studying and collaborating on problem sets are encouraged, as working together is a great way to understand new material. Students are free to discuss the homework problems with anyone under the following conditions: Students are encouraged to read CMU's Policy on Cheating and Plagiarism.

Acknowledgments

This course is based in part on material developed by Fei Sha, Ameet Talwalkar, Matt Gormley, and Emily Fox. We also thank Anit Sahu and Joao Saude for their help with course development.


Tentative Schedule

Date Lecture Readings Announcements
Tue, 15th Jan Lecture 1 : Intro & Math Quiz [Slides]
  • KM, Ch. 1.

Thu, 17th Jan Lecture 2 : Probability and Linear Algebra Review [Slides] [HW 1 Jupyter Notebook] (Right click + save link)

HW 1 released

HW 1 Latex Template

Gradescope Code :- M4JXD3

Fri, 18th Jan Recitation : Linear Algebra & Math Quiz Review [Slides]

Tue, 22nd Jan Lecture 3 : Linear Regression, part I [Slides]

Thu, 24nd Jan Lecture 4 : Linear Regression, part II [Slides]

HW 2 released

HW 1 due

Fri, 25th Jan Recitation : Linear Regression Review

Tue, 29nd Jan Lecture 5 : Overfitting, Bias/variance tradeoff, Evaluation [Slides]

Thu, 31st Jan Lecture 6 : Class Cancelled due to Severe Weather

Fri, 1st Feb Recitation : Linear Regression II Review

Tue, 5th Feb Lecture 7 : Naive Bayes / Logistic Regression, part I [Slides]

Thu, 7th Feb Lecture 8 : Naive Bayes / Logistic Regression, part II
[Slides]

HW 3 released

HW 2 due

Fri, 8th Feb Recitation : Logistic Regression

Tue, 12th Feb Lecture 9 : Multi-class Classification/Perceptron
[Slides]

  • KM, Ch. 8.5

Thu, 14th Feb No class

Fri, 15th Feb Recitation : Logistic Regression

Tue, 19th Feb Lecture 10 : SVM, part I [Slides]

HW 4 released

HW 3 due

Thu, 21st Feb Lecture 11 : SVM, part II [Slides]

Fri, 22nd Feb Lecture 12 : No Class

Tue, 26th Feb Mid-term Review

Thu, 28th Feb Lecture 13 : Nearest Neighbors [Slides]

HW 4 due

Fri, 1st Mar Recitation : Mid-term Office Hours

Tue, 5th Mar Mid-term exam

Thu, 7th Mar Lecture 14 : Decision Trees [Slides]
  • CIML,
  • KM, Ch. 16.2
  • ESL, Ch. 9.2
  • Ch. 1.3

HW 5 released

Fri, 8th Mar No Recitation

Tue, 12th Mar No Class; Spring Break

Thu, 14th Mar No Class; Spring Break

Fri, 15th Mar No Class; Spring Break

Tue, 19th Mar Lecture 15 : Boosting, Random Forest [Slides]
  • KM, Ch. 16.4

Thu, 21st Mar Lecture 16 : Neural Networks, Part I [Slides]
HW 5 due

Fri, 22th Mar Recitation : Nearest Neighbors, Decision trees, Boosting

Tue, 26th Mar Lecture 17 : Neural Networks, Part II [Slides]

HW 6 released

Thu, 28th Mar Lecture 18 : Neural Networks, Part III [Slides]

Fri, 29th Mar Recitation : Neural Networks, TensorFlow

Tue, 2nd Apr Lecture 19 : TensorFlow Intro

Thu, 4th Apr Lecture 20 : Clustering, Part I [Slides]

Tue, 9th Apr Lecture 21 : Clustering, Part II [Slides]
  • ESL, Ch. 14.3.1-14.3.9

HW 7 released

Wed, 10th Apr

HW 6 due

Thu, 11th Apr No class (Spring Carnival)

Fri, 12th Apr Recitation : No Class; Spring Carnival

Tue, 16th Apr Lecture 22 : Dimensionality Reduction [Slides]

Thu, 18th Apr Lecture 23 : Large-scale ML [Slides]

Fri, 19th Apr Recitation : EM, Dim. Reduction, Large-scale ML

HW 7 one notebook due

Tue, 23rd Apr Lecture 24 : Guest talk by Liam Li on Hyperparameter Tuning

Thu, 25th Apr Lecture 25 : Wrap-up and Summary [Slides]

Fri, 26th Apr Recitation : Final Exam Review

HW 7 remaining notebooks due

Tue, 30th Apr Lecture 26 : Final Exam Review

Thu, 2nd May Final Exam