Course Overview
This course provides an introduction to machine learning with a special focus on engineering applications. The course starts with a mathematical background required for machine learning and covers approaches for supervised learning (linear models, kernel methods, decision trees, neural networks) and unsupervised learning (clustering, dimensionality reduction), as well as theoretical foundations of machine learning (learning theory, optimization). Evaluation will consist of mathematical problem sets and programming projects targeting realworld engineering applications.
Prerequisites
This course is intended for graduate students and qualified undergraduate students with a strong mathematical and programming background. Undergraduate level training or coursework in algorithms, linear algebra, calculus, probability, and statistics is suggested. A background in programming will also be necessary for the problem sets; students are expected to be familiar with python or learn it during the course. At CMU, this course is most similar to MLD's 10701, though this course is meant specifically for students in engineering.
Textbooks
There will be no required textbooks, though we suggest the following to help you to study (all available online): (KM): Machine Learning: A Probabilistic Perspective, Kevin Murphy. Online access is free through CMU’s library. Note that to access the library, you may need to be on CMU’s network or VPN.
 (ESL): Elements of Statistical Learning Trevor Hastie, Robert Tibshirani and Jerome Friedman.
 (TM): Machine Learning, Tom Mitchell.
 (CIML): A Course in Machine Learning, Hal Daumé III.
Piazza
We will use Piazza for class discussions. Please go to this Piazza website to join the course forum (note: you must use a cmu.edu email account to join the forum). We strongly encourage students to post on this forum rather than emailing the course staff directly (this will be more efficient for both students and staff). Students should use Piazza to:
 Ask clarifying questions about the course material.
 Share useful resources with classmates (so long as they do not contain homework solutions).
 Look for students to form study groups.
 Answer questions posted by other students to solidify your own understanding of the material.
Grading Policy
Grades will be based on the following components:
 Homework (50%): There will be 7 homeworks.
 Late submissions will not be accepted.
 There is one exception to this rule: You are given 3 “late days” (selfgranted 24hr extensions) which you can use to give yourself extra time without penalty. At most one late day can be used per assignment. This will be monitored automatically via Gradescope.
 Solutions will be graded on both correctness and clarity. If you cannot solve a problem completely, you will get more partial credit by identifying the gaps in your argument than by attempting to cover them up.
 Midterm (20%)
 Final (30%)
Collaboration Policy
Group studying and collaborating on problem sets are encouraged, as working together is a great way to understand new material. Students are free to discuss the homework problems with anyone under the following conditions: Students must write their own solutions and understand the solutions that they wrote down.
 Students must list the names of their collaborators (i.e., anyone with whom the assignment was discussed).
 Students may not use old solution sets from other classes under any circumstances, unless the instructor grants special permission.
Acknowledgments
This course is based in part on material developed by Fei Sha, Ameet Talwalkar, Matt Gormley, and Emily Fox. We also thank Anit Sahu and Joao Saude for their help with course development.
Tentative Schedule
Date  Lecture  Readings  Announcements 

Tue, 15th Jan  Lecture 1 : Intro & Math Quiz [Slides] 


Thu, 17th Jan  Lecture 2 : Probability and Linear Algebra Review [Slides]
[HW 1 Jupyter Notebook] (Right click + save link) 

HW 1 Latex Template
Gradescope Code : M4JXD3 
Fri, 18th Jan 
Recitation : Linear Algebra & Math Quiz Review [Slides] 


Tue, 22nd Jan 
Lecture 3 : Linear Regression, part I [Slides] 


Thu, 24nd Jan 
Lecture 4 : Linear Regression, part II [Slides] 

HW 2 released HW 1 due 
Fri, 25th Jan 
Recitation : Linear Regression Review 


Tue, 29nd Jan 
Lecture 5 : Overfitting, Bias/variance tradeoff, Evaluation [Slides] 


Thu, 31st Jan 
Lecture 6 : Class Cancelled due to Severe Weather 


Fri, 1st Feb 
Recitation : Linear Regression II Review 


Tue, 5th Feb 
Lecture 7 : Naive Bayes / Logistic Regression, part I [Slides] 


Thu, 7th Feb 
Lecture 8 : Naive Bayes / Logistic Regression, part II [Slides] 

HW 3 released HW 2 due 
Fri, 8th Feb 
Recitation : Logistic Regression 


Tue, 12th Feb 
Lecture 9 : Multiclass Classification/Perceptron [Slides] 


Thu, 14th Feb 
No class 


Fri, 15th Feb 
Recitation : Logistic Regression 


Tue, 19th Feb 
Lecture 10 : SVM, part I [Slides] 

HW 4 released HW 3 due 
Thu, 21st Feb 
Lecture 11 : SVM, part II [Slides] 


Fri, 22nd Feb 
Lecture 12 : No Class 


Tue, 26th Feb 
Midterm Review 


Thu, 28th Feb 
Lecture 13 : Nearest Neighbors [Slides] 
HW 4 due


Fri, 1st Mar 
Recitation : Midterm Office Hours 


Tue, 5th Mar 
Midterm exam 


Thu, 7th Mar 
Lecture 14 : Decision Trees [Slides] 

HW 5 released

Fri, 8th Mar 
No Recitation 


Tue, 12th Mar 
No Class; Spring Break 


Thu, 14th Mar 
No Class; Spring Break 


Fri, 15th Mar 
No Class; Spring Break 


Tue, 19th Mar 
Lecture 15 : Boosting, Random Forest [Slides] 


Thu, 21st Mar 
Lecture 16 : Neural Networks, Part I [Slides] 
HW 5 due


Fri, 22th Mar 
Recitation : Nearest Neighbors, Decision trees, Boosting 


Tue, 26th Mar 
Lecture 17 : Neural Networks, Part II [Slides] 
HW 6 released


Thu, 28th Mar 
Lecture 18 : Neural Networks, Part III [Slides] 


Fri, 29th Mar 
Recitation : Neural Networks, TensorFlow 


Tue, 2nd Apr 
Lecture 19 : TensorFlow Intro 


Thu, 4th Apr 
Lecture 20 : Clustering, Part I [Slides] 


Tue, 9th Apr 
Lecture 21 : Clustering, Part II [Slides] 

HW 7 released

Wed, 10th Apr 


HW 6 due

Thu, 11th Apr 
No class (Spring Carnival) 


Fri, 12th Apr 
Recitation : No Class; Spring Carnival 


Tue, 16th Apr 
Lecture 22 : Dimensionality Reduction [Slides] 


Thu, 18th Apr 
Lecture 23 : Largescale ML [Slides] 


Fri, 19th Apr 
Recitation : EM, Dim. Reduction, Largescale ML 

HW 7 one notebook due 
Tue, 23rd Apr 
Lecture 24 : Guest talk by Liam Li on Hyperparameter Tuning 


Thu, 25th Apr 
Lecture 25 : Wrapup and Summary [Slides] 


Fri, 26th Apr 
Recitation : Final Exam Review 

HW 7 remaining notebooks due

Tue, 30th Apr 
Lecture 26 : Final Exam Review 


Thu, 2nd May 
Final Exam 

