Course Overview

This course provides an introduction to machine learning with a special focus on engineering applications. The course starts with a mathematical background required for machine learning and covers approaches for supervised learning (linear models, kernel methods, decision trees, neural networks) and unsupervised learning (clustering, dimensionality reduction), as well as theoretical foundations of machine learning (learning theory, optimization). Evaluation will consist of mathematical problem sets and programming projects targeting real-world engineering applications.

Prerequisites

This course is intended for graduate students and qualified undergraduate students with a strong mathematical and programming background. Undergraduate level training or coursework in algorithms, linear algebra, calculus, probability, and statistics is suggested. A background in programming will also be necessary for the problem sets; students are expected to be familiar with python or learn it during the course. At CMU, this course is most similar to MLD's 10-701, though this course is meant specifically for students in engineering.

Textbooks

There will be no required textbooks, though we suggest the following to help you to study (all available online): We will provide suggested readings from these books in the schedule below.

Piazza

We will use Piazza for class discussions. Please go to this Piazza website to join the course forum (note: you must use a cmu.edu email account to join the forum). We strongly encourage students to post on this forum rather than emailing the course staff directly (this will be more efficient for both students and staff). Students should use Piazza to:

The course Academic Integrity Policy must be followed on the message boards at all times. Do not post or request homework solutions! Also, please be polite.

Grading Policy

Grades will be based on the following components:

Staff Contact Info

Instructors:

Prof. Yuejie Chi  yuejiec@andrew.cmu.edu
Prof. Gauri Joshi  gaurij@andrew.cmu.edu

TAs:

Tuhinangshu Choudhury   tuhinanc@andrew.cmu.edu
Divyansh Jhunjhunwala  djhunjhu@andrew.cmu.edu
Muqiao Yang  muqiaoy@andrew.cmu.edu
Yafei Hu  yafeih@andrew.cmu.edu
Tianshu Huang  tianshu2@andrew.cmu.edu
Keane Lucas  kjlucas@andrew.cmu.edu
Madhav Mahendra Wagh  mwagh@andrew.cmu.edu

Collaboration Policy

Group studying and collaborating on problem sets are encouraged, as working together is a great way to understand new material. Students are free to discuss the homework problems with anyone under the following conditions: Students are encouraged to read CMU's Policy on Cheating and Plagiarism.

Acknowledgments

This course is based in part on material developed by Fei Sha, Ameet Talwalkar, Matt Gormley, and Emily Fox. We also thank Anit Sahu and Joao Saude for their help with course development.


Tentative Schedule

Date Lecture Readings Announcements
Tue, 30th Aug Lecture 1 : Intro & Math Quiz [Slides]
  • KM, Ch. 1.
Thu, 1st Sep Lecture 2 : MLE/MAP and Linear Algebra Review [Slides]

  • HW1 release

Fri, 2nd Sep Recitation [Slides]
Tue, 6th Sep Lecture 3 : Linear Regression, part I [Slides]

Thu, 8th Sep Lecture 4 : Linear Regression, part II [Slides]
Fri, 9th Sep Recitation [Slides]
  • HW1 due
  • HW2 release
Tue, 13th Sep Lecture 5 : Overfitting, Bias/variance tradeoff, Evaluation [Slides]
Thu, 15th Sep Lecture 6 : Naive Bayes [Slides]
  • CIML, Ch. 9
  • KM, Ch. 3.5
Fri, 16th Sep Recitation [Slides]
Tue, 20th Sep Lecture 7 : Logistic Regression [Slides]
Thu, 22nd Sep Lecture 8 : Multi-class Classification [Slides]
  • KM, Ch. 8.5
Fri, 23rd Sep Recitation
Sun, 25th Sep
  • HW2 due
  • HW3 release
Tue, 27th Sep Lecture 9 : SVM, part I [Slides]
Thu, 29th Sep Lecture 10 : SVM, part II [Slides]
Fri, 30th Sep Recitation [Slides, Codes]
Tue, 4th Oct Lecture 11 : Nearest Neighbors [Slides]
Thu, 6th Oct Lecture 12 : Decision Trees [Slides]
  • CIML, Ch. 1.3
  • KM, Ch. 16.2
    ESL, Ch. 9.2
Fri, 7th Oct Recitation
  • HW3 due
  • HW4 release
Tue, 11th Oct Lecture 13 : Boosting, Random Forest [Slides]
Thu, 13th Oct Lecture 14 : Graphical Models I [Slides]
Fri, 14th Oct Recitation
Tue, 18th Oct Fall Break
Thu, 20th Oct Fall Break
Fri, 21st Oct Fall Break
Tue, 25th Oct Lecture 15 : Graphical Models II [Slides]
  • HW4 due
  • HW5 release
Thu, 27th Oct Midterm
Fri, 28th Oct No Recitation - Community Day
Tue, 1st Nov Lecture 16 : Neural Networks, Part I [Slides]
Thu, 3rd Nov Lecture 17 : Neural Networks, Part II [Slides]
Fri, 4th Nov Pytorch Recitation
Sun, 6th Nov
  • HW6 release
Tue, 8th Nov Lecture 18 : Clustering, Part I [Slides]
  • HW5 due
Thu, 10th Nov Lecture 19 : Clustering, Part II [Slides]
  • ESL, Ch. 14.3.1-14.3.9
Fri, 11th Nov Recitation
Tue, 15th Nov Lecture 20 : Dimensionality Reduction [Slides]
Thu, 17th Nov Lecture 21 : Online Learning (Bandits) [Slides]
Fri, 18th Nov Recitation
  • HW6 due
  • HW7 release
Tue, 22nd Nov Lecture 22 : Reinforcement Learning, Part I [Slides]
Thu, 24th Nov Thanksgiving
Fri, 25th Nov Thanksgiving
Tue, 29nd Nov Reinforcement Learning, Part II [Slides]
Thu, 1st Dec Last Lecture (Review) [Slides]
Fri, 2nd Dec Cancelled - Office hours before Final
Tue, 6th Dec Guest Lecture on modern topics in ML
Thu, 8th Dec Final Exam