All times are listed in Pittsburgh time
Class time and location:
Instructor: George Chen (email: georgechen ♣ cmu.edu) ‐ replace "♣" with the "at" symbol
Teaching assistants:
Office hours (starting second week of class):
Office hours are all held remotely over Zoom; Zoom links for office hours are posted in Canvas.
Contact: Please use Piazza (follow the link to it within Canvas) and, whenever possible, post so that everyone can see (if you have a question, chances are other people can benefit from the answer as well!).
Companies, governments, and other organizations now collect massive amounts of data such as text, images, audio, and video. How do we turn this heterogeneous mess of data into actionable insights? A common problem is that we often do not know what structure underlies the data ahead of time, hence the data often being referred to as "unstructured". This course takes a practical approach to unstructured data analysis via a twostep approach:
Prerequisite: If you are a Heinz student, then you must have already completed 95791 "Data Mining" and also one of either 95888 "DataFocused Python" or 90819 "Intermediate Programming with Python". If you are not a Heinz student and would like to take the course, please contact the instructor and clearly state what Python courses you have taken/what Python experience you have.
Helpful but not required: Math at the level of calculus and linear algebra may help you appreciate some of the material more
Grading:
Letter grades are determined based on a curve.
Syllabus: [pdf]
Previous lecture slides and demos: spring 2021 version of 94775 (click to see course webpage)
Date  Topic  Supplemental Material 

Part I. Exploratory data analysis  
Tue Mar 15 
Lecture 1: Course overview, analyzing text using frequencies
[slides] Please install Anaconda Python 3 and spaCy by following this tutorial (needed for HW1 and the demo next lecture): [slides] Note: Anaconda Python 3 includes support for Jupyter notebooks, which we use extensively in this class HW1 released (check Canvas) 

Thur Mar 17 
Lecture 2: Basic text analysis demo (requires Anaconda Python 3 & spaCy), cooccurrence analysis
[slides] [Jupyter notebook (basic text analysis)] 

Fri Mar 18 
Recitation: Basic Python review
[Jupyter notebook] 

Tue Mar 22 
Lecture 3: Cooccurrence analysis (cont'd), visualizing highdimensional data
[slides] [Jupyter notebook (cooccurrence analysis)] 
What is the maximum phisquared/chisquared value? (technical)
[Stack Exchange answer] 
Thur Mar 24 
Lecture 4: PCA, manifold learning
[slides] [Jupyter notebook (PCA)] 
Additional reading (technical): [Abdi and Williams's PCA review] [The original Isomap paper (Tenenbaum et al 2020)] 
Fri Mar 25 
Recitation: More on PCA, practice with argsort
[Jupyter notebook] 

Tue Mar 29 
Lecture 5: Manifold learning (cont'd), clustering
[slides] [Jupyter notebook (manifold learning)] [required reading: "How to Use tSNE Effectively" (Wattenberg et al, Distill 2016)] HW1 due 11:59pm 
Python examples for dimensionality reduction:
[scikitlearn example (PCA, Isomap, tSNE, and many other methods)] Some technical details for tSNE: [slides] Even more technical reading for tSNE: [Simon Carbonnelle's tSNE slides] [tSNE webpage] 
Thur Mar 31 
Lecture 6: Clustering (cont'd)
[slides] [Jupyter notebook (dimensionality reduction with images)***] ***For the demo on tSNE with images to work, you will need to install some packages: pip install torch torchvision
[Jupyter notebook (dimensionality reduction and clustering with drug data)] 
Clustering additional reading (technical): [see Section 14.3 of the book "Elements of Statistical Learning" on clustering] 
Fri Apr 1 
Recitation slot — Lecture 7: Clustering (cont'd) — interpreting GMMs, automatically selecting the number of clusters
[slides] We continue using the same demo from last time: [Jupyter notebook (dimensionality reduction and clustering with drug data)] 
See supplemental clustering reading posted for previous lecture 
Tue Apr 5 
Lecture 8: Clustering (cont'd), topic modeling
[slides] [Jupyter notebook (topic modeling with LDA)] Final project proposals due 11:59pm by email (1 email per group) 
Topic modeling reading:
[David Blei's general intro to topic modeling] 
Thur Apr 7 Fri Apr 8 
No class (CMU Spring Carnival)  
Mon Apr 11  HW2 due 11:59pm (note that we release HW solutions 2 days after the due date in case anyone is using 2 late days, so this due date is set up so that you get solutions before the quiz)  
Tue Apr 12  Lecture slot: Quiz review  
Thur Apr 14  Lecture slot: Quiz  
Fri Apr 15  No recitation (instead sign up to meet with TAs to discuss final project status)  
Part II. Predictive data analysis  
Tue Apr 19 
Lecture 9: Topic modeling (cont'd), intro to predictive data analysis
[slides] 

Thur Apr 21 
Lecture 10: Hyperparameter tuning, classifier evaluation, intro to neural nets & deep learning
[slides] [Jupyter notebook (prediction and model validation)] 
Some nuanced details on crossvalidation (technical):
[Andrew Ng's article Preventing "Overfitting" of CrossValidation Data] [BragaNeto and Dougherty's article Is crossvalidation valid for smallsample microarray classification? (this article applies more generally rather than only to microarray data from biology)] [Bias and variance as we change the number of folds in kfold crossvalidation] Mike Jordan's Medium article on where AI is at (April 2018): ["Artificial Intelligence  The Revolution Hasn't Happened Yet"] 
Fri Apr 22 
Recitation: More on classifier evaluation
[slides] [Jupyter notebook] 

Tue Apr 26 
Lecture 11: Neural nets & deep learning
[slides] For the neural net demo below to work, you will need to install some packages: pip install torch torchvision torchaudio
pip install torchsummaryX
pip install pytorchnlp
[Jupyter notebook (handwritten digit recognition with neural nets; be sure to scroll to the bottom to download UDA_pytorch_utils.py)] Be sure to edit two pytorchnlp files as indicated in the following slides (resolves some issues with recent updates to PyTorch & spaCy): [slides] HW3 due 11:59pm 
PyTorch tutorial (at the very least, I suggest going over the first page of this tutorial to familiarize yourself with going between NumPy arrays and PyTorch tensors, and also understand the basic explanation of how tensors can reside on either the CPU or a GPU): [PyTorch tutorial] Additional reading: [Chapter 1 "Using neural nets to recognize handwritten digits" of the book Neural Networks and Deep Learning] Video introduction on neural nets: ["But what *is* a neural network?  Chapter 1, deep learning" by 3Blue1Brown] 
Thur Apr 28 
Lecture 12: Image analysis with CNNs, time series analysis with RNNs, deep learning and course wrapup
[slides] We continue using the demo from the previous lecture 
Additional reading:
[Stanford CS231n Convolutional Neural Networks for Visual Recognition] [(technical) Richard Zhang's fix for max pooling] [Christopher Olah's "Understanding LSTM Networks"] [A tutorial on word2vec word embeddings] [A tutorial on BERT word embeddings] RNN demo — Sentiment analysis with IMDb reviews (95865 demo): [Jupyter notebook (sentiment analysis with IMDb reviews; requires UDA_pytorch_utils.py from the previous demo)] 
Fri Apr 29  Recitation slot: Final project presentations  
Mon May 2  Final project slide decks + Jupyter notebooks due 11:59pm by email (1 email per group) 