94-775: Unstructured Data Analytics for Policy
(Spring 2021 Mini 4)

Lecture 1: Course overview
[slides]

Lecture 2: Basic text analysis, co-occurrence analysis
[slides]

For the basic text analysis demo to work, please install Anaconda Python 3, Jupyter, and spaCy first
[Jupyter notebook (basic text analysis)]

Recitation: Basic Python review
[Jupyter notebook]

Lecture 3: Finding possibly related entities
[slides]
[Jupyter notebook (co-occurrence analysis)]

What is the maximum phi-squared/chi-squared value? (technical)
[stack exchange answer]

Lecture 4: Visualizing high-dimensional data with PCA
[slides]
[Jupyter notebook (PCA)]

Causality additional reading:
[Computational and Inferential Thinking, "Causality and Experiments" chapter]

PCA additional reading (technical):
[Abdi and Williams's PCA review]

Recitation slot — Lecture 5: Manifold learning with Isomap
[slides]
[Jupyter notebook (manifold learning)]

Python examples for dimensionality reduction:
[scikit-learn example (PCA, Isomap, t-SNE, and many other methods)]

Lecture 6: Wrap up manifold learning (t-SNE), a first look at analyzing images, and an introduction to clustering phenomena
[slides]
[Jupyter notebook (manifold learning); same demo as previous lecture]
[required reading: "How to Use t-SNE Effectively" (Wattenberg et al, Distill 2016)]
For the demo below to work (on t-SNE with images), you will need to install some packages:
pip install torch torchvision
[Jupyter notebook (t-SNE with images)]
[Jupyter notebook (PCA, t-SNE, clustering on UCI drug consumption data)]
[slides with some technical details for t-SNE]

HW1 due 11:59pm

See supplementary materials from the previous lecture; in addition, here's some reading for t-SNE (technical):
[Simon Carbonnelle's t-SNE slides]
[t-SNE webpage]

Lecture 7: Distance and similarity functions, clustering (k-means, GMMs)
[slides]
[Jupyter notebook (PCA, t-SNE, clustering on UCI drug consumption data); same demo as previous lecture]

Additional clustering reading (technical):
[see Section 14.3 of the book "Elements of Statistical Learning" on clustering]

Recitation slot — Lecture 8: More on clustering (interpreting clustering results, automatically choosing the number of clusters for GMM-related models, density-based clustering with DBSCAN)
[slides]
[Jupyter notebook (PCA, t-SNE, clustering on UCI drug consumption data); same demo as previous lecture]

Reading on DP-means, DP mixture models for which a DP-GMM is a special case (technical):
[Revisiting k-means: New Algorithms via Bayesian Nonparametrics]

Lecture 9: Wrap up clustering; topic modeling
[slides]
[Jupyter notebook (topic modeling with LDA)]

Topic modeling reading:
[David Blei's general intro to topic modeling]
[(technical) Topic Modelling Meets Deep Neural Networks: A Survey]

No class (CMU break day)

Lecture slot: Quiz review

Lecture slot: Quiz

Recitation slot — Lecture 10: Introduction to predictive data analytics
[slides]
For the demo below to work, you will need to install some packages:
pip install torch torchvision
[Jupyter notebook (prediction and model validation)]

Some nuanced details on cross-validation (technical):
[Andrew Ng's article Preventing "Overfitting" of Cross-Validation Data]
[Braga-Neto and Dougherty's article Is cross-validation valid for small-sample microarray classification? (this article applies more generally rather than only to microarray data from biology)]
[Bias and variance as we change the number of folds in k-fold cross-validation]

HW2 and final project proposal due 11:59pm

Lecture 11: Wrap up predictive model evaluation and classical classifiers
[slides]
[Jupyter notebook (prediction and model validation; same demo as last time)]

Lecture 12: Intro to neural nets and deep learning
[slides]
For the neural net demo below to work, you will need to install some packages:
pip install torch torchvision torchaudio
pip install torchsummaryX
python -m spacy download en
pip install pytorch-nlp
[Jupyter notebook (handwritten digit recognition with neural nets; be sure to scroll to the bottom to download UDA_pytorch_utils.py)]

PyTorch tutorial (at the very least, go over the first page of this tutorial to familiarize yourself with going between NumPy arrays and PyTorch tensors, and also understand the basic explanation of how tensors can reside on either the CPU or a GPU):
[PyTorch tutorial]

Additional reading:
[Chapter 1 "Using neural nets to recognize handwritten digits" of the book Neural Networks and Deep Learning]

Video introduction on neural nets:
["But what *is* a neural network? | Chapter 1, deep learning" by 3Blue1Brown]

Mike Jordan's Medium article on where AI is at (April 2018):
["Artificial Intelligence - The Revolution Hasn't Happened Yet"]

No recitation; please schedule with your TA Jingbo for a project checkup

HW3 due 11:59pm

Lecture 13: Image analysis with convolutional neural nets
[slides]
[Jupyter notebook (handwritten digit recognition with neural nets; same demo as previous lecture]

Additional reading:
[Stanford CS231n Convolutional Neural Networks for Visual Recognition]
[(technical) Richard Zhang's fix for max pooling (presented at ICML 2019)]

Lecture 14: Time series analysis with recurrent neural nets; some other deep learning topics; course wrap-up
[slides]

Supplemental demo for lecture (that isn't actually covered in lecture due to time constraints):
For the demo below to work, be sure to install the prerequisite packages as mentioned for the lecture 12 demo.
[Jupyter notebook (sentiment analysis with IMDb reviews)]

Additional reading:
[A tutorial on word2vec word embeddings]
[A tutorial on BERT word embeddings]

Recitation slot: final project presentations

Final projects due 11:59pm (slide deck + Jupyter notebook)