STAT 453: Introduction to Deep Learning and Generative Models

Course Website: http://pages.stat.wisc.edu/~sraschka/teaching/stat453-ss2020/

Topics Summary (Planned)

Below is a list of the topics I am planning to cover. Note that while these topics are numerated by lectures, note that some lectures are longer or shorter than others. Also, we may skip over certain topics in favor of others if time is a concern. While this section provides an overview of potential topics to be covered, the actual topics will be listed in the course calendar at the bottom of the course website.

Part 1: Introduction

L01: Course overview, introduction to deep learning
L02: The brief history of deep learning
Single-layer neural networks: The perceptron algorithm

Part 2: Mathematical and computational foundations

Linear algebra and calculus for deep learning
Parameter optimization with gradient descent
Automatic differentiation
Cluster and cloud computing resources

Part 3: Introduction to neural networks

Multinomial logistic regression
Multilayer perceptrons
Regularization
Input normalization and weight initialization
Learning rates and advanced optimization algorithms
Project proposal (online submission)

Part 4: Deep learning for computer vision and language modeling

Introduction to convolutional neural networks 1
Introduction to convolutional neural networks 2
Introduction to recurrent neural networks 1
Introduction to recurrent neural networks 2
Midterm exam

Part 5: Deep generative models

Autoencoders
Autoregressive models
Variational autoencoders
Normalizing Flow Models
Generative adversarial networks 1
Generative adversarial networks 2
Evaluating generative models

Part 6: Class projects and final exam

Course summary
Student project presentations 1
Student project presentations 2
Student project presentations 3
Final exam
Final report (online submission)