Course material

Dog vs cat dataset (used in most ipython notebooks)

Background						78 min
	Background				pdf key	2 min
		Linear Algebra and Gradients			pdf key	19 min
		Probability, Likelihood, Sampling, and Expectation			pdf key	16 min
		Tensors			pdf key	5 min
		Data as Tensors			ipynb html	9 min
		Broadcasting			ipynb html	11 min
		Setup and Tensorboard			ipynb html	15 min
	Summary				pdf key	2 min
First Example						107 min
	Linear Classifiers and Regression				pdf key	7 min
		Linear Regression			pdf key	6 min
		Linear Classification			pdf key	8 min
		Linear Classification in Action			ipynb html	13 min
		Linear Multi-class Classification			pdf key	11 min
		Optimization and Gradient Descent			pdf key	12 min
		Gradient Descent in Action			ipynb html	10 min
		Computation Graphs			pdf key	6 min
		Building a Computation Graph			ipynb html	10 min
		Gradient Computation on Computation Graphs			pdf key	10 min
		Back Propagation in Action			ipynb html	12 min
	Summary				pdf key	2 min
Deep Networks						135 min
	Beyond Linear Models				pdf key	2 min
		Limitations of Linear Models			pdf key	9 min
		Non-linearities (ReLU)			pdf key	11 min
		Output Representations			pdf key	9 min
		Loss Functions			pdf key	8 min
		Building a Deep Network in PyTorch			ipynb html	15 min
		Optimization of Deep Networks			pdf key	6 min
			Stochastic Gradient Descent		pdf key	11 min
			Mini-batches		pdf key	7 min
			Momentum		pdf key	7 min
		Optimization in PyTorch			ipynb html	20 min
		What Is a Layer?			pdf key	6 min
		Activation Functions			pdf key	15 min
		Hyper-parameters			pdf key	2 min
	Summary, a Practical Guide to Deep Network Design				pdf key	7 min
Convolutional Networks						172 min
	Images and Structure				pdf key	4 min
		High Dimensional Inputs			pdf key	4 min
		Convolutions			pdf key	22 min
		Convolutional Network in PyTorch			ipynb html	8 min
		Convolutional Operators and Their Structure			pdf key	21 min
		Average Pooling			pdf key	6 min
		Max Pooling			pdf key	6 min
		Convolutional Operations in PyTorch			ipynb html	7 min
		Receptive Fields			pdf key	12 min
		Design Principles of Convolutional Networks			pdf key	16 min
		Building Efficient Convolutional Networks			ipynb html	13 min
		Deep Representations and Exploiting the Structure of the Data			pdf key	10 min
		Examining the Structure of Deep Networks			ipynb html	15 min
		Dilation			pdf key	9 min
		Up-Convolution			pdf key	11 min
	Summary				pdf key	5 min
Making It Work						307 min
	Practical Deep Learning				pdf key	7 min
		Looking at Your Data			pdf key	6 min
		Training, Validation, and Test Sets			pdf key	14 min
			Distribution of Data		ipynb html	12 min
		Network Initialization			pdf key	12 min
			Random Initialization		pdf key	9 min
			Xavier and Kaiming Initialization		pdf key	20 min
			Initialization in PyTorch		ipynb html	5 min
		Optimization			pdf key	2 min
			Input Normalization		pdf key	12 min
			Vanishing and Exploding Gradients		pdf key	12 min
			Normalization		pdf key	2 min
				Batch Normalization	pdf key	9 min
				Layer Normalization	pdf key	3 min
				Instance Normalization	pdf key	3 min
				Group Normalization and Local Response Normalization	pdf key	6 min
				Where to Add Normalizations?	pdf key	8 min
				Normalizations in PyTorch	ipynb html	5 min
			Residual Connections		pdf key	13 min
				Residual Connections in Practice	ipynb html	8 min
			Optimization Algorithms		pdf key	10 min
			Learning Rate		pdf key	9 min
				Learning Rate Schedules in PyTorch	ipynb html	10 min
			Open Problem: Pruning and Compression		pdf key	12 min
		Overfitting and How to Detect It			pdf key	7 min
			Early Stopping		pdf key	3 min
			Data Augmentation		pdf key	12 min
			Dropout		pdf key	11 min
			Weight Decay		pdf key	6 min
			Ensembles		pdf key	8 min
			Reducing Overfitting		repo	20 min
			Transfer Learning		pdf key	9 min
			Open Problem: Understanding Generalization		pdf key	13 min
	Summary, a Practical Guide to Deep Network Optimization				pdf key	11 min
Computer Vision						254 min
	Computer Vision Tasks				pdf key	15 min
		Image Classification			pdf key	13 min
			Case Study: AlexNet		pdf key	19 min
			Case Study: VGG		pdf key	11 min
			1x1 Convolutions and Factorization		pdf key	7 min
			Case Study: Network in Network		pdf key	3 min
			Case Study: Inception Architecture		pdf key	8 min
			Case Study: Residual Networks		pdf key	11 min
			Factorization and Light-weight Networks		pdf key	5 min
			Case Study: MobileNet		pdf key	5 min
			Using Pre-trained Architectures		ipynb html	11 min
		Object Detection			pdf key	11 min
			Case Study: RCNN		pdf key	13 min
			Case Study: Faster RCNN		pdf key	14 min
			Case Study: RetinaNet		pdf key	10 min
		Segmentation			pdf key	10 min
			Case Study: FCN		pdf key	11 min
			Case Study: Dilated Convolutional Networks		pdf key	10 min
			Case Study: Mask RCNN		pdf key	9 min
			Open Problem: Object Representations		pdf key	12 min
		Temporal Models			pdf key	6 min
			3D Convolutions		pdf key	5 min
			2+1D Convolutions		pdf key	3 min
			Case Study: I3D		pdf key	6 min
			Open Problem: Effective Temporal Operations		pdf key	10 min
		Open Problem: What Should We Infer or Label?			pdf key	12 min
	Summary				pdf key	4 min
Reinforcement Learning						210 min
	Acting in an Environment				pdf key	23 min
		Acting in SuperTuxKart			ipynb html	17 min
		Imitation Learning			pdf key	11 min
		Dagger			pdf key	8 min
		Dagger vs Imitation Learning			ipynb html	30 min
		Non-differentiability			pdf key	13 min
		REINFORCE			pdf key	11 min
		Implementing REINFORCE			ipynb html	30 min
		Policy Gradient			pdf key	20 min
		Gradient Free Optimization			pdf key	17 min
		Gradient Free Optimization in PyTorch			ipynb html	13 min
		Open Problem: Structure Vs Data			pdf key	13 min
	Summary				pdf key	5 min
Sequence Modeling						131 min
	Sequence Models				pdf key	6 min
		Recurrent Neural Networks			pdf key	16 min
		Training Recurrent Networks			pdf key	14 min
		LSTMs and GRUs			pdf key	15 min
		Temporal Convolutions			pdf key	16 min
		Sampling in Sequence Models			pdf key	10 min
		Case Study: WaveNet			pdf key	12 min
		Sequence Models in PyTorch			ipynb html	20 min
		Attention and Transformers			pdf key	16 min
	Summary				pdf key	6 min
Special Topics						156 min
	Embedding Learning					38 min
	Learning with an Expanding Set of Labels				pdf key	4 min
		Embedding Learning			pdf key	7 min
		Contrastive Losses			pdf key	8 min
		Triplet Losses			pdf key	5 min
		Selecting Training Examples			pdf key	11 min
	Summary				pdf key	4 min
	Generative Models					80 min
	Image Generation				pdf key	3 min
		Autoencoders			pdf key	9 min
		Variational Autoencoders			pdf key	12 min
		Transforming Noise			pdf key	4 min
		Generative Adversarial Networks			pdf key	11 min
		Pix2Pix			pdf key	5 min
		CycleGan			pdf key	10 min
		Image Editing			pdf key	5 min
		Style Transfer			pdf key	11 min
		Open Problem: Understanding Generative Models and Invariances			pdf key	8 min
	Summary				pdf key	2 min
	Adversarial Attacks					38 min
	Fooling Deep Networks				pdf key	5 min
		Finding Adversarial Examples			pdf key	11 min
		Defense Through Data Augmentation			pdf key	5 min
		White vs Black Box Attacks			pdf key	7 min
		Open Problem: Realistic Attacks and Defenses			pdf key	9 min
	Summary				pdf key	2 min
Final Words						39 min
	Open Problem: Bias, Fairness, and Ethics in Deep Learning				pdf key	29 min
	Course Summary and Further Topics				pdf key	10 min