Homework 6

In this homework you’ll train your first truly deep networks. You’ll build your code on the network you’ve trained for homework 5. You can skip the ensemble and use 500k parameters for a single deep model.

starter code data

Deeper networks

Design a network containing more than 15 convolutional layers The order of the convolution and pooling layers are up to you.

Once you have specified the network, train it. You should notice that it is a lot harder to train than the simple 5 layer network used in the previous assignment.

Batch normalization

Adding batch-norm can greatly improve training accuracy for deeper models. Try adding batch-norm after each convolutional layer in your network. Make sure to neither add batch-norm nor a ReLU after the last linear layer (classification layer).

Resnets

In order to further decrease training time add residual connections to at least half of your convolutions. Note that if a convolution changes the number of input and output channels residual connections are not possible. Make sure you have a few convolutions with the same input and output channels.

Getting Started

We provide you with starter code that loads the image dataset and the corresponding labels from a training and validation set.

The code will measure classification accuracy as you train the model. We also provice a data augmentation starter code so that you can also use your data augmentation in hw5 to help reduce overfitting of your resnet.

Input example

tux

Output example

[-10.2, 4.3, 1.2, 8.7, -1.3, 2.8]

Grading

We will have a linear grading scheme depending on your test accuracy and your model depth:

model depth < 15: 0 points (your model has to have at least 15 conv layers to receive any credit. Note that having a thick architecture of conv layers and adding them up does not count. )
94% accuracy: 0 points (the master solution to hw5 will reach at least this)
96% accuracy: 100 points
(On test set) >93%: +10 extra points
(On test set) >94%: +10 extra points

Note the test set is slightly harder than the validation set, expect the validation accuracy to be 2% higher than the test accuracy. There are no guarantees about the test performance though if you tune your parameters too much, or train on the validation set.

Getting Started

We provide you with starter code that loads the image dataset and the corresponding labels from a training and validation set.

The code will measure classification accuracy as you train the model. We also provide an optional tensorboard interface.

Define your model in models.py.
Train your model.
```
 python3 -m homework.train
```
Optionally, you can use tensorboard to visualize your training loss and accuracy.
```
 python3 -m homework.train -l myRun
```
and in another terminal tensorboard --logdir myRun, where myRun is the log directory. Pro-tip: You can run tensorboard on the parent directory of many logs to visualize them all.
Test your model.
```
 python3 -m homework.test
```
To evaluate your code against grader, execute:
```
 python3 -m grader homework
```
Create the submission file
```
 python3 -m homework.bundle
```

Hint

You can take a look at our solution to homework 5 and upgrade it to a resnet.

Relevant operations

nn.BatchNorm2d
nn.Conv2d
operations of prior assignments