Skip to content
Snippets Groups Projects
Commit 94e88d31 authored by Niklas Eberts's avatar Niklas Eberts
Browse files

Deleted duplicate initial jupyter files

parent 302df1d7
No related branches found
No related tags found
No related merge requests found
Expand all Show whitespace changes
Inline Side-by-side
exercises/exercise2/Exercise_2(4).ipynb deleted 100644 → 0
+ 0
− 579
View file @ 302df1d7
%% Cell type:markdown id: tags:
<a href="https://colab.research.google.com/github/mindgarage/very-deep-learning-wise2324/blob/main/exercises/Exercise_2(4).ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
%% Cell type:markdown id: tags:
# Programming Exercise 2(4): CIFAR Challenge
## Very Deep Learning (VDL) - Winter Semester 2023/24
---
### Group Details:
- **Group Name:** \[Enter OLAT Group Name Here\]
### Members:
- \[Participant 1 Name\], \[Matrikel-Nr 1\]
- \[Participant 2 Name\], \[Matrikel-Nr 1\]
- ...
---
**Instructions**: The tasks in this notebook are a part of Sheet 1. Look for `TODO` tags throughout the notebook and complete the sections with missing code. Once done, ensure all outputs are visible and correctly displayed. Save your notebook and submit the `.ipynb` file together with the exercise sheet PDF in a single ZIP file.
%% Cell type:code id: tags:
```
%matplotlib inline
```
%% Cell type:code id: tags:
```
import torch
if torch.cuda.is_available():
device = torch.device('cuda')
else:
device = torch.device('cpu')
print(device)
```
%% Cell type:markdown id: tags:
Note: Tasks in this notebook can be solved without a GPU as well
%% Cell type:markdown id: tags:
__Objective of this notebook is to give hands on experience in training a Convolutional Neural Network(CNN) from scratch using Pytorch framework. Further the tasks will also give an insight on how to change the network architecture and hyperparameter settings to achieve better results.__
Fill the code in the notebook where it is explicitly mentioned and solve all tasks to complete the assignment
%% Cell type:markdown id: tags:
We divide our task in followind steps:
1- Load the CIFAR-10 dataset and look at some properties of the dataset like:
* size of the images
* number of images in the dataset
* number of channels in the image (RGB=3, grayscale=1)
* number of classes in the dataset
2- Define a network architecture which takes batches of the images in the form of a 4-dimensional tensor and compute the output scores for each class in the dataset.
3- Define a loss function for training the network, and an optimizer (SGD, Adam, or Adagrad) which defines the strategy to update weights of the network.
4- Finally, train the network and find out the accuracy on validation set
5- Improve the network architecture to get better accuracy.
%% Cell type:markdown id: tags:
---
__Step-1__ : Loading the CIFAR-10 dataset is very easy using ``torchvision`` package, which has readily availabe famous datasets and API to load these datasets into batches known as dataloaders via ``torchvision.Datasets``and ``torch.utils.data.DataLoader'``
%% Cell type:code id: tags:
```
###########################################################################################
## Define transform variable ##
## ##
## ##
## First define a transform to convert RGB images into tensors and another transform to ##
## normalize them with the channel-wise mean and standard deviation of the CIFAR-10 ##
## dataset ##
## ##
###########################################################################################
import torchvision
import torchvision.transforms as transforms
transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(mean=[0.5, 0.5, 0.5], # approx. mean and std of R, G, and B channels of
std=[0.25, 0.25, 0.25]) # all the images in CIFAR-10. You can find out the
]) # exact values and replace these with them.
```
%% Cell type:code id: tags:
```
## TODO ##
###########################################################################################
## Use above transform as an argument to load the training and the test set for CIFAR-10 ##
## https://pytorch.org/vision/stable/generated/torchvision.datasets.CIFAR10.html ##
###########################################################################################
trainset =
testset =
```
%% Cell type:code id: tags:
```
## TODO ##
###########################################################################################
## Write code to print below mentioned statistics: ##
## ##
## - size of the images (height, width and number of channels) ##
## - size of the training images ##
## - size of the test images ##
## - classes in CIFAR-10 dataset ##
## ##
###########################################################################################
print("CIFAR-10 DATASET STATS")
print(" Image size (h, w, c):")
print(" # of training images:")
print(" # of test images:")
print(" # of classes:")
```
%% Cell type:code id: tags:
```
###########################################################################################
## Define your data loaders and split training dataset into train and validation sets ##
## ##
###########################################################################################
import torch
batch_size = 64
# Create dataloader for testset
test_loader = torch.utils.data.DataLoader(testset, batch_size=batch_size, shuffle=False, num_workers=2)
# Split the trainset into train_data and val_data with 1000 images in validation set and
# rest in the training set (use random_split in pytorch to split the dataset)
val_data, train_data = torch.utils.data.random_split(trainset, [1000, 49000])
# Create dataloaders for train and validation sets
trainloader = torch.utils.data.DataLoader(train_data, batch_size=batch_size, shuffle=True, num_workers=2)
valloader = torch.utils.data.DataLoader(val_data, batch_size=batch_size, shuffle=True, num_workers=2)
```
%% Cell type:code id: tags:
```
## TODO ##
###########################################################################################
## Write code to load a batch of images and visualize them ##
## Hint: Use plt.imshow and torchvision.utils.make_grid ##
###########################################################################################
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
def convert_img(img):
mean = np.array([0.5, 0.5, 0.5])
std = np.array([0.25, 0.25, 0.25])
np_img = img.numpy()
np_img = np.transpose(np_img, (1, 2, 0))
np_img = (np_img*std) + mean
return np_img
```
%% Cell type:markdown id: tags:
---
__Step-2:__ Define a simple convolutional Neural network with two convolutional layers and 2 fully-connected layers
(conv1): Conv2d(3, 6, kernel_size=(5, 5), stride=(1, 1)) \
(pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False) \
(conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1)) \
(fc1): Linear(in_features=400, out_features=120, bias=True) \
(fc2): Linear(in_features=120, out_features=84, bias=True) \
(fc3): Linear(in_features=84, out_features=10, bias=True)
%% Cell type:code id: tags:
```
## TODO ##
###########################################################################################
## Define forward pass of the network ##
## ##
###########################################################################################
import torch.nn as nn
import torch.nn.functional as F
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
def forward(self, x):
return x
```
%% Cell type:markdown id: tags:
__Step-3:__ Defining the loss function
%% Cell type:code id: tags:
```
## TODO ##
## Define the loss function here
criterion =
```
%% Cell type:markdown id: tags:
__Step 4:__ Train the network, for 10 epochs and calculate the accuracy on the validation set
Define some utility functions and then functions to train the model on the test set and validate it on the validation set
%% Cell type:code id: tags:
```
###########################################################################################
## DO NOT CHANGE THE CODE HERE!! ##
## ##
###########################################################################################
def weight_reset(m):
if isinstance(m, nn.Conv2d) or isinstance(m, nn.Linear):
m.reset_parameters()
```
%% Cell type:code id: tags:
```
###########################################################################################
## DO NOT CHANGE THE CODE HERE!! ##
## ##
###########################################################################################
def get_accuracy(output, target, topk=(1,)):
"""Computes the accuracy over the
k top predictions for the specified values of k"""
with torch.no_grad():
maxk = max(topk)
batch_size = target.size(0)
_, pred = output.topk(maxk, 1, True, True)
pred = pred.t()
correct = pred.eq(target.view(1, -1).expand_as(pred))
res = []
for k in topk:
correct_k = correct[:k].view(-1).float().sum(0, keepdim=True)
res.append(correct_k.mul_(100.0 / batch_size))
return res
```
%% Cell type:code id: tags:
```
###########################################################################################
## DO NOT CHANGE THE CODE HERE!! ##
## ##
###########################################################################################
def validate(net, valoader, criterion):
val_loss = 0
val_accuracy = 0
net.eval()
with torch.no_grad():
for batch_idx, (inputs, targets) in enumerate(valloader):
inputs, targets = inputs.to(device), targets.to(device)
outputs = net(inputs)
loss = criterion(outputs, targets)
val_loss += loss.item()
accuracy = get_accuracy(outputs, targets)
val_accuracy += accuracy[0].item()
avg_loss = val_loss/(batch_idx+1)
avg_acc = val_accuracy/(batch_idx+1)
return avg_loss, avg_acc
```
%% Cell type:code id: tags:
```
###########################################################################################
## DO NOT CHANGE THE CODE HERE!! ##
## ##
###########################################################################################
def train_one_epoch(model, trainloader, optimizer, criterion):
model.train()
train_loss = 0
train_accuracy = 0
for batch_idx, (inputs, targets) in enumerate(trainloader):
inputs, targets = inputs.to(device), targets.to(device)
optimizer.zero_grad()
outputs = model(inputs)
loss = criterion(outputs, targets)
loss.backward()
optimizer.step()
train_loss += loss.item()
accuracy = get_accuracy(outputs, targets)
train_accuracy += accuracy[0].item()
avg_train_loss = train_loss/(batch_idx+1)
avg_train_acc = train_accuracy/(batch_idx+1)
return avg_train_loss, avg_train_acc, model
```
%% Cell type:code id: tags:
```
## TODO ##
###########################################################################################
## Define an optimizer (SGD, learning rate = 0.01 and momentum 0.9) and ##
## train the model for 10 epochs ##
## ##
###########################################################################################
import torch.optim as optim
model = Net().to(device)
model.apply(weight_reset)
optimizer =
num_epochs = 10
for epoch in range(num_epochs):
train_loss, train_acc, model = train_one_epoch(model, trainloader, optimizer, criterion)
val_loss, val_acc = validate(model, valloader, criterion)
message = (f"epoch:{epoch+1} train_acc:{train_acc:.2f} "
f"val_acc:{val_acc:.2f}")
print(message)
```
%% Cell type:markdown id: tags:
### Task: Define your own network, such that the validation acccuracy of the model is greater than 70%. Following constraints should be strictly followed: ###
1. The network should be trained from scratch (randomly initialized weigths), transfer learning is not allowed
Note: Apply __model.apply(weight_reset)__ method over your model before training to make sure model is randomly initialized
2. Use the training methods as defined above (train_one_epoch() and validate() should be used and not changed)
3. Your network should reach atleast 70% accuracy on validatin set under 10 epochs
%% Cell type:code id: tags:
```
## TODO ##
###########################################################################################
## Write the implementation of the model here according to the task defined above. ##
## Define any other changes you made in optimizer or transforms (if required) in this ##
## cell only. Train the model for 10 epochs and print validation accuracy after the ##
## 10th epoch ##
###########################################################################################
import torch.nn as nn
import torch.nn.functional as F
from collections import OrderedDict
class GhostModel(nn.Module):
def __init__(self):
super(GhostModel, self).__init__()
def forward(self, x):
return x
```
%% Cell type:markdown id: tags:
__Hint:__ One might try following ways to obtain the required results:
1) __Filter Size:__ The filter size can be decreased from 5 to 3, this can increase efficiency and reduce overfitting
2) __Number of Filters:__ Increase the number of filters in each convoltuional layer. More convolutional filters gives a network higher representation power
3) __Add more convolutions:__ Increase from two to three convolutional layers. This indeed helps in better receptive fields of the filters and increase represtation power of the network.
4) __Apply Batch Normalizations:__ Batch Normalization will help your model to train faster. Hence it can help the network to converge to better accuracy in less number of epochs
5) __Global Average Pooling:__ Get rid of all the layers and replace them by average pooling and a linear layer for classification. This idea is from the Inception network.
6) __Regularization:__ Apply L2 regularization and dropout, if you see validation accuracy going much lower than the training accuracy
7) __Different Optimizer:__ Instead of SGD, optimizers like Adam and Adagrad can be tried
%% Cell type:code id: tags:
```
import torch.optim as optim
model = GhostModel().to(device)
model.apply(weight_reset)
optimizer =
num_epochs = 10
for epoch in range(num_epochs):
train_loss, train_acc, model = train_one_epoch(model, trainloader, optimizer, criterion)
val_loss, val_acc = validate(model, valloader, criterion)
message = (f"epoch:{epoch+1} train_acc:{train_acc:.2f} "
f"val_acc:{val_acc:.2f}")
print(message)
```
exercises/exercise2/Exercise_2(5).ipynb deleted 100644 → 0
+ 0
− 444
View file @ 302df1d7
This diff is collapsed.
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment