DPS921/PyTorch: Convolutional Neural Networks

Convolutional Neural Networks Using Pytorch

The basic idea was to create a convolutional neural network using the python machine learning Framework PyTorch. The actual code will

be written in Jupyter Lab both for demonstration and implementation purposes. Furthermore, using the the torchvision dataset, the goal

was to show the training of the neural network, and show the classification of several images which have a single digit from 0 - 9.

A successful execution will show the correct determination of what number resides in that specific image. As part our of research,

We will explain in detail how an actual convolution neural network works at a fundamental level. Will we will both take a graphical

and mathematical approach to explaining the different parts of the neural network and how it comes together as whole. Furthermore,

we will briefly explain how it relates to parallel computing and how parallel computing plays a significant role in driving the

implementation of the neural network.

Group Members

1. Shervin Tafreshipour

2. Parsa Jalilifar

3. Novell Rasam

Intro to Neural Networks

Artificial intelligence is an umbrella term with many levels. Machine learning is a subset of AI that focuses on these self-teaching algorithms. Deep learning is a further subset of machine learning where multi-layered artificial neural networks are employed to allow for more versatile, independent learning [1]. Machine learning algorithms that do not utilize deep learning would be less versatile in what they could learn and would need more hand-holding by programmers. A handy infographic is shown below:

Implementation of a Neural Network

In order to implement a Neural Network using the PyTorch Framework and Jupyter Lab, there are some key steps that need to be followed:

1. Import the required modules to download the datasets required to train the neural network.

import torch 
import torchvision
from torchvision import transforms, datasets

2. Download the needed datasets from the MNIST database, partition them into feasible data batch sizes.

train = datasets.MNIST(, train = True, download = True, transform=transforms.Compose([transforms.ToTensor()]))
test = datasets.MNIST(, train = False, download = True, transform=transforms.Compose([transforms.ToTensor()]))

trainset = torch.utils.data.DataLoader(train, batch_size = 10, shuffle = True)
testset = torch.utils.data.DataLoader(test, batch_size = 10, shuffle = False)

3. Import the necessary modules to define the structure of the neural network

import torch.nn as nn 
import torch.nn.functional as F

4. Define the class structure of the NN, in the following case, we have defined 4 linear layers, that output to three functions that execute rectified linear processing as an activation method. Finally instantiate the NN class.

class Net(nn.Module):
   
   def __init__(self):
       super().__init__()
       self.fc1 = nn.Linear(784, 64)
       self.fc2 = nn.Linear(64, 64)
       self.fc3 = nn.Linear(64, 64)
       self.fc4 = nn.Linear(64, 10)
   
   def forward(self, x):
       x = F.relu(self.fc1(x))
       x = F.relu(self.fc2(x))
       x = F.relu(self.fc3(x))
       x = self.fc4(x)
       return F.log_softmax(x, dim = 1)
 
net = Net()

5. Use an optimizer to train the neural network. By passing the 1-dimensional tensor of pixel maps of each image in the training data set to the optimizer, it allows the optimizer to update the weight value of each of the layers in our neural network.

import torch.optim as optim
optimizer = optim.Adam(net.parameters(), lr=0.001)
EPOCHS = 3

 for epoch in range(EPOCHS):
    for data in trainset:
        # data is a batch of featuresets and labels
        X, y = data
        net.zero_grad()
        output = net(X.view(-1,28*28))
        loss = F.nll_loss(output, y)
        loss.backward()
        optimizer.step()
    print(loss)

6. Iterate through the trainset data again to verify current accuracy rating of our neural network using the training data we just trained on.

correct = 0 
total = 0 

with torch.no_grad():
    for data in trainset:
        X, y = data
        output = net(X.view(-1,784))
        for idx,i in enumerate(output):
            if torch.argmax(i) == y[idx]:
                correct += 1
            total += 1
            
print("Accuracy: ", round(correct/total, 3))

7. View the data at index of 0 in the training data set as 28 x 28 Image, Making sure to import 'matplotlib'.

import matplotlib.pyplot as plt
plt.imshow(X[0].view(28,28))
plt.show()

8. Test the trained neural network to check whether the digit value shown in the image is in fact the number the neural network determined it to be.

print(torch.argmax(net(X[0].view(-1, 784))[0]))

Getting Started With Jupyter

What is Jupyter Notebook?

This is basically a way for us to run the code interactively within a web browser alongside some visualizations and some markdown text to explain the process of what is going on.

User Interaction Model

Installation methods

• Anaconda (https://www.anaconda.com/products/individual)
• pip (https://jupyter.org/install)
• Vs code by using Anaconda plugin

In order to do Jupyter code, the easiest way is using plugin in vs code to install anaconda but if you don’t like environment of vs code IDE to do coding, you can install anaconda directly on your pc. This gives you chance to code on browser or use suggested IDES by anaconda. Also, pip can be used which is a package-management system written in Python used to install and manage software packages.

Points:

• It does not matter how many operations we have inside a cell it only returns the last one.

1+1
2+3

output:5

• There are different modes such as command and edit. Command is used to add or removed cells and when we are in this mode the cell cover with the line of blue colour while we are in edit mode to do cell editing cell will cover with the line of green colour.
• Numbers in '[ ]' shows the order of the execution
• To go to the new line we simply can use Enter
• To run the cell and to insert a new cell we just use Shift + Enter
• running the cell is by using Ctrl + Enter

Some Sample Codes

1. Printing a value

msg = "Hello World!"
print(msg)

2. Finding type of a value

type(1.5)

3. Making an array

my_list = [1,2,3,4,5]
my_list

4. Size of an array

len(my_list)

5. Object

dictionary = {"name":"Matt","age":"21"}
print(dictionary['name'])

6. for loop

for counter in[1,2,3,4]:
    print(counter)

7. Drawing a graph

import matplotlib.pylab as plt
import numpy as np

x = [0,1,2]
y = [0,1,4]

# to increase the size of the figure we can use 
fig = plt.figure(figsize=[12,8])
axes = fig.add_subplot(111)  # we are working only with only one chart
# axes.plot(x, y, color="red", linestyle='dashed') # to give style
# axes.plot(x,y)
axes.plot(x, y, marker='o', markerfacecolor="blue", markersize=6)
plt.show()

Installing PyTorch

The following command line instructions are directed at installing the PyTorch framework for Ubuntu Linux, Mac OS, and Windows using the Python Package Manager.

1. Ubuntu Linux:

  pip install torchvision

2. Mac OS:

  pip install torch torchvision torchaudio

3. Windows:

  pip install torch===1.7.0 torchvision===0.8.1 torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html