00. PyTorch Fundamentals Exercise Solutions¶

1. Documentation reading¶

A big part of deep learning (and learning to code in general) is getting familiar with the documentation of a certain framework you're using. We'll be using the PyTorch documentation a lot throughout the rest of this course. So I'd recommend spending 10-minutes reading the following (it's okay if you don't get some things for now, the focus is not yet full understanding, it's awareness):

• The documentation on torch.Tensor.
• The documentation on torch.cuda.

In [1]:

# No code solution (reading)

# No code solution (reading)

2. Create a random tensor with shape (7, 7).¶

In [2]:

# Import torch
import torch 

# Create random tensor
X = torch.rand(size=(7, 7))
X, X.shape

# Import torch import torch # Create random tensor X = torch.rand(size=(7, 7)) X, X.shape

Out[2]:

(tensor([[0.5656, 0.4012, 0.1987, 0.2464, 0.6861, 0.4953, 0.3433],
         [0.0032, 0.0228, 0.9020, 0.1267, 0.8009, 0.5274, 0.7453],
         [0.9123, 0.8138, 0.1667, 0.5998, 0.4657, 0.4473, 0.8367],
         [0.5302, 0.2213, 0.4747, 0.6485, 0.4770, 0.8675, 0.3054],
         [0.4226, 0.1398, 0.4495, 0.6974, 0.1808, 0.5872, 0.6931],
         [0.2153, 0.7517, 0.3505, 0.3815, 0.3244, 0.2511, 0.4269],
         [0.1158, 0.6696, 0.3733, 0.2633, 0.4102, 0.1101, 0.1613]]),
 torch.Size([7, 7]))

3. Perform a matrix multiplication on the tensor from 2 with another random tensor with shape (1, 7) (hint: you may have to transpose the second tensor).¶

In [3]:

# Create another random tensor
Y = torch.rand(size=(1, 7))
# Z = torch.matmul(X, Y) # will error because of shape issues
Z = torch.matmul(X, Y.T) # no error because of transpose
Z, Z.shape

# Create another random tensor Y = torch.rand(size=(1, 7)) # Z = torch.matmul(X, Y) # will error because of shape issues Z = torch.matmul(X, Y.T) # no error because of transpose Z, Z.shape

Out[3]:

(tensor([[1.0888],
         [1.7506],
         [1.8468],
         [1.7496],
         [1.9022],
         [1.2684],
         [0.8617]]), torch.Size([7, 1]))

4. Set the random seed to 0 and do 2 & 3 over again.¶

The output should be:

(tensor([[1.8542],
         [1.9611],
         [2.2884],
         [3.0481],
         [1.7067],
         [2.5290],
         [1.7989]]), torch.Size([7, 1]))

In [4]:

# Set manual seed
torch.manual_seed(0)

# Create two random tensors
X = torch.rand(size=(7, 7))
Y = torch.rand(size=(1, 7))

# Matrix multiply tensors
Z = torch.matmul(X, Y.T)
Z, Z.shape

# Set manual seed torch.manual_seed(0) # Create two random tensors X = torch.rand(size=(7, 7)) Y = torch.rand(size=(1, 7)) # Matrix multiply tensors Z = torch.matmul(X, Y.T) Z, Z.shape

Out[4]:

(tensor([[1.8542],
         [1.9611],
         [2.2884],
         [3.0481],
         [1.7067],
         [2.5290],
         [1.7989]]), torch.Size([7, 1]))

5. Speaking of random seeds, we saw how to set it with torch.manual_seed() but is there a GPU equivalent? (hint: you'll need to look into the documentation for torch.cuda for this one)¶

• If there is, set the GPU random seed to 1234.

In [5]:

# Set random seed on the GPU
torch.cuda.manual_seed(1234)

# Set random seed on the GPU torch.cuda.manual_seed(1234)

6. Create two random tensors of shape (2, 3) and send them both to the GPU (you'll need access to a GPU for this). Set torch.manual_seed(1234) when creating the tensors (this doesn't have to be the GPU random seed). The output should be something like:¶

Device: cuda
(tensor([[0.0290, 0.4019, 0.2598],
         [0.3666, 0.0583, 0.7006]], device='cuda:0'),
 tensor([[0.0518, 0.4681, 0.6738],
         [0.3315, 0.7837, 0.5631]], device='cuda:0'))

In [6]:

# Set random seed
torch.manual_seed(1234)

# Check for access to GPU
device = "cuda" if torch.cuda.is_available() else "cpu"
print(f"Device: {device}")

# Create two random tensors on GPU
tensor_A = torch.rand(size=(2,3)).to(device)
tensor_B = torch.rand(size=(2,3)).to(device)
tensor_A, tensor_B

# Set random seed torch.manual_seed(1234) # Check for access to GPU device = "cuda" if torch.cuda.is_available() else "cpu" print(f"Device: {device}") # Create two random tensors on GPU tensor_A = torch.rand(size=(2,3)).to(device) tensor_B = torch.rand(size=(2,3)).to(device) tensor_A, tensor_B

Device: cuda

Out[6]:

(tensor([[0.0290, 0.4019, 0.2598],
         [0.3666, 0.0583, 0.7006]], device='cuda:0'),
 tensor([[0.0518, 0.4681, 0.6738],
         [0.3315, 0.7837, 0.5631]], device='cuda:0'))

7. Perform a matrix multiplication on the tensors you created in 6 (again, you may have to adjust the shapes of one of the tensors).¶

The output should look like:

(tensor([[0.3647, 0.4709],
         [0.5184, 0.5617]], device='cuda:0'), torch.Size([2, 2]))

In [7]:

# Perform matmul on tensor_A and tensor_B
# tensor_C = torch.matmul(tensor_A, tensor_B) # won't work because of shape error
tensor_C = torch.matmul(tensor_A, tensor_B.T)
tensor_C, tensor_C.shape

# Perform matmul on tensor_A and tensor_B # tensor_C = torch.matmul(tensor_A, tensor_B) # won't work because of shape error tensor_C = torch.matmul(tensor_A, tensor_B.T) tensor_C, tensor_C.shape

Out[7]:

(tensor([[0.3647, 0.4709],
         [0.5184, 0.5617]], device='cuda:0'), torch.Size([2, 2]))

8. Find the maximum and minimum values of the output of 7.¶

In [8]:

# Find max
max = torch.max(tensor_C)

# Find min
min = torch.min(tensor_C)
max, min

# Find max max = torch.max(tensor_C) # Find min min = torch.min(tensor_C) max, min

Out[8]:

(tensor(0.5617, device='cuda:0'), tensor(0.3647, device='cuda:0'))

9. Find the maximum and minimum index values of the output of 7.¶

In [9]:

# Find arg max
arg_max = torch.argmax(tensor_C)

# Find arg min
arg_min = torch.argmin(tensor_C)
arg_max, arg_min

# Find arg max arg_max = torch.argmax(tensor_C) # Find arg min arg_min = torch.argmin(tensor_C) arg_max, arg_min

Out[9]:

(tensor(3, device='cuda:0'), tensor(0, device='cuda:0'))

10. Make a random tensor with shape (1, 1, 1, 10) and then create a new tensor with all the 1 dimensions removed to be left with a tensor of shape (10). Set the seed to 7 when you create it and print out the first tensor and it's shape as well as the second tensor and it's shape.¶

The output should look like:

tensor([[[[0.5349, 0.1988, 0.6592, 0.6569, 0.2328, 0.4251, 0.2071, 0.6297,
           0.3653, 0.8513]]]]) torch.Size([1, 1, 1, 10])
tensor([0.5349, 0.1988, 0.6592, 0.6569, 0.2328, 0.4251, 0.2071, 0.6297, 0.3653,
        0.8513]) torch.Size([10])

In [10]:

# Set seed
torch.manual_seed(7)

# Create random tensor
tensor_D = torch.rand(size=(1, 1, 1, 10))

# Remove single dimensions
tensor_E = tensor_D.squeeze()

# Print out tensors
print(tensor_D, tensor_D.shape)
print(tensor_E, tensor_E.shape)

# Set seed torch.manual_seed(7) # Create random tensor tensor_D = torch.rand(size=(1, 1, 1, 10)) # Remove single dimensions tensor_E = tensor_D.squeeze() # Print out tensors print(tensor_D, tensor_D.shape) print(tensor_E, tensor_E.shape)

tensor([[[[0.5349, 0.1988, 0.6592, 0.6569, 0.2328, 0.4251, 0.2071, 0.6297,
           0.3653, 0.8513]]]]) torch.Size([1, 1, 1, 10])
tensor([0.5349, 0.1988, 0.6592, 0.6569, 0.2328, 0.4251, 0.2071, 0.6297, 0.3653,
        0.8513]) torch.Size([10])