How to Calculate L2/MSE Loss in PyTorch?

by | Dec 20, 2023 | Internet

Neural Network algorithms can perform different tasks like classification and clustering to predict the future of different events using the dataset. The model needs to use evaluation methods to check its performance and make it even better by using loss or accuracy functions. Configuring the correct loss function is very important because it allows the user to make the model according to their choice. PyTorch provides different loss functions like MSE, MAE, Dice, and many more to evaluate and optimize the Deep Learning model.

Quick Outline

This guide explains the following sections:

  • What is L2/MSE Loss
  • How to Calculate L2/MSE Loss in PyTorch
  • Method 1: Calculate L2/MSE Loss Using MSELoss() Function
  • Method 2: Calculate L2/MSE Loss of Trained Model
  • Method 3: Calculate L2/MSE Loss Using PyTorch Functions
  • Conclusion

What is L2/MSE Loss

Mean Squared Error (MSE) also known as L2 evaluates the loss function by taking the squared difference of the input and output values. The MSE finds the sum of the squared difference values for all the data points available in the dataset. After that, it will find the mean of the squared difference by dividing it by the value of n (total number of data points in the dataset). The f mathematical representation of the Mean Squared Error or MSE is mentioned below:

Here:

MSE: Mean Squared Error.

X: Actual Values.

Y: Predicted Values.

n: Total number of data points.

i: number of iterations for summation.

How to Calculate L2/MSE Loss in PyTorch

To calculate the L2 or MSE loss in PyTorch, use the built-in methods from the torch library with the functional dependency of the environment. The user can also build the deep learning model and find the MSE loss values to optimize its performance. Numpy is another library that can be used to find the MSE or L2 loss value of the binary classification as the observed values range from -1 to 1. To learn the process of calculating the L2 or MSE loss in PyTorch, go through the following sections:

Note: The Python code to calculate the L2 or MSE loss is available here

Method 1: Calculate L2/MSE Loss Using MSELoss() Function

The numpy library can also be used to calculate the MSE or L2 loss values using the observed and predicted tensors. To learn the process of calculating loss value, implement the following steps:

Step 1: Access Python Notebook

To write the code in the Python language, the user needs to create a notebook such as Jupyter or Google Colab. This guide uses the Google Colab Notebook to write the Python code that can accessed from the official website:

Step 2: Install Modules

In the Google Colab Notebook, simply install the torch module by executing the following command:

pip install torch

Step 3: Importing Libraries

Firstly, import the required libraries like pandas, numpy, torch, and matplotlib to call the functions for creating and analyzing the data. The torch library can be used to call the MSELoss() method and then plot the values on the graph using the matplotlib library:

import pandas as pdimport torch
import torch.nn as nn
import numpy as np
import matplotlib.pyplot as plt

%matplotlib inline

Create the tensors with the input tensor containing the values between -1 to 1 and the predicted tensors only have the zeros stored. Create the graph using these values using the numpy() method with both the tensors:

x = torch.Tensor(np.linspace(-1, 1, 100))
y = torch.Tensor(np.zeros(100))

plt.plot(x.numpy(), y.numpy(), '.-');

Output

The following screenshot displays the horizontal line of dots with an x-axis range between -1 to 1 and the y-axis at 0 for all the values:

Step 4: Calculating Loss

Now, calculate the loss value using the MSELoss() method with the values of x and y tensors:

criterion = nn.MSELoss()
criterion(x, y)

Once the loss values are extracted, simply plot the values for all the instances on the graph using the following code:

plt.plot(x.numpy(), (x.numpy()-y.numpy())**2);
plt.title('MSE - L2 Loss')
plt.xlabel('true y');
plt.ylabel('predicated y');

The code:

  • Applies the squared formula for calculating the MSE loss in the plot() function to display their values on the graph.
  • The xlabel() takes the true values and the ylabel() takes the predicted values on the graph:

Output

The graph changes shape from a horizontal line to a V-shaped graph after the application of the MSELoss() method on the data:

After calculating the MSE loss values on two randomly built tensors, head into the next method to train a model and optimize its performance using MSE loss value:

Method 2: Calculate L2/MSE Loss of Trained Model

This method uses the sequential model that takes one input and produces a single output but it can be optimized with multiple iterations. The transitional processes are sequentially arranged in this model and all of them will be trained for each iteration to produce a new loss value. To implement this method, simply go through the following steps:

Step 1: Building Sequential Model

Start the process by importing the libraries required to build the sequential model to solve the regression problem and plot the loss values on a graph. The data set is gathered from the “datasets” dependency of the sklearn library with SGD optimizer and Denser layer for the neural network model. The matplotlib library is used to import pyplot for building the graphs to visualize the optimization of the model:

from sklearn.datasets import make_regression
from sklearn.preprocessing import StandardScaler
from keras.models import Sequential
from keras.layers import Dense
from keras.optimizers import SGD
from matplotlib import pyplot

Build the x variable to store the sample set of actual values from the dataset and the y variable to store the predicted values extracted from the model. Apply the StandardScaler() method to remove the mean values and take each value from the dataset as a unit:

X, y = make_regression(n_samples=1000, n_features=20, noise=0.1, random_state=1)

X = StandardScaler().fit_transform(X)
y = StandardScaler().fit_transform(y.reshape(len(y),1))[:,0]

Split the sample data into training/testing sets for both observed and predicted values and store it in another variable like 500 in the n_train variable:

n_train = 500
trainX, testX = X[:n_train, :], X[n_train:, :]
trainy, testy = y[:n_train], y[n_train:]

The following code builds the neural network model with its layers, neurons, and optimizations with the help of the learning rate. Implement the following code to design and build the model with all its intermediary processes to get enhanced results:

model = Sequential()
model.add(Dense(25, input_dim=20, activation='relu', kernel_initializer='he_uniform'))
model.add(Dense(1, activation='linear'))
opt = SGD(learning_rate=0.01, momentum=0.9)
model.compile(loss='mean_squared_error', optimizer=opt)

The code:

  • Calls the Sequential() method in the model variable and then adds the layers in the model.
  • The neural network contains 20 neurons and every neuron is connected with each of the neurons from the next layer as the model is sequential.
  • Add the activation function called relu and he_uniform as the model initializer to get on with the processes in the model.
  • Add another layer with the one neuron and linear activation function to get optimized results for the multi-layer neural network.
  • Now that the model is ready to work, provide the values for learning_rate and momentum so it can start learning the hidden patterns from the dataset.
  • Calculate the loss value using the mean_squared_error while evaluating the performance of the model:

Step 2: Training the Model

Train the model using the actual and predicted data using their variables inside the fit() method. The model also contains the epochs value which means that the model would go through the data set 100 times. Training the model 100 times makes the model more accurate with the predictions by understanding the hidden patterns of the dataset:

history = model.fit(trainX, trainy, validation_data=(testX, testy), epochs=100, verbose=0)

Once the model is trained and predicted some values, simply call the evaluate() to store the values of the train and test datasets:

train_mse = model.evaluate(trainX, trainy, verbose=0)
test_mse = model.evaluate(testX, testy, verbose=0)
print('Train: %.3f, Test: %.3f' % (train_mse, test_mse))

Simply print the evaluated values for the training and testing values as displayed in the screenshot below:

Step 3: Plotting MSE/L2 Loss

Once the loss values for both training and testing data are evaluated, simply plot the graph for loss values from the first epoch to 100 using the following code:

pyplot.title('Mean Squared Error - L2 Loss')
pyplot.plot(history.history['loss'], label='train')
pyplot.plot(history.history['val_loss'], label='test')
pyplot.legend()
pyplot.show()

The above code:

  • Calls the title() method to print the name of the graph on the screen
  • The plot() method uses the loss values from the train and test values to plot the line graph
  • Applies the legend() method to plot the elements on the graph and show() method to print the graph on the screen:

The following graph shows both the lines for the training and testing loss values which keep improving as the epochs increase. It means that the model performs well with each epoch and understands the actual insights from the data:

Method 3: Calculate L2/MSE Loss Using PyTorch Functions

The first method uses the PyTorch functional dependency to call the mse_loss() method to calculate the L2 or MSE loss value. The mse_loss() is the built-in method from the torch environment and it also enables the user to customize their MSE loss function to calculate the loss value. The pairwise_distance() and numpy can be used to build the customized function and the following steps are used to implement the code:

Step 1: Import Libraries

After the successful installation of the module, import its library so that the session can use its functions provided by the torch module:

import torch

Verify the successful installation of the torch module by printing its version using the following command:

print(torch.__version__)

Step 2: Implement PyTorch Code

Once the torch environment is set up, import the numpy and functional libraries for using their respective functions to calculate the loss value. The following code uses three types of functions and calculates the MSE or L2 loss values for all of them:

import torch.nn.functional as F

x1 = torch.rand(10,2)
x2 = torch.rand(10,2)
mse_torch =F.mse_loss(x1,x2)
print(" Functional MSE Loss:", mse_torch)

mse_manual = 0
x3 = torch.zeros(10,2)
for i in range(len(x1)) :
  x3[i,:1] +=(F.pairwise_distance(x1[i,:1],x2[i,:1],eps=0.0)**2)/len(x1)
  x3[i,1:] += (F.pairwise_distance(x1[i,1:],x2[i,1:],eps=0.0)**2)/len(x1)
  mse_manual += x3[i]
print("\n Pairwise MSE Loss:", mse_manual.mean())

mse_manual = 0
for i in range(len(x1)) :
  mse_manual += np.square(x1[i]-x2[i]) / len(x1)
print("\n Manual MSE Loss:", mse_manual.mean())

The above code:

  • Builds a couple of tensors like x1 and x2 to store the actual and predicted values to evaluate the model’s performance using the MSE loss value.
  • Create another variable called x3 to store the mean value by applying the pairwise_distance() method to the x1 and x2 tensors.
  • The numpy library is used to call the square() method to build the formula for the MSE loss and store the loss value in the mse_manual variable.
  • Calculating the MSE loss value for the last two methods uses the for loop to keep all the values in the data set into account and find the accurate loss value:

That’s all about the process of calculating the L2 or MSE loss in PyTorch.

Conclusion

To calculate the Mean Squared Error or L2 loss value in PyTorch, simply use the MSELoss() method or mse_loss with functional dependency. The deep learning models are to be evaluated using difference loss functions and MSE is used for regression problems. While compiling the neural network model in deep learning, the model can be evaluated using the mean_squared_error value in the loss argument. This guide has evaluated the process of calculating the MSE or L2 loss value in PyTorch and for the deep learning learning model.