**PyTorch** is a framework containing dependencies and libraries to build deep learning models. **Deep Learning (DL) models** use the neural network architecture to predict the future based on historical data. The historical data is authentic so it can be used to evaluate the performance of the model with the **loss** value using built-in loss functions. PyTorch also allows the user to **customize** their loss function according to the requirements

**Outline**

**What is a Custom Loss Function****How to Design a Custom Loss Function in Deep Learning Models****Advantages of Custom Loss Functions**

**What is a Custom Loss Function**

A **custom loss function** allows the user to calculate the wrongly predicted values according to their problem. To do so, the user needs to understand the problem at hand like **regression**, **classification**, **clustering**, or others. By understanding the problem, it becomes easy to design the customized loss function or even use the built-in ones effectively. Now let’s understand how to design a custom function using the PyTorch framework to enhance deep learning models.

The following is the **syntax** to define a basic customized loss function in the PyTorch framework:

```
import torch def custom_loss(inp, out):
loss = torch.mean((out - inp)/ out)
return loss
```

- Import the
**torch**library to get the functions required to build the loss function. - Now,
**define**the loss function followed by the name(**custom_loss**) of the function with the input(**inp**) and output(**out**) as arguments. - Here, the “
**inp**” variable contains the historical data, and the “**out**” stores the predicted data. - After that, provide the formula in the body to
**calculate**the loss value using its variable. - Finally,
**return**the loss value or the variable containing the loss values extracted from the formula

Call the **custom_loss()** function with the arguments to calculate the loss values while training the model.

**How to Design a Custom Loss Function in Deep Learning Models**

To build the deep learning model with the customized loss function, simply build or extract a dataset. After that, design and train the deep learning model using the dataset created using the torch library. With the training of the model, design the custom loss function and add it in the process to find the model’s performance.

To learn the process of designing the custom loss function in deep learning models in detail, go through the following steps:

**Step 1: Importing Libraries**

First, get the dependencies or functions by importing the **torch** library using the following code:

```
import torch #building deep learning models using neural networks
import torch.nn as nn #neural network dependency to get multiple methods
import torch.optim as optim #optimizer dependency to get methods like Adam or SGD
import matplotlib.pyplot as p #plot the results in the graphical representations
```

- Import the
**nn**dependency to design neural networks and**optim**to get the optimizer from the torch library. - After that, import the
**pyplot**API from the**matplotlib**library to design the graphs for visual representations.

**Step 2: Building Dataset**

Now, start building the dataset and store it in the input and output variables using the following code:

```
inp = torch.rand(10000, 1) * 10 #input values containing the actual data
out = 2 * inp + 1 + torch.randn(10000, 1) #output values containing the labeled data
```

- Create the “
**inp**” variable and initialize it with the**rand()**method to store values between 0 and 1. - Here, the second variable is the “
**out**” with random values using the**inp**variable in the mix as well. - The
**out**variable uses the**randn()**method that generates the values with**mean=0**and**sd=1**.

**Note**: **Preprocessing** is usually the next step after building the dataset for the deep learning model. However, we have created the data from scratch for the model, so we can skip this step and move forward with the model building. Additionally, this step becomes important when using the existing dataset, extracted from public libraries or built for other models.

**Step 3: Designing Neural Networks**

After that, build the structure of the neural network model to get the predictions using the given data:

```
class Model(nn.Module): # class to define Neural Network Model
def __init__(self): # constructor to initialize the Linear() model
super(Model, self).__init__()
self.linear = nn.Linear(1, 1) #Linear model to set the flow of the network
def forward(self, inp): #Feed-forward technique to start from the input layer
return self.linear(inp)
```

- Create a class named
**Model**with the Module argument from the**neural networks**. - In the class, define the constructor’s body with the
**Linear(1, 1)**method with a single input and output value. - Define a method called
**forward()**to apply the feed-forward technique for training the model.

**Step 4: Defining Loss Function**

Here, it is time to customize a loss function to evaluate the performance of the model designed in the previous step. Use the following code to define the **custom_loss()** function in the PyTorch framework:

```
def custom_loss(output, target): #defining the custom loss function with its method
return torch.mean(torch.abs((target - output) / target))
```

- Define the custom loss function using the
**def**keyword followed by the name of the function(**custom_loss)**. - After that, get two arguments in the custom_loss() method to get the
**output**value from the original data and the model’s**predicted**value. - Use the
**abs()**method from the torch library to get the**absolute**value (its distance from 0). - Use the formula from the loss value using the
**mean()**method to get the average loss value for each prediction. - Finally, return the extracted loss value by calling the customized loss function.

**Step 5: Calling the Optimizer**

Now, integrate all the components of the model and the optimizer in their variable using the following code:

```
model = Model()
optimizer = optim.SGD(model.parameters(), lr=0.01) #gradient descent optimizer with learning rate
```

- Create the
**model**variable and initialize it with the**Model()**class designed in step 3 of this guide. - It is done to easily train the model and also to avoid the complexities in the process.
- After that, call the
**SGD()**method and store it in the**optimizer**variable with the arguments of the optimizer. - The model’s
**parameters**are fine-tuned using the**learning rate(lr)**value given as the argument:

**Step 6: Training the Model**

Finally, use the following code to train the model on the given data to get the loss value from the customized function:

```
num_epochs = 1000
losses = []
for epoch in range(num_epochs):
predictions = model(inp) #Getting the predictions using the input data
loss = custom_loss(predictions, out) #calculating the loss value
optimizer.zero_grad() #applying the gradient descent to improve model
loss.backward() #backpropagating technique to improve loss value
optimizer.step()
losses.append(loss.item()) #add all the loss values in the list
if (epoch + 1) % 50 == 0:
print(f'Epoch [{epoch + 1}/{num_epochs}], Loss: {loss.item()}')
print("\nTraining Finished")
```

- Create a
**num_epochs**variable with**1000**values suggesting the number of iterations to train the model. - The
**losses**variable stores the loss values for each iteration in the form of a list. - Create a
**for**loop with the number of iterations to get the predictions and**loss**value of each iteration. - After that, call the optimizer to apply backpropagation using the
**zero_grad()**method to reduce the loss value. - Store all the loss values in the losses variable using the
**append()**method. - Finally,
**print**the**epoch**number with the**loss**value after 50 iterations as shown in the following screenshot:

The above screenshot displays that the loss value is decreasing with each training iteration. It means that the model is gradually improving its performance and it ends at **20%** after 1000 iterations. To understand it even further, plot the loss values on the graph.

**Step 7: Plotting the Loss Value**

To represent the values graphically, there are multiple graphs like scatter, bar, or line graphs to display the results effectively. Finally, display the loss values using the line graph by executing the following code:

```
p.plot(losses)
p.xlabel('Epochs')
p.ylabel('Loss')
p.title('Loss Curve')
p.show()
```

- Call the
**plot()**method with the**losses**variable containing the loss values for all the training iterations. - Use the
**xlabel()**and**ylabel()**methods to give the titles to the x and y-axis on the graph. - After that, use the
**title()**method to write the name of the graph. - Finally, use the
**show()**method to display the graph on the screen.

The above snippet shows that the loss value drops suddenly from almost 65% to 25% in 400 iterations. After that, it drops from 25% to 20% in the next 600 iterations suggesting that the model is optimized. If the user wants to improve it further, simply fine-tune the hyperparameters or preprocess the dataset.

**Advantages of Custom Loss Functions**

Some of the major advantages of building customized loss functions are mentioned below:

- It adds the flexibility aspect as the user has control over the calculating formula for the function and its use.
- Customizing the loss method improves the performance of the evaluation process of the model.
- The user can design a custom loss function to make it compatible with complex models with better accuracy.

That’s all about how to design a custom loss function in deep learning models using PyTorch.

**Conclusion**

To design the custom loss function in deep learning models using PyTorch, start by building the dataset and normalizing it. After that, design the deep learning model using neural networks to predict the future using the dataset. Then, define the customized loss function to improve the performance of the model using multiple training iterations.