How to Find Binary Cross Entropy in PyTorch?

by | Nov 22, 2023 | How-To/Tips

Binary cross entropy is a logarithmic loss function utilized in machine learning. It computes the difference between the estimated probabilities of a model and actual labels of data. Sometimes, users may need to find/compute the binary cross entropy between two (input and target) tensors. PyTorch provides the “BCELoss()” method to perform this operation. 

This blog will demonstrate the method to compute/find binary cross entropy in PyTorch.

How to Find Binary Cross Entropy in PyTorch?

To find the binary cross entropy in PyTorch, follow the below-listed steps:

  • Import required libraries 
  • Define and print input tensor
  • Define and print target tensor
  • Define a criterion to compute binary cross-entropy
  • Find/compute binary cross-entropy
  • Print calculated binary cross-entropy

Step 1: Install Required Libraries 

First, install the “torch” library and “torch.nn” module that will be utilized to measure binary cross entropy:

import torch
import torch.nn as nn

Step 2: Define Input Tensor 

Then, define the input tensor using the “torch.tensor()” function and display its elements. For example, we are making simple “input_tens” tensor:

input_tens = torch.tensor([[0.123, 0.984, 0.452],
                          [0.977, 0.491, 0.830],
                          [0.421, 0.382, 0.628]],
                          requires_grad=True)

print('input: \n', input_tens)

This has created an input tensor:

Step 3: Define Target Tensor 

Next, define the target tensor and display its elements. Here, we are defining a “target_tens” tensor:

target_tens = torch.tensor([[0.289, 0.999, 0.693],
                            [0.899, 0.231, 0.384],
                            [0.452, 0.071, 0.049], ])

print('target: \n', target_tens)

This has created the target tensor:

 Note: The value of target tensor elements must be between 0 and 1.

Step 4: Define Criterion

Now, use the “BCELoss()” method to create a criterion for computing binary cross entropy:

bce_loss_cr = nn.BCELoss()

Step 5: Find/Compute Binary Cross Entropy

Next, measure the binary cross entropy by passing the “input_tens” and “target_tens”:

output = bce_loss_cr(input_tens, target_tens)
output.backward()

Step 6: Print Binary Cross Entropy

Finally, display computed binary cross-entropy loss:

print('Binary Cross Entropy Loss: ', output)

The below output shows the binary cross-entropy loss:

Note: Click on the provided link to access our Google Colab Notebook.

Conclusion

To find the binary cross entropy in PyTorch, first, import the necessary torch libraries. Then, create the input and target tensors, and print their elements. After that, use the “BCELoss()” method to create a criterion for computing binary cross entropy. Next, measure the binary cross entropy loss and display it. This blog has demonstrated the method to compute/find binary cross entropy in PyTorch.