PyTorch Binary Cross Entropy

In this Python tutorial, we will learn about PyTorch Binary Cross Entropy in python and we will also cover different examples related to Binary Cross Entropy. Moreover, we will cover these topics.

• PyTorch Binary cross entropy
• PyTorch Binary cross entropy example
• PyTorch Binary cross entropy with logits
• PyTorch Binary cross entropy weight
• PyTorch Binary cross entropy loss function
• PyTorch Binary cross entropy pos_weight
• PyTorch Binary cross entropy sigmoid

PyTorch Binary cross entropy

In this section, we will learn about the PyTorch binary cross entropy in python.

It creates a norm that calculates the Binary cross entropy between the target probabilities and input probabilities.

Syntax:

The following syntax of Binary cross entropy in PyTorch:

torch.nn.BCELoss(weight=None,size_average=None,reduce=None,reduction='mean)

Parameters:

1. weight A recomputing weight is given to the loss of every element.
2. size_average The losses are averaged over every loss element in the batch.
3. reduce The losses are mean observations for every minibatch depending upon the size_average.
4. reduction state the reduction applied to the output: ‘none’, ‘mean’, ‘sum’.
   • ‘none’ is defined as no reduction will be applied.
   • ‘mean’ is defined as the sum of the output will be divided by the number of element in the output.
   • ‘sum’ is defined as the given output will be summed.

Read PyTorch Model Summary

PyTorch Binary cross entropy example

In this section, we will learn about how to implement binary cross entropy with the help of an example in PyTorch.

The norm is created which calculates the binary cross entropy between the target and input probabilities. It is also used for calculating the error of reconstruction.

Code:

In the following code, we will import the torch module from which we can calculate the binary cross entropy.

• x = nn.Sigmoid() is used to ensure that the output of the unit is in between 0 and 1.
• loss = nn.BCELoss() is used to calculate the binary cross entropy loss.
• input_prob = torch.randn(4, requires_grad=True) is used to calculate the input probability.
• target_prob = torch.empty(4).random_(3) is used to calculate the target probability.
• output_prob = loss(x(input_prob), target_prob) is used to get the output probability.
• print(output_prob) is used to print the output probability on the screen.

import torch
import torch.nn as nn
x = nn.Sigmoid()
loss = nn.BCELoss()
input_prob = torch.randn(4, requires_grad=True)
target_prob = torch.empty(4).random_(3)
output_prob = loss(x(input_prob), target_prob)
output_prob.backward()
print(output_prob)

Output:

After running the above code, we get the following output in which we can see that the binary cross entropy value is printed on the screen.

Read: Keras Vs PyTorch – Key Differences

PyTorch Binary cross entropy with logits

In this section, we will learn about the PyTorch Binary cross entropy with logits in python.

Binary cross entropy contrasts each of the predicted probability to actual output which can be 0 or 1.

It also computes the score that deals with the probability based on the distance from the expected value.

Code:

In the following code, we will import the torch module from which we can compute the binary cross entropy with logits.

• Bceloss = nn.BCEWithLogitsLoss() is used to calculate the binary cross entropy logit loss.
• inp_var = torch.randn(7, requires_grad=True) is used as an input variable.
• tar_var = torch.empty(7).random_(6) is used as an target variable.
• out_var = Bceloss(inp_var, tar_var) is used to calculate the output variable.
• print(out_var) is used to print the output variable.

import torch
import torch.nn as nn
Bceloss = nn.BCEWithLogitsLoss()
inp_var = torch.randn(7, requires_grad=True)
tar_var = torch.empty(7).random_(6)
out_var = Bceloss(inp_var, tar_var)
out_var.backward()
print(out_var)

Output:

In the following output, we can see that the PyTorch binary cross entropy with logits value is printed on the screen.

Read: PyTorch MSELoss – Detailed Guide

PyTorch Binary cross entropy weight

In this section, we will learn about the PyTorch binary cross entropy weight in python.

It creates the criterion that measures the binary cross entropy between the input and the target probabilities. The weight in the binary cross entropy is iteratively adjustable

Code:

In the following code, we will import the torch module from which we can calculate the binary cross entropy weight.

• critrion = nn.BCEWithLogitsLoss(pos_weight=poswight) is used to calculate the binary cross entropy.
• loss = critrion(a, b) is used to calculate the loss.
• weight = torch.ones_like(lossrw) is used to generate weight.
• critrionrw_sig = nn.BCELoss(reduction=’none’) is used to create criterion raw sigmoid.
• lossrw_sig = critrionrw_sig(torch.sigmoid(a), b) is used to calculate loss raw sigmoid.
• print(loss – lossrw) is used to print the loss value on the screen.

import torch
import torch.nn as nn
for x in range(70):
    poswight = torch.randint(1, 100, (1,)).float()
    
    critrion = nn.BCEWithLogitsLoss(pos_weight=poswight)
    a = torch.randn(15, 3, requires_grad=True)
    b = torch.randint(0, 4, (15, 3)).float()
    
    loss = critrion(a, b)
    
    critrionrw = nn.BCEWithLogitsLoss(reduction='none')
    lossrw = critrionrw(a, b)
    weight = torch.ones_like(lossrw)
    weight[b==1.] = poswight
    lossrw = (lossrw * weight).mean()
    
    critrionrw_sig = nn.BCELoss(reduction='none')
    lossrw_sig = critrionrw_sig(torch.sigmoid(a), b)
    lossrw_sig = ((lossrw_sig) * weight).mean()

    print(loss - lossrw)    
    print(lossrw_sig - loss)

Output:

After running the above code, we get the following output in which we can see that the PyTorch binary cross entropy weight values are printed on the screen.

Read: PyTorch Batch Normalization

PyTorch Binary cross entropy loss function

In this section, we will learn about the PyTorch cross-entropy loss function in python.

Binary cross entropy is a loss function that compares each of the predicted probabilities to actual output that can be either 0 or 1.

Code:

In the following code, we will import the torch module from which we can calculate the binary cross entropy loss function.

• ypredic = num.array([0.1582, 0.4139, 0.2287]) is used predict the y value.
• ttlbce_loss = num.sum(-ytrue * num.log(ypredic) – (1 – ytrue) * num.log(1 – ypredic)) is used to calculate the total binary cross entropy loss value.
• numbof_sampls = ypredic.shape[0] is used to get the number of sample.
• meanbce_loss = ttlbce_loss / numbof_sampls is used to calculate the mean value.
• print (“BCE error is: ” + str(bceval)) is used to print the binary cross entropy error.

import torch
import torch.nn as nn
import numpy as num
ypredic = num.array([0.1582, 0.4139, 0.2287])
ytrue = num.array([0.0, 0.0, 1.0]) 
def BCE(ypredic, ytrue):
    ttlbce_loss = num.sum(-ytrue * num.log(ypredic) - (1 - ytrue) * num.log(1 - ypredic))
    # Getting the mean BCE loss
    numbof_sampls = ypredic.shape[0]
    meanbce_loss = ttlbce_loss / numbof_sampls
    return meanbce_loss
bceval = BCE(ypredic, ytrue)
print ("BCE error is: " + str(bceval))

Output:

After running the above code, we get the following output in which we can see that the binary cross entropy loss value is printed on the screen.

Read: PyTorch Load Model + Examples

PyTorch Binary cross entropy pos_weight

In this section, we will learn about the PyTorch cross-entropy pos_weight in python.

The pos_weight is defined as a weight of positive examples. It must be a vector and the length equal to several classes.

Code:

In the following code, we will import some modules from which we can calculate the weight of positive examples.

• tar_var = torch.ones([15, 69], dtype=torch.float32) is used calculate the target value with batch size is 15 and 69 classes.
• out_var = torch.full([15, 69], 2.0) is used as an prediction logit.
• posweight = torch.ones([69]) is used as weight of positive example and all the weights are equal to 1.
• critrion(out_var, tar_var) is used to create a criterion that measures the binary cross entropy.

import torch
import torch.nn as nn
tar_var = torch.ones([15, 69], dtype=torch.float32)  
out_var = torch.full([15, 69], 2.0)  
posweight = torch.ones([69])  
critrion = torch.nn.BCEWithLogitsLoss(pos_weight=posweight)
critrion(out_var, tar_var)  

Output:

In the following output, we can see that the PyTorch cross-entropy pos_weight value in which all the weights are equal to 1 is printed on the screen.

Read: PyTorch nn linear + Examples

PyTorch Binary cross entropy sigmoid

In this section, we will learn about the PyTorch Binary cross entropy sigmoid in python.

The sigmoid function is a real function that defined all the input values and has a non-negative derivative at each point.

Code:

In the following code, we will import the torch module from which we can calculate the binary cross entropy sigmoid.

• inp_var = torch.randn(5, 4, requires_grad=True) is used to generate the input variables.
• tar_var = torch.rand(5, 4, requires_grad=False) is used to generate the target variables.
• loss = fun.binary_cross_entropy(torch.sigmoid(inp_var), tar_var) is used to calculate the binary cross entropy sigmoid function.
• print(loss) is used to print the loss value on the screen.

import torch
from torch.nn import functional as fun
inp_var = torch.randn(5, 4, requires_grad=True)
tar_var = torch.rand(5, 4, requires_grad=False)
loss = fun.binary_cross_entropy(torch.sigmoid(inp_var), tar_var)
loss.backward()
print(loss)

Output:

After running the above code, we get the following output in which we can see that the PyTorch binary cross entropy sigmoid value is printed on the screen.

You may also like to read the following Python tutorials.

• Adam optimizer PyTorch
• PyTorch Activation Function
• Cross Entropy Loss PyTorch
• PyTorch Tensor to Numpy
• Jax Vs PyTorch [Key Differences]
• PyTorch Save Model

So, in this tutorial, we discussed PyTorch binary cross entropy and we have also covered different examples related to its implementation. Here is the list of examples that we have covered.

• PyTorch Binary cross entropy
• PyTorch Binary cross entropy example
• PyTorch Binary cross entropy with logits
• PyTorch Binary cross entropy weight
• PyTorch Binary cross entropy loss function
• PyTorch Binary cross entropy pos_weight
• PyTorch Binary cross entropy sigmoid