Binary Cross Entropy TensorFlow

In this Python tutorial, we will learn how to calculate a Binary Cross-Entropy loss in Python TensorFlow. Also, we will cover the following topics.

• Binary Cross Entropy TensorFlow
• Weighted binary cross entropy TensorFlow
• Binary_cross_entropy_with_logits TensorFlow
• TensorFlow binary cross-entropy sigmoid
• Sparse binary cross-entropy TensorFlow
• Binary Cross Entropy loss function TensorFlow

Binary Cross entropy TensorFlow

• In this section, we will discuss how to calculate a Binary Cross-Entropy loss in Python TensorFlow.
• To perform this particular task we are going to use the tf.Keras.losses.BinaryCrossentropy() function and this method is used to generate the cross-entropy loss between predicted values and actual values.
• In TensorFlow, the binary Cross-Entropy loss is used when there are only two label classes and it also comprises actual labels and predicted labels.

Syntax:

Let’s have a look at the Syntax and understand the working of tf.Keras.losses.BinaryCrossentropy() in Python TensorFlow.

tf.keras.losses.BinaryCrossentropy(
    from_logits=False,
    label_smoothing=0.0,
    axis=-1,
    reduction=losses_utils.ReductionV2.AUTO,
    name='binary_crossentropy'
)

• It consists of a few parameters.
  • from_logits: This parameter indicates the logit values and it contains probabilities values that are [0,1].
  • label_smoothing: By default, it takes 0.0 values and it will check the condition when it is greater than 0 and compute the loss between the true values.
  • axis: By default, it takes a -1 value and the axis along which to generate cross-entropy.
  • name: By default, it is binary_crossentropy value and it specifies the name of the operation.

Example:

import tensorflow as tf

new_values = [[1, 0, 1, 1, 0, 0], [1, 1, 0, 1, 0, 0], [1, 0, 1, 1, 0, 0]]
new_values2 = [[.2, .1, .8, .7, .1, .2],[.3, .4, .5, .6, .7, .7], [.1, .2, .3, .4, .5, .6] ]
new_bin_cross_entropy = tf.keras.losses.BinaryCrossentropy()
result = new_bin_cross_entropy(new_values, new_values2).numpy()
print(result)

In the following given code, we have imported the TensorFlow library and then created actual values and predicted values. After that, we have used the tf.keras.losses.BinaryCrossentropy() function and within this function, we have assigned the predicted and true values in it.

Here is the implementation of the following given code.

Read: TensorFlow Multiplication

Weighted binary cross entropy TensorFlow

• In this section, we will discuss how to use the weight parameter in the BinaryCrossentropy function in Python TensorFlow.
• In this example, we have mentioned the weights in tf.Keras.losses.BinaryCrossentropy() function and this function is used to generate the cross-entropy loss between predicted values and actual values.

Syntax:

Here is the Syntax of tf.Keras.losses.BinaryCrossentropy() in Python TensorFlow

tf.keras.losses.BinaryCrossentropy(
    from_logits=False,
    label_smoothing=0.0,
    axis=-1,
    reduction=losses_utils.ReductionV2.AUTO,
    name='binary_crossentropy'
)

Example:

Let’s take an example and check how to use the weight parameter in the BinaryCrossentropy function in Python TensorFlow.

Source Code:

import tensorflow as tf

new_act_val = [[1, 0], [1, 1]]
new_pred_val = [[-23, 56], [1.92, -24.6]]
weight=[0.8, 0.2]
new_result = tf.keras.losses.BinaryCrossentropy(from_logits=True)
new_result(new_act_val, new_pred_val).numpy()
new_result(new_act_val, new_pred_val, weight).numpy()

Here is the Screenshot of the following given code.

Read: TensorFlow Get Variable + Examples

Binary_cross_entropy_with_logits TensorFlow

• In this Program, we will discuss how to use the binary cross-entropy with logits in Python TensorFlow.
• To do this task we are going to use the tf.nn.sigmoid_cross_entropy_with_logits() function and this function is used to calculate the cross-entropy with given logits.
• If you want to find the sigmoid cross-entropy between logits and labels. To do this task we are going to use the tf.nn.sigmoid_cross_entropy_with_logits() function.

Example:

Let’s take an example of how to use the binary cross-entropy with logits in Python TensorFlow.

Source Code:

import tensorflow as tf
tf.compat.v1.disable_eager_execution()

new_logit = tf.constant([0, 0., 1., -1., 0., 1., 1.,0.])
new_label = tf.constant([1., 1., 1., 0., 0., 1., 0.,1.])
new_output=tf.nn.sigmoid_cross_entropy_with_logits(
    labels=new_label, logits=new_logit)
with tf.compat.v1.Session() as val:
    new_result=val.run(new_output)
    print(new_result)

Here is the Output of the following given code.

Read: Module ‘tensorflow’ has no attribute ‘log’

TensorFlow binary cross-entropy sigmoid

• In this section, we will discuss how to use the sigmoid in binary cross-entropy in Python TensorFlow.
• To perform this particular task we are going to use the tf.nn.sigmoid_cross_entropy_with_logits() function and within this function, we have mentioned the sigmoid_logits values and this will calculate cross-entropy with labels.

Syntax:

Here is the Syntax of tf.nn.sigmoid_cross_entropy_with_logits() in Python TensorFlow.

tf.nn.sigmoid_cross_entropy_with_logits(
    labels=None, logits=None, name=None
)

• It consists of a few parameters
  • labels: This parameter indicates the tensor of the same type and the values are between 0 and 1.
  • logits: By default, it takes none value and specifies the real number.
  • name: This parameter indicates the name of the operation.

Example:

Let’s take an example and check how to use the sigmoid in binary cross-entropy in Python TensorFlow.

Source Code:

import tensorflow as tf
tf.compat.v1.disable_eager_execution()

sigmoid_logits = tf.constant([0., -1., 1.,0])
soft_binary_labels = tf.constant([0., 1., 1.,0.])
new_logit = tf.constant([0, 0., 1., -1., 0., 1., 1.,0.])
new_label = tf.constant([1., 1., 1., 0., 0., 1., 0.,1.])
new_output=tf.nn.sigmoid_cross_entropy_with_logits(
    labels=soft_binary_labels, logits=sigmoid_logits)
with tf.compat.v1.Session() as val:
    new_result=val.run(new_output)
    print(new_result)

In the following given code, we have imported the TensorFlow library and then created a sigmoid value by using the tf.constant() function and within this function, we have assigned the 0 and 1 values.

After that, we have used the tf.nn.sigmoid_cross_entropy_with_logits() function and within this function we assigned the labels and logits values.

Here is the Screenshot of the following given code.

Read: Tensorflow iterate over tensor

Sparse binary cross entropy TensorFlow

• In this section, we will discuss how to sparse the binary cross-entropy in Python TensorFlow.
• To perform this particular task we are going to use the tf.Keras.losses.SparseCategoricalCrossentropy() function and this function will calculate the cross-entropy loss between the prediction and labels.

Syntax:

Let’s have a look at the Syntax and understand the working of tf.Keras.losses.SparseCategoricalCrossentropy() function in Python TensorFlow.

tf.keras.losses.SparseCategoricalCrossentropy(
    from_logits=False,
    reduction=losses_utils.ReductionV2.AUTO,
    name='sparse_categorical_crossentropy'
)

• It consists of a few parameters
  • from_logits: This parameter indicates the logit values and it contains probabilities values that are [0,1].
  • name: By default, it takes sparse_categorical_crossentropy value and it specifies the name of the operation.

Example:

Let’s take an example and check how to sparse the binary cross-entropy in Python TensorFlow.

Source Code:

import tensorflow as tf

new_true_value = [0,1]
new_pred_value = [[1, 0.32, 1], [0, 1.0, 0.23]]

result = tf.keras.losses.SparseCategoricalCrossentropy()
new_output=result(new_true_value, new_pred_value)
print(new_output)

You can refer to the below Screenshot.

Read: Convert list to tensor TensorFlow

Binary cross entropy loss function TensorFlow

• In this section, we will discuss how to use the loss function in binary cross entropy by using Python TensorFlow.
• By using the tf.keras.losses.BinaryCrossentropy() function we are going to assign the actual and predicted values in it.

Example:

Let’s take an example and check how to use the loss function in binary cross entropy by using Python TensorFlow.

Source Code:

import tensorflow as tf

new_true = [[1.,0.],
         [1.,1.]]
new_predict = [[0.9,1.0],
         [0.3,1.0]]

new_binar_cross = tf.keras.losses.BinaryCrossentropy()

result=new_binar_cross(new_true,new_predict)
print(result)

Here is the implementation of the following given code.

You may also like to read the following TensorFlow tutorials.

• Python TensorFlow expand_dims
• TensorFlow global average pooling
• Batch Normalization TensorFlow
• Python TensorFlow truncated normal
• Python TensorFlow random uniform
• Python TensorFlow reduce_sum
• PyTorch Numpy to Tensor

So, in this tutorial, we have learned how to calculate a Binary Cross-Entropy loss in Python TensorFlow. Also, we have covered the following topics.

• Binary Cross entropy TensorFlow
• Weighted binary cross entropy TensorFlow
• Binary_cross_entropy_with_logits TensorFlow
• TensorFlow binary cross-entropy sigmoid
• Sparse binary cross-entropy TensorFlow
• Binary cross-entropy loss function TensorFlow