TensorFlow Sparse Tensor + Examples

In this Python tutorial, we will learn how to use the sparse tensor in Python TensorFlow. Also, we will cover the following topics.

• TensorFlow sparsetensor to numpy
• TensorFlow sparse tensor to dense
• TensorFlow sparse_tensor_dense_matmul
• TensorFlow sparse tensor multiplication
• TensorFlow sparse tensor slice
• TensorFlow sparse tensor reshape
• TensorFlow sparse tensor gather
• TensorFlow manipulating sparse tensor
• TensorFlow tf.sparse_tensor_to_dense
• Module ‘TensorFlow’ has no attribute ‘sparse_tensor_dense_matmul’
• TensorFlow dataset sparsetensor
• TensorFlow concat sparse tensor

TensorFlow sparse tensor

• In this section, we will discuss how to use the sparse tensor in Python TensorFlow.
• In TensorFlow, sparse tensors enable efficient process and storage of tensors that contain most of the values zero and it is also used in natural language processing applications and for pre-processing images with a dark pixel in computer vision applications.
• To perform this particular task, we are going to use the tf.sparse.SparseTensor() function and this function is used to represent a sparse tensor.

Syntax:

Let’s have a look at the Syntax and understand the working of tf.sparse.SparseTensor() in Python TensorFlow.

tf.sparse.SparseTensor(
    indices, values, dense_shape
)

• It consists of a few parameters.
  • indices: This parameter indicates the indices of the values in the sparse tensor that stores non-zero values for example indices=[[2,9],[15,14]] indicate that the values with indexes have nonzero values.
  • values: This parameter specifies the input tensor and shape that supplies the values for each element in indices.
  • dense_shape: It specifies the dense shape of the sparse tensor and it also specifies the number of values in each dimension. Suppose you have a dense shape =[2,4] that indicates a two-dimensional 2*4.

Example:

Let’s take an example and check how to use the sparse tensor in Python TensorFlow.

Source Code:

import tensorflow as tf

indices=[[1, 0], [0, 1]]
values=[2, 3]
dense_shape=[3, 4]
result=tf.sparse.SparseTensor(indices, values, dense_shape)
print(result)

In the following given code we have imported the TensorFlow library and then created indices and dense shape values in the form of a list.

After that, we used the tf.sparse.SparseTensor() function and inside this function we have assigned the indices and dense_shape() value as an argument.

Here is the Screenshot of the following given code.

Read: TensorFlow Tensor to numpy

TensorFlow sparse tensor to numpy

• In this example, we will discuss how to convert sparse tensor values to numpy. To do this task we are going to use the eager_execution() function for running the session.
• First, we will import the TensorFlow library, and then we will create the session by using the tf.compat.v1.Session() function and then use the tf.sparse tensor() function.

Example:

Let’s take an example and check how to convert sparse tensor values to numpy.

Source Code:

import tensorflow as tf

tf.compat.v1.disable_eager_execution()
with tf.compat.v1.Session() as val:
    new_output = val.run(tf.SparseTensor([[2,3],[45,33]],[12,24],[3,2]))
    print(new_output)

In the above code we have imported the TensorFlow library and then created a sparse tensor by using the tf.sparseTensor() function and within this function, we assigned the integer values.

Here is the implementation of the following given code.

Read: Convert list to tensor TensorFlow

TensorFlow sparse tensor to dense

• In this section, we will discuss how to convert the sparse tensor to dense in Python TensorFlow.
• To do this task we are going to use the tf.sparse.to_dense() function and this function will help the user to convert a sparse tensor into a dense tensor.

Syntax:

Here is the Syntax of tf.sparse.to_dense() function in Python TensorFlow.

tf.sparse.to_dense(
    sp_input, default_value=None, validate_indices=True, name=None
)

• It consists of a few parameters.
  • sp_input: This parameter indicates the input Sparse Tensor.
  • default_value: By default, it takes the None value and it will set for indices not given in sp_input.
  • validate_indices: It is a boolean value and if it is true then indices are checked and there are no repeated values.
  • name: By default, it takes none value and specifies the name of the operation.

Example:

Let’s take an example and check how to convert the sparse tensor to dense in Python TensorFlow.

Source Code:

import tensorflow as tf

tens = tf.SparseTensor(dense_shape=[3, 5],values=[4, 2, 1],indices =[[0, 1],
            [0, 3],
            [2, 0]])
result=tf.sparse.to_dense(tens).numpy()
print(result)

In the following given code we have imported the TensorFlow library and then to convert the sparse tensor to dense, we have used the tf.sparse.to_dense() function.

Here is the implementation of the following given code.

Read: Python TensorFlow expand_dims

TensorFlow sparse_tensor_dense_matmul

• In this section, we will discuss how to multiply the sparse tensor in Python TensorFlow.
• To do this task, we are going to use the tf.sparse.sparse_dense_matmul() function and this method are used to multiply the sparse tensor by the dense tensor.

Syntax:

Let’s have a look at the Syntax and understand the working of tf.sparse.sparse_dense_matmul() function in Python TensorFlow.

tf.sparse.sparse_dense_matmul(
    sp_a, b, adjoint_a=False, adjoint_b=False, name=None
)

• It consists of a few parameters
  • sp_a: This parameter indicates the sparse tensor A.
  • b: It is a dense tensor with the same data type.
  • adjoint_a: By default, it takes a false value and is used for an adjoint of tensor A.
  • name: By default, it takes none value and specifies the name of the operation.

Example:

Let’s take an example and check how to multiply the sparse tensor in Python TensorFlow.

Source Code:

import tensorflow as tf

new_tens = tf.SparseTensor(indices=([1, 0], [0, 1], [0, 1]),
                       values=[67, 56, 74],
                       dense_shape=(2,2))

tensor = tf.constant([[2], [3]])
new_result = tf.sparse.sparse_dense_matmul(new_tens, tensor)

print(new_result)

Here is the Screenshot of the following given code

Read: Python TensorFlow truncated normal

TensorFlow sparse tensor slice

• In this example, we will discuss how to slice a sparse tensor in Python TensorFlow.
• To do this task, we are going to use the tf.sparse.slice() function and this function is used to slice a sparse tensor based on the size and start.

Syntax:

Here is the Syntax of tf.sparse.slice() function in Python TensorFlow.

tf.sparse.slice(
    sp_input, start, size, name=None
)

• It consists of a few parameters.
  • sp_input: This parameter indicates the input Sparse Tensor.
  • start: It represents the start of the slice
  • size: This parameter specifies the size of the tensor.
  • name: It is an optional parameter and it specifies the name of the operation.

Example:

Let’s take an example and check how to slice a sparse tensor in Python TensorFlow.

Source Code:

import tensorflow as tf

indices=[[1, 0], [0, 1]]
values=[2, 3]
dense_shape=[3, 4]
result=tf.sparse.SparseTensor(indices, values, dense_shape)
print(result)

Here is the Screenshot of the following given code.

Read: Python TensorFlow one_hot

TensorFlow sparse tensor reshape

• In this section, we will discuss how to reshape the sparse tensor in Python TensorFlow.
• To do this task, we are going to use the tf.sparse.reshape() method and this method is used to represent the value in a new dense shape and it specifies the reshape of a given sparse tensor.

Syntax:

Here is the Syntax of tf.sparse.reshape() function in Python TensorFlow.

tf.sparse.reshape(
    sp_input, shape, name=None
)

• It consists of a few parameters.
  • sp_input: This parameter indicates the input Sparse Tensor.
  • shape: It indicates the shape of the sparse tensor.
  • name: By default, it takes none value and indicates the name of the operation.

Example:

Let’s take an example and check how to reshape the sparse tensor in Python TensorFlow.

Source Code:

import tensorflow as tf
tf.compat.v1.disable_eager_execution()

new_tensor = tf.SparseTensor(indices=[[14, 25], [17, 44]],
                      values=[22, 31],
                      dense_shape=[3, 10])
result=tf.sparse.reshape(new_tensor,shape=[3,10])
with tf.compat.v1.Session() as val:
    new_output=val.run(result)
    print("Reshape of sparse tensor:",new_output)

In the above code, we have used the tf.sparse.reshape() function for getting the reshape of sparse tensor and within this function, we assigned the tensor as an argument.

Here is the execution of the following given code.

Read: Python TensorFlow random uniform

TensorFlow sparse tensor gather

• In this example, we will discuss how to gather the sparse tensor in Python TensorFlow.
• To perform this particular task, we are going to create a sparse tensor and then we are going to use the tf.gather() function.
• This function is used to break the input tensor based on the given indices.

Syntax:

Let’s have a look at the Syntax and understand the working of tf.gather() function in Python TensorFlow.

tf.gather(
    params,
    indices, 
    validate_indices=None, 
    axis=None, 
    batch_dims=0, 
    name=None
)

• It consists of a few parameters.
  • params: This parameter indicates that the tensor which we want to gather values the rank must be at least rank axis+1.
  • indices: This parameter specifies the index of input tensor and the values must be in the range of [0,params].
  • name: By default it takes none value and it specifies the name of the operation.

Example:

Let’s take an example and check how to gather the sparse tensor in Python TensorFlow.

Source Code:

import tensorflow as tf
tf.compat.v1.disable_eager_execution()

new_tens1 = tf.SparseTensor(indices=[[1, 3], [2, 1]],
                       values=[56, 25], 
                       dense_shape=[4, 10])

new_tens2 = tf.SparseTensor(indices=[[1, 1], [5, 2]],
                       values=[15, 28],
                       dense_shape=[4, 10])

result = tf.gather(new_tens1, new_tens2)

with tf.compat.v1.Session() as val:
    new_output=val.run(result)
    print("Manipulate sparse tensor:",new_output)

You can refer to the below Screenshot

As you can see in the Screenshot the output displays error that indicates tf.gather does not support sparse tensor as an input.

Read: Python TensorFlow reduce_mean

TensorFlow manipulating sparse tensor

• In this example, we are going to manipulate the sparse tensor in Python TensorFlow.
• To perform this particular task, we are going to use mathematical operations like tf.math.add() function and in this example, we will add a sparse tensor with the same shape and size.

Example:

Let’s take an example and check how to manipulate the sparse tensor in Python TensorFlow.

Source Code:

import tensorflow as tf
tf.compat.v1.disable_eager_execution()

new_tens1 = tf.SparseTensor(indices=[[1, 3], [2, 1]],
                       values=[56, 25], 
                       dense_shape=[4, 10])

new_tens2 = tf.SparseTensor(indices=[[1, 1], [5, 2]],
                       values=[15, 28],
                       dense_shape=[4, 10])

result = tf.sparse.add(new_tens1, new_tens2)
with tf.compat.v1.Session() as val:
    new_output=val.run(result)
    print("Manipulate sparse tensor:",new_output)

In the following given code, we have used the tf.sparse.add() function for manipulating the sparse tensor and then we used the tf.compat.v1.Session() for creating the session.

Here is the implementation of the following given code

Read: Python TensorFlow reduce_sum

TensorFlow tf.sparse_tensor_to_dense

• In this section, we will discuss how to use the tf.sparse_tensor_to_dense function in Python TensorFlow.
• This function is used to convert a Sparse tensor into a dense tensor. In Python TensorFlow, the sparse tensor is used to store a lot of zeros values and extensively in encoding schemes. While in the case of dense tensors they are used to store values in the form of a contiguous block of memory.

Syntax:

Let’s have a look at the Syntax and understand the working of tf.sparse_tensor_to_dense() in Python TensorFlow.

tf.sparse.to_dense(
    sp_input, default_value=None, validate_indices=True, name=None
)

• It consists of a few parameters
  • sp_input: This parameter indicates the input Sparse Tensor.
  • default_value: By default, it takes the None value and it will set for indices not given in sp_input.
  • validate_indices: It is a boolean value and if it is true then indices are checked and there are no repeated values.
  • name: By default, it takes none value and specifies the name of the operation.

Example:

Let’s take an example and check how to convert a sparse tensor into a dense tensor in Python TensorFlow.

Source Code:

import tensorflow as tf
tf.compat.v1.disable_eager_execution()

tens = tf.SparseTensor(dense_shape=[3, 5],values=[56, 13, 26],indices =[[0, 1],
            [0, 3],
            [2, 0]])
result=tf.sparse.to_dense(tens)

with tf.compat.v1.Session() as val:
    new_output=val.run(result)
    print("Sparse dense tensor:",new_output)

In the above code, we have imported the TensorFlow library and then we used the tf.SparseTensor() function and within this function we assigned the dens_shape() and indices values as an argument.

After that, we used the tf.sparse.to_dense() function for converting the sparse tensor into a dense tensor.

Here is the execution of the following given code.

Read: Import error no module named TensorFlow

Module ‘TensorFlow’ has no attribute ‘sparse_tensor_dense_matmul’

Here we are going to discuss the error module ‘TensorFlow’ has no attribute ‘sparse_tensor_dense_matmul’.

The reason behind this error is tf.sparse_tensor_dense_matmul() function does not support in TensorFlow.

Example:

import tensorflow as tf
tf.compat.v1.disable_eager_execution()

tens = tf.SparseTensor(dense_shape=[3, 3],values=[56, 13, 26],indices =[[0, 1],
            [0, 3],
            [2, 0]])
new_tens=tf.ones(shape=[3,3],dtype=tf.int32)
result=tf.sparse.tensor_dense_matmul(tens,new_tens)


with tf.compat.v1.Session() as val:
    new_output=val.run(result)
    print("Sparse dense tensor:",new_output)

Here is the Screenshot of the following given code.

Solution:

Here is the solution to this error

In this example, we are going to use the tf.sparse.sparse_dense_matmul() function for multiplication of sparse and dense tensor.

import tensorflow as tf
tf.compat.v1.disable_eager_execution()

tens = tf.SparseTensor(dense_shape=[3, 3],values=[56, 13, 26],indices =[[0, 1],
            [0, 3],
            [2, 0]])
new_tens=tf.ones(shape=[3,3],dtype=tf.int32)
result=tf.sparse.sparse_dense_matmul(tens,new_tens)
print(result)

In the above code we have imported the tensorflow library and then used the tf.sparseTensor() function for creating the sparse tensor and then we used the tf.ones() function for dense shape.

After that, we used the tf.sparse.sparse_dense_matmul() function. And within this function, we assigned the dense and sparse tensor as an argument.

Here is the implementation of the following given code.

Read: Binary Cross Entropy TensorFlow

TensorFlow Concat sparse tensor

• In this section, we will discuss how to concat sparse tensors in Python TensorFlow.
• To do this task, we are going to use the tf.sparse.concat() function and this function is used to combine a list of sparse tensors along with a given dimension.

Syntax:

Let’s have a look at the Syntax and understand the working of tf.sparse.concat() in Python TensorFlow.

tf.sparse.concat(
    axis, sp_inputs, expand_nonconcat_dims=False, name=None
)

• It consists of a few parameters
  • axis: This parameter indicates the number of dimensions of sparse tensors and also it specifies dimensions to combine along.
  • sp_inputs: This parameter indicates the input Sparse Tensor.
  • expand_nonconcat_dims: By default, it takes a False value and it will check the condition whether to allow the non-concat dimensions.
  • name: By default, it takes none value and specifies the name of the operation.

Example:

Let’s take an example and check how to concat sparse tensor in Python TensorFlow.

Source Code:

import tensorflow as tf
tf.compat.v1.disable_eager_execution()
input_tens1 = tf.SparseTensor(indices = [[1,2], [3,2], [1,4], [2,4], [2,3], [4,3]],
                         values = [45,12,3,45,6,8],
                         dense_shape = [8,5])
input_tens2 = tf.SparseTensor(indices = [[2,1], [3,1], [1,3], [2,0], [2,4], [2,5], [3,5], 
                                              [4,5], [5,0], [5,4], [5,5], [6,1], [6,3], [7,2]],
                         values = [67,15,14,21,78,44,23,19,27,34,19,45,96,45],
                         dense_shape = [8,6])
new_list = [input_tens1,input_tens2]
concat_sparse = tf.sparse.concat(axis=1, sp_inputs=new_list)
result=tf.sparse.to_dense(concat_sparse)
with tf.compat.v1.Session() as val:
    new_output=val.run(result)
    print("Concat of two tensors:",new_output)

In the following given code, we have imported the TensorFlow library and then created the list of sparse tensors and stored them into a ‘new_list’ variable. After that, we used the tf.sparse.concat() function for concatenating the sparse tensor.

Here is the implementation of the following given code

Read: TensorFlow clip_by_value

TensorFlow dataset sparsetensor

• In this section, we will discuss the dataset in a sparse tensor by using the Python TensorFlow.
• To perform this particular task, we are going to use the tf.data.Dataset.from_tensor_slices() function and this function is used when we have a large set of values or elements.
• First, we will create the sparse tensor by using the tf.SparseTensor() and within this function, we are going to assign the indices and dense_shape values as an argument.

Syntax:

Let’s have a look at the Syntax and understand the working of tf.data.Dataset() in Python TensorFlow.

tf.data.Dataset(
    variant_tensor
)

• It consists of only one parameter
  • variant_tensor: This parameter indicates the dataset and it is a DT_Variant tensor.

Example:

Let’s take an example and check how to use the dataset function in sparse tensor.

Source Code:

import tensorflow as tf
tf.compat.v1.disable_eager_execution()

input_tens1 = tf.SparseTensor(indices = [[1,2], [3,2], [1,4], [2,4], [2,3], [4,3]],
                         values = [45,12,3,45,6,8],
                         dense_shape = [8,5])
dataset = tf.data.Dataset.from_tensor_slices(input_tens1)
for element in dataset: 
  print(element)

In the above code, we have iterated over the dataset and processed the values. After that, we used the tf.data.Dataset.from_tensor_slices() function and within this function we assigned the sparse tensor as an argument.

You can refer to the below Screenshot.

Also, take a look at some more TensorFlow tutorials.

• TensorFlow Multiplication
• TensorFlow Get Variable
• TensorFlow Placeholder
• TensorFlow mean squared error
• Tensorflow custom loss function

So, in this Python tutorial, we have learned how to use the sparse tensor in Python TensorFlow. Also, we have covered the following topics.

• TensorFlow sparsetensor to numpy
• TensorFlow sparse tensor to dense
• TensorFlow sparse_tensor_dense_matmul
• TensorFlow sparse tensor multiplication
• TensorFlow sparse tensor slice
• TensorFlow sparse tensor reshape
• TensorFlow sparse tensor gather
• TensorFlow manipulating sparse tensor
• TensorFlow tf.sparse_tensor_to_dense
• module ‘TensorFlow’ has no attribute ‘sparse_tensor_dense_matmul’
• TensorFlow dataset sparsetensor
• TensorFlow concat sparse tensor