Tensor in TensorFlow

Tensors in TensorFlow are a basic building block of the TensorFlow framework; knowing about tensors is very important.

In this tensorflow tutorial, I will explain the ins and outs of tensors, such as ‘What is a tensor?’, how it is constructed, and the mathematical concepts behind it, such as scalar, vector and matrix.

Whether a beginner or a Python expert, you can easily understand tensor concepts as they are explained through Python concepts.

After this, you will learn about the properties of tensors, such as their shape, dimension, and datatype. Then, you will learn how to create a tensor in tensorflow using the function. Finally, you will learn about the rank of tensor with an example.

What is a tensor in TensorFlow?

A tensor is a generalized matrix form; if you are a mathematician or physicist, you probably know very well about a matrix. Also, you may have heard terms like scalars and vectors; if you are familiar with these terms, you somehow know about tensors.

As you know, a matrix consists of rows and columns; for example, a two-dimensional matrix contains an array of numbers; see below.

[ 3, 5,7 ]
[ 4, 6, 2]

The above matrix has two rows and three columns; as you know, the matrix represents the data. You can even manipulate this matrix data using mathematical operations such as multiplying two matrices, adding numbers to matrix elements, etc.

But why is a tensor called a generalized matrix form? A tensor allows you to work with more than two dimensions, whereas a matrix is limited to rows and columns. So, that is why the tensor is a generalized matrix form.

First, let me explain the definition of the matrix; from the above example, you can say a matrix is a collection of elements arranged in a two-dimensional array.

Here, understand the two terms;

  • Elements can be numbers such as integers, decimals, real numbers, etc. These elements are kept in rows and columns.
  • Two-dimensional arrays are rows and columns,
    • Rows are the horizontal lines of elements in a matrix. If the matrix contains m rows, it has m horizontal lines of elements.
    • Columns are the vertical lines of elements in the matrix. If a matrix has n columns, that means the matrix contains n vertical lines of elements.

Create arrays and matrices in Python; arrays are represented using the list; for example, look at the code below.

If you don’t know how to create a Python list, visit the tutorial How to Create a List in Python [+5 Examples].

array = [ 3, 5, 7, 4, 6, 2]

The above is an array containing six different elements of data type, an integer, also called a 1-dimensional array or vector, because there is one horizontal line of elements.

Also, what is the shape of this array? The shape of the array refers to the number of elements it has in each dimension, so this array has a 1-Dimension, which contains the 6 elements, so its shape is (6).

It becomes a matrix if you have one list into another list or include one list within another. For example, look at the code below.

matrix = [
  [ 3, 5,7 ],
  [ 4, 6, 2]
]

The above matrix contains two lists: the [3, 5, 7] and the second [4, 6, 2].

As you can see, each array or list is separated by a comma, and each inner list represents the row of the matrix. Again, the datatype of this matrix is an integer.

It is called a 2-dimensional matrix because there are two horizontal lines of elements and three vertical lines of elements. Also, what is the shape of this matrix? Here, the shape is the number of rows and columns.

So it has 2-dimension; as you can see, this matrix contains two rows, each containing three elements and three columns, each containing two elements. So its shape is (2,3), which is 2 rows and 3 columns.

  • Remember, shape and dimensions are two concepts you will soon learn in creating tensors section.

You can even create a three-dimensional matrix, as shown below.

matrix = [
  [ 3, 5,7 ],
  [ 4, 6, 2],
  [1, 6, 5 ]
]

The above is a 3-D matrix containing 3 rows and 3 columns, so its shape is (3,3). I hope you understand the concept of an array or matrix and how to create it in Python.

Coming back to the concept of tensor, according to Tensorflow documentation, the definition of a tensor is ‘tensor represents the multidimensional array of elements’, which means a tensor is the same as a matrix. Still, you can work on higher dimensions using the tensor, even 4-D, 5-D, or n-D.

So here, the easy definition of a tensor is ‘ A tensor is a collection of arrays organized in n-dimensions.

  • Collection of Array: Tensor comprises one or more arrays (called lists in Python). These arrays (lists) can be numbers, symbols, or other data formats.
  • Organized in N-Dimensions: The essential characteristic of a tensor is its ability to have any number of dimensions (n-dimensions). Using this concept, tensors can represent lines of data(1-D) and tables of data like matrices (2-D), but they can also represent higher data like cubes of data (3-D) or even higher dimensions.

So, if you are a Python programmer and know only about lists in Python, which represent arrays, you can say a tensor is a collection of lists (nested lists), and each list corresponds to one dimension. Having more lists means more dimensions.

Understanding the Dimensions and Shape of Tensor in TensorFlow

Dimension of tensor is how many indices are required to locate a specific element in a tensor. In tensorflow, dimensions are the rank of a tensor. This dimension differs from the one you learned in the mathematics geometry chapter.

So, let’s understand Rank,

A single number is called a Rank 0 Tensor; indices are not required for this kind of tensor. For example, look at below, which is a scalar value.

3  // this is the single number, or any single number is called Rank 0 Tensor

A 1-dimensional array is called a Rank 1 Tensor (Vector); this tensor requires one index; for example, look at below, which is a 1-D Tensor.

[2,5,7]

A 2-dimensional array is called a Rank 2 Tensor (Matrix); this tensor requires two indices. For example, look at the below, which is a 2-D Tensor.

[
  [ 3, 5,7 ],
  [ 4, 6, 2]
]

A 3-dimensional array is called a Rank 3 Tensor; this tensor requires three indices; for example, look at the below, which is a 3-D Tensor.

[
  [ 3, 5,7 ],
  [ 4, 6, 2],
  [1, 6, 5 ]
]

An N-dimensional array is called a Rank N Tensor; this tensor requires n-indices, also called an N-D Tensor.

This is all about dimension in tensorflow, but you will understand it better while creating a tensor.

Next, let’s know ‘What is the tensor shape in tensorflow?’.

Shape of Tensor in TensorFlow

As you already know, shape is the number of elements along each dimension. In tensorflow, the shape is represented using the tuple ‘()’. For example, look at the examples below to determine the shape of the tensors.

Take a single number such as shown below.

8 

The above is called Rank 0 Tensor, which is a scalar value, so the shape for this value is an empty tuple like this ‘()’ because it has no dimensions.

Take a list of numbers as shown below.

[2,5,7]

This list of numbers is called Rank 1 Tensor, a vector, so it has the shape of (3, ), meaning it has 3 elements in one dimension.

Take a nested list of numbers as shown below.

[
  [ 3, 5,7 ],
  [ 4, 6, 2]
]

The above-nested list is called Rank 2 Tensor, a matrix, so it has the shape of (2, 3), meaning it has 2 rows and 3 columns.

For Higher-Rank Tensors, the shape is going to have more elements. For example, a Rank 3 tensor with shape (2, 3, 4) has 2 elements in the first dimension, 3 in the second, and 4 in the third.

Now, let’s move on and see how to create a tensor in a tensorflow and find the shape and dimensions of that tensor.

Creating Tensor in TensorFlow

In TensorFlow, there are several functions that you can use to create a tensor with specific characteristics.

The first function is the tf.constant() function in TensorFlow, so the syntax of this function is given below.

tf.constant(value, dtype=None, shape=None, name='const')

Before explaining the parameter of the tf.constant() function, I want to show you how to create a tensor and then explain each parameter one by one.

To create a tensor in tensorflow, import the TensorFlow library and give it an alternate name as tf; look at the code below.

import tensorflow as tf

Access the constant() function of TensorFlow using the dot operator (.), and specify the single value as 20 within the constant() function as shown below.

scalar = tf.constant(20)

Use Python’s print() function and print the scalar’s value as shown below.

print(scalar)
Creating Scalar Tensor in Tensorflow usinig tf.constant() function

Here, you have created the Rank 0 Tensor (called scalar tensor), a scalar value with the shape of the empty tuple ().

Let me explain, to the tf.constant(20), a single number is passed, and this single number is scalar, so it doesn’t have any dimensions or shape.

Now look at the output, which is tf.Tensor(20, shape=(), dtype=int32), the first value in this output is 20, which is a single number; the second is the shape, which is equal to an empty tuple (), and the third is the dtype, which is the data type of this number, here type is int32 which represent an integer.

The value, shape, and dtype are the properties of the tensor, which are inferred automatically based on the passed value. You can customize this property while passing the value to the tf.constant() function.

For example, pass the value 10.5 and give a shape as an empty tuple (); because this is a single number, dtype as float32, as shown below.

float_scalar = tf.constant(10.5, shape=(), dtype=tf.float32)

print(float_scalar)
Creating float Tensor in Tensorflow usinig tf.constant function

You can see in the output the float_scalar tensor has a value of 10.5, the shape is an empty tuple, and the data type of this value is float32.

Next, let’s create a vector or Rank 1 tensor; for that, pass the list of values [3, 5, 6, 2] to the tf.constant() function, as shown below.

vector = tf.constant([3, 5, 6, 2])

print(vector)
Creating Rank 1 Tensor in Tensorflow usinig tf.constant() function

As you can see, the values of this vector are [3, 5, 6, 2], it has the shape of (4,), which means it contains 4 elements, and the dtype is int32, which data type of each element is an integer.

You can access each property individually, such as vector. shape, and vector.dtype, as shown below.

print('Shape of the vector', vector.shape)

print('Data type of elements', vector.dtype)
Accessing Properties of Tensor in Tensorflow usinig tf.constant() function

From the output, you can print or access the property value separately.

It also has one dimension because you need to specify one index to find any element in the vector. For example, run the code below to access the element at index 1.

print(vector[1])
Accessing Value at index of Tensor in Tensorflow

You can see that element 5 is at index 1, so here you have specified index value using brackets [], and you can access any element through the index. As I told you, a tensor’s dimension is the tensor’s rank.

So, tensorflow provides a function called tf.rank(); using this function, you can find the rank of the given tensor. For example, to find the rank of the above vector, use the code below.

print(tf.rank(vector))
Rank of Tensor in Tensorflow usinig tf.rank() function

Look at the tf.rank() and return the rank of the vector as 1, which means it has one dimension.

Let’s create a Rank 2 tensor (called a matrix tensor). For example, you have a nested list like this [ [3, 2, 6], [8, 2, 6] ], and pass this nested list to tf.constant() function as shown below.

matrix_tensor = tf.constant([ [3, 2, 6], [8, 2, 6] ])

print(matrix_tensor)
Creating Matrix Tensor in Tensorflow using tf.constant() function

You can see above is the matrix tensor, which contains the values as [ [3, 2, 6], [8, 2, 6] ], has the shape of (2, 3), and the dtype is int32.

Now, let’s access the properties such as shape and dtype using the code below.

print('Shape of the matrix is', matrix_tensor.shape)

print('Data type of matrix elements are', matrix_tensor.dtype)
Accessing Properties of Matrix Tensor in Tensorflow using tf.shape and tf.dtype

To access the element of the matrix tensor, you need two indices. For example, let’s access the element at index [1][2] as shown in the below code.

print(matrix_tensor[1][2])
Accessing Element at Index of Matrix Tensor in Tensorflow

The element at index [1][2] is 6. You can see in the output that you have specified two index values in separate brackets.

Also, check the rank of this matrix tensor; for that, use the code below.

print(tf.rank(matrix_tensor))
Rank of Matrix Tensor in Tensorflow

As you can see, the rank of the matrix tensor is 2, which means it has two dimensions.

Finally, create a Rank 3 tensor which is higher in dimensions; for example, pass the nested list of the list [ [[3, 2, 6]], [[8, 2, 6]],[[1, 9, 3]] ] to tf.constant() function as shown below.

rank_3_tensor = tf.constant([ [[3, 2, 6]], [[8, 2, 6]],[[1, 9 , 3]] ])

Check the shape and data type of the rank_3_tensor using the code below.

print('Shape of the rank_3_tensor is', rank_3_tensor.shape)

print('Data type of rank_3_tensor elements are', rank_3_tensor.dtype)
Accessing Properties of Rank 3 Tensor in Tensorflow

The shape of the Rank 3 tensor is (3, 1, 3), and the dtype is int32.

To access the elements of this Rank 3 tensor, you need to use three indices; for example, locate or access the element at index [0][0][0] as shown below.

print(rank_3_tensor[0][0][0])
Accessing Value at index of Rank 3 Tensor in Tensorflow

The element at index [0][0][0] is 3; here, three indexes are specified using the brackets.

Again, check the rank of the Rank 3 tensor using the code below.

print(tf.rank(rank_3_tensor))
Checking Dimension of Rank 3 Tensor in Tensorflow

The rank of the Rank 3 tensor is 3, which has 3 dimensions, as you can see in the output.

This is how to create a rank 0, 1, 2, and 3 tensors using the tf.constant() function in TensorFlow. You can even create higher tensors like rank 4, 5, etc.

  • Remember, when you create the tensor in tensorflow using the tf.constant(), you can’t change the value of that tensor after creating it, as the function’s name implies that you use it when you don’t want to change the value of the tensor throughout the computation.

But, as you know, while building a neural network, you may need a tensor that can updated as a requirement arises; for that, there is another function to create that kind of tensor, so visit the tutorial Variable in Tensorflow to know about that function.

Conclusion

You learned how to create a TensorFlow using the tf.constants() function; you have learned how the tensor is derived where you have learned the rank of tensors such as Rank 0, 1, 2 tensor.

You also learned to find a tensor’s shape, dimension or datatype. You created different types of tensors, such as scalar, vector, and matrix tensors and even learned how to create higher-dimension tensors.

You may like to read: