Convert a Pandas DataFrame to a TensorFlow Tensor

This TensorFlow tutorial will teach you how to convert pandas dataframe to tensor dataset.

I created a model in TensorFlow to predict housing prices. I collected data related to houses, such as square footage, location, and number of bedrooms, and then organized the collected data into a Pandas dataframe for preprocessing.

After preprocessing, I had to feed this data to a model, but it wasn’t in tensor format, as the model works on a tensor dataset. So, after going through TensorFlow documentation, I found two ways to convert the dataframe to a tensor.

Then, I converted the dataframe to tensor and fed to ml model. So, in this tutorial, I have explained both methods to convert dataframe to tensor.

Table of Contents:

Convert Pandas Dataframe to Tensor Dataset

To convert the pandas dataframe to a tensor, you can use TenosrFlow’s two functions, tf.convert_to_tensor, and tf.data.Dataset.from_tensor_slices().

Let’s start,

Convert Pandas Dataframe to Tensor Dataset using tf.convert_to_tensor Function

tf.convert_to_tensor function converts the given dataframe into tensor objects.

First, import the TensorFlow and pandas library using the code below.

import tensorflow as tf
import pandas as pd

Create a new data frame named df using df.pandasDataFrame() function, as shown in the code below.

df = pd.DataFrame({'Department1':[78,16,89],
                   'Department2': ['Science','Maths','Biology']})

View and check the created dataframe type using the code below.

print(df)
print('Type is :', type(df))

Look at the above output, which shows the dataframe and its type, which is ‘pandas.core.frame.DataFrame’. Now, we have to convert this panda dataframe to a tensor.

You must pass this dataframe (df) to function tf.convert_to_tensor() function, generating an error because the dataframe contains numeric and non-numeric data.

So, first, you will need to convert the non-numeric data into numeric; for that, you can use the one-hot encoding technique.

The code below can convert all the non-numerical data in the Department2 dataframe into one-hot encoding.

df_encoded = pd.get_dummies(df, columns=['Department2'])
print(df_encoded)

Now, you can see in the output all the data of the dataframe (df) encoded or converted into numeric form. Here, the code pd.get_dummies(df, columns=[‘Department2’]), called the get_dummies() function of pandas, converts the given column (Department2) values of the dataframe (df) into a numeric value.

Pass the encoded or converted data (numeric data ) to tf.convert_to_tensor function as shown below.

df_to_tensor = tf.convert_to_tensor(df_encoded, dtype=tf.float32)
print(df_to_tensor)

Look at the output; the dataframe (df_encoded) is converted into a tensor, as you can see.

This is how to convert the dataframe to tensor using the tf.convert_to_tensor function.

Convert Pandas Dataframe to Tensor Dataset using tf.data.Dataset.from_tensor_slices Function

The second way to convert the given dataframe to tensor dataset is to use the tf.data.Dataset.from_tensor_slices function.

Before converting pandas dataframe, you must understand the following things:

Each input tensor from tensors creates a dataset similar to a row of your dataset. In contrast, each input tensor from tensor slices creates a dataset identical to a column of your data. Therefore, in this case, all tensors must be the same length, and the elements (rows) of the resulting dataset are tuples with one element each.
Using tf.data as a resource, the slices of an array can be obtained as objects using the Dataset.from tensor slices() function and tf.data.

For example, let’s take the same dataframe which is shown below.

df = pd.DataFrame({'Department1':[78,16,89],
                   'Department2': ['Science','Maths','Biology']})

Next, convert the dataframe (df) into a tensor dataset using the code below.

df_to_tensordataset = tf.data.Dataset.from_tensor_slices(dict(df))
print(df_to_tensordataset)

Now print the dataset elements (df_to_tensordataset) using the code below.

for i in df_to_tensordataset.take(3):
    print(i)

convert pandas dataframe to tensorflow dataset

Look at the output; the dataframe (pdf) is converted into the tensor dataset.

Let’s decode the line tf.data.Dataset.from_tensor_slices(dict(df)). The from_tensor_slices method creates a Dataset whose elements are slices of the given dictionary (which is made from the DataFrame df). Basically, this dataset represents each row of the DataFrame as a separate element.

The next line of code ‘for i in df_to_tensordataset is take(3)’. This loop iterates through the first three dataset elements created from the dataframe (df). The take(3) method limits the iteration to the first three elements.

In each iteration, i is a dictionary where keys are column names from the dataframe (df), and values are tensor slices corresponding to each row.

This is how to convert the dataframe to tensor using the tf.data.Dataset.from_tensor_slices.

Conclusion

This TensorFlow tutorial taught you how to convert the dataframe to tensor using the tf.convert_to_tensor and tf.data.Dataset.from_tensor_slices.