How to Deploy AI Model with Django

In this Python tutorial, I will show you how to deploy an AI model with Django, This tutorial is based on the previous tutorial How to Add Items to Cart in Django in Python?

But deployment steps of the AI model are common you can use these steps in any Django project.

In the last 10 years, you have seen a lot of improvement with artificial Intelligence like self-driving cars or chatbots that can chat with a human or computer vision that can see or identify real-time things.

So one of the biggest problems for AI makers is to make it accessible to everyone, that’s where Django comes in, By integrating the AI model with Django a user can interact with AI and AI responds accordingly.

So in this tutorial, I will show you how to make a model using TensorFlow and deploy it with Django. Basically, you will understand how to build a product recommender model based on the description of the product and deploy it in your Django project to recommend the product.

Building AI model with Django

To build a product recommender model, first configure the project environment for TensorFlow or another library like Sickit learn which is required while building the model.

So open the terminal and install the following package:

Install TensorFlow.

pip install tensorflow

Then Install the scikit learn library which is used for machine learning.

pip install scikit-learn

Finally, Install Pandas.

pip install pandas

Now create a new folder named recommendations (you can give any name you want) within the Django app ‘profile_app’. In the recommendations folder create a new folder named dataset and add the dataset that you want to use for training the AI model.

In my case, I have added the product.csv file which contains information about the product name, description, price etc.

Now create a file called model.py in the folder recommendations of your Django app, and add the following lines of code to build the AI recommender model based on the description of the product.

import pandas as pd
import tensorflow as tf
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.preprocessing import StandardScaler
import joblib
import re

df = pd.read_csv('profile_app/recommendations/dataset/product.csv')
df = df.head(1000)
def preprocess_description(text):
    if not isinstance(text, str):
        return ""
    text = text.lower()
    text = re.sub(r'[^a-zA-Z\s]', '', text)
    return text
df['Processed_Description'] = df['description'].apply(preprocess_description)

tfidf_vectorizer = TfidfVectorizer(max_features=5000)
tfidf_matrix = tfidf_vectorizer.fit_transform(df['Processed_Description'])
scaler = StandardScaler(with_mean=False)  
tfidf_matrix = scaler.fit_transform(tfidf_matrix)
joblib.dump(tfidf_vectorizer, 'profile_app/recommendations/tfidf_vectorizer.pkl')
joblib.dump(tfidf_matrix, 'profile_app/recommendations/tfidf_matrix.pkl')
joblib.dump(scaler, 'profile_app/recommendations/scaler.pkl')

model = tf.keras.Sequential([
    tf.keras.layers.Dense(1024, activation='relu', input_shape=(tfidf_matrix.shape[1],)),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(512, activation='relu'),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(256, activation='relu'),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(tfidf_matrix.shape[1], activation='linear')
])

model.compile(optimizer='adam', loss='mse')

X_train, X_val = train_test_split(tfidf_matrix, test_size=0.2, random_state=42)
model.fit(X_train.toarray(), X_train.toarray(), epochs=20, batch_size=32, validation_data=(X_val.toarray(), X_val.toarray()))

model.save('profile_app/recommendations/recommender_model')

In the above code, first import the required function, model and libraries from lines 1 to 7. Then load the dataset (product.csv) using the read_csv() function of pandas, and store it as Pandas DataFrame into a variable df.

At line 10, limit the data to the first 1000 rows using the head(1000) function on the DatFrame df. The head() function of Pandas returns the first n-rows of the DataFrame and replaces the original DataFrame rows with the returned n-rows. So now the DataFrame df only contains the first 1000 rows.

Then from lines 11 to 16, a function named preprocess_description is defined which accepts a value text and this text is the description of the product. The purpose of this function is to clean and prepare the text data (description).

• As you know cleaning and preparing data are necessary steps in data analysis and machine learning. The dataset you have loaded is raw data and it comes with various issues such as noise, missing values, inconsistencies, and irrelevant information.
• Data preprocessing is necessary because it resolves all of these kinds of issues and makes the data suitable for data analysis and model training.

At line 12, first, it checks if the provided text is not a string, if it is not, then it returns an empty string. After this, at line 14 text is converted into lowercase using the text.lower() function on the text data.

Then at line 15, using regular expressions re.sub(r'[^a-zA-Z\s]’, ”, text) removes all the unnecessary punctuation, numbers, and other special characters from the text except the alphabet and spaces.

Finally, on line 17, apply the function preprocess_description using the function apply(preprocess_description) on the column ‘description’ of the dataframe df and storing the cleaned description in the new column ‘Processed_Descripton’ of the dataframe df.

Now at line 19 TfidfVectorizer(max_features=5000) initializing the TF-IDF vectorizer, the TF-IDF stands for Term Frequency-Inverse Document Frequency which is used for feature extraction. This tool takes the collection of text documents and converts them into a matrix of numerical values.

This matrix containing numerical values represents the importance of each word in the given document relative to the entire collection. The parameter max_features = 5000 means only included the top 5000 words or token (term).

• Then the code tfidf_vectorizer.fit_transform(df[‘Processed_Description’]) fits the TF-IDF vectorizer to the processed description that exists in the dataframe df in the column ‘Processed_Description’. Here fitting means transforming those descriptions into a matrix.
• After this, the code StandardScaler(with_mean=False) initialized the standard scaler to scale the dataset feature with a mean of 0 and a standard deviation equal to 1. The parameter (with_mean = False) means don’t subtract the mean because TF-IDF is the sparse matrix.
• When the mean is subtracted the sparse matrix becomes dense which can be memory intensive. Then the code scaler.fit_transform(tfidf_matrix) fits the scaler to the TF-IDF matrix and scales the matrix, after this, the values of the matrix become standardized values.
• The code joblib.dump() from lines 23 to 25 saves the object to the disk, so in this case, it saves the trained TF-IDF vectorizer, scaled TF-IDF matrix, and standard scaler in the folder recommendations.

In the above picture, the code from 27 to 37 defines the model using the tf.keras.Sequential() and this code create a sequential neural network model where layers are put after one another in the order they are added.

• Layers are created using tf.keras.layers.Dense(1024, activation=’relu’), these layers accept two parameters, the first is the number of neurons and the second is the activation function used by the neurons.
• Also, there is a third parameter provided only to the first layer which is the input_shape and it defines the shape of the input data which means a number of columns in the tfidf_matrix.
• Then one more type of layer is used which is tf.keras.layers.Drop(0.2). This layer is called the dropout layer and it is used to prevent overfitting. It randomly sets 20% (0.2) of the input units to 0 at each update during training by passing the value 0.2 as the parameter value.
• But if you look at the final layer which is the tf.keras.layers.Dense(tfidf_matrix.shape[1], activation=’linear’), the dense layer matches the number of neurons to the number of input features and this tells that the network is the autoencoder and trying to rebuild its input.

Next in line 39 compiling the model, the code model.compile(optimizer=’adam’, loss=’mse’) makes ready the model for training. The compile(optimizer, loss) takes two parameters optimizer which optimization algorithm such as Adam, and the loss function which is the mse (mean squared error) in this case.

• At line 41, splitting the dataset (tfidf_matrix) into training and validation set using the train_test_split(tfidf_matrix, test_size=0.2, random_state=42).
• The first parameter is the dataset that you want to split, the second, is the test_size = 0.2 which means use 20% of the dataset for model validation and the third is the random_state = 42 which means make sure the reproducibility of the split.

Then at line 42, train the model using the model.fit() method. This method trains the model on training data for 20 epochs, using batches of 32 samples at a time.

• The model tries to reconstruct its input, so both input and target data are X_train.toarray(). toarray() is used to convert the sparse matrix from the TF-IDF transformation into a dense array, which Keras requires. validation_data provides data for the model to validate its performance after each epoch.

Finally, at line 44, the trained model is saved using model.save() in the directory ‘profile_app/recommendations/recommender_model’.

Now you know how to load the dataset, preprocess the dataset, build the model, compile the model, train the model, and save the trained model in a specific directory.

Training and Saving Model with Django

Now run the python file model.py in your terminal and see in action how to models are trained and saved to your specified directory.

After training the model, the mode is saved in the directory recommender_model of the folder recommendations in your Django app as shown below.

Now you have trained and saved the trained model. So how are we going to use these models to recommend the product based on the description?

• Let me tell you whenever a user adds a product to the cart, that time using signals we will get the information about that product and extract the description of that product, and then using some function, the extracted product description will be fed to the model.
• After that model will predict a similar product and the output of the model will be the index or ID of the similar products, then using these predicted IDs of the products, we fetch all products and store them in the new table called Recommended Product.

First, create a model RecommendedProduct using the below code.

class RecommendedProduct(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    product = models.ForeignKey(Product, on_delete=models.CASCADE)
    def __str__(self):
        return f"Recommendation for {self.user.username}: {self.product.name}"

Then migrate the model RecommendedProduct by running the below command in your terminal.

python manage.py makemigratons
python manage.py migrate

Recommendation Functionality with Django

Now create a file called recommender_views.py in the recommendations folder of your Django app ‘profile_app’ and add the following lines of code.

import tensorflow as tf
import joblib
import re
from django.shortcuts import render
from ..models import Product, RecommendedProduct
from sklearn.metrics.pairwise import cosine_similarity
import numpy as np

model = tf.keras.models.load_model("profile_app/recommendations/recommender_model")
tfidf_vectorizer_product = joblib.load("profile_app/recommendations/tfidf_vectorizer.pkl")
tfidf_matrix = joblib.load('profile_app/recommendations/tfidf_matrix.pkl')
scaler = joblib.load('profile_app/recommendations/scaler.pkl')

def preprocess_description(text):
    if not isinstance(text, str):
        return ""
    text = text.lower()
    text = re.sub(r'[^a-zA-Z\s]', '', text)
    return text

In the above code, from lines 1 to 6 import the required functions, modules, and libraries. Then at line 8, load the model named ‘recommender_model’ from the directory ‘profile_app/recommendations’ using the method load_model of the tf.keras.

After this, from lines 10 to 12 load the tfidf_vectorizer, tfidf_matrix, and scaler objects using the load() method of the module joblib. From lines 14 to 19 define the preprocess_description function to clean and prepare the text which you already know.

Next, add the two functions get_recommendations and fetch_similar_products by adding the following lines of code.

def get_recommendations(description):
    preprocessed_description = preprocess_description(description)
    tfidf_vector = tfidf_vectorizer_product.transform([preprocessed_description])
    tfidf_vector = scaler.transform(tfidf_vector)
    embedding_or_output = model.predict(tfidf_vector.toarray())
    similar_products = fetch_similar_products(embedding_or_output)
    return similar_products

def fetch_similar_products(embedding_or_output, top_n=5):
    similarities = cosine_similarity(embedding_or_output, tfidf_matrix)
    top_indices = similarities[0].argsort()[-top_n:][::-1]
    indices = [i  for i in top_indices]
    similar_products = Product.objects.filter(id__in=indices)
    return similar_products

In the above code from lines 23 to 29, a function get_recommendations is defined which accepts a parameter description, this function performs the following operations:

• First, it takes the description and applies the preprocess_description function to remove unnecessary special characters, converting text to lowercase. This is the same function that you have defined while building the model.
• After this, the preprocessed description is converted into a TF-IDF vector using an object named tfidf_vectorizer_product.transform().
• Then this tfidf_vector is scaled using the object named scaler.transform(), The purpose of scaling the tfidf_vector is to keep the vector values in a particular range.
• Next, the scaled tfidf_vector is fed into the model using model.predict(tfidf_vector.toarray()) which predicts the output or embedding.
• This embedding could be a lower-dimensional representation of the product description or some feature extracted from the description that can be used to find similar products.
• Finally, the embedding or output is passed to the function fetch_similar_products() to get a list of similar products.

Now you know how the get_recommendations function works, let’s know about the second function fetch_similar_products from lines 31 to 36. This function gets all the products from the database that are similar to the input embedding.

• First, this function takes the embedding (or output) and tfidf_matrix and computes the cosine similarity between the given embedding and tfidf_matrix using the cosine_similarity() function.
• The cosine similarity is the measure of the similarity between two non-zero vectors. Here in this context, it is used to find how similar the product (represented by the embedding or output) is to other products (represented in tfidf_matrix).
• Then the line similarities[0].argsort()[-top_n:][::-1] extracts the top five similar product indices and stores them into variable top_indices. Then again all the top indices from the top_indices are copied to the new variable named indices.
• Finally, all the similar products are fetched from the Product table based on the ID of the product using Product.objects.filter(id__in=indices).

After defining the get_recommendations and fetch_similary_products functions, create a new file called signals.py in your Django app ‘profile_app’ and add the following code.

from django.db.models.signals import post_save
from .models import RecommendedProduct, CartItem
from .recommendations.recommeder_views import get_recommendations

def product_added_to_cart(sender, instance, created, **kwargs):
    if created:

        product_description = instance.product.description
        recommendations = get_recommendations(product_description)
        store_recommendations_for_user(instance.cart.user, recommendations)
post_save.connect(product_added_to_cart, sender=CartItem)


def store_recommendations_for_user(user, recommendations):
    RecommendedProduct.objects.filter(user=user).delete()
    for product in recommendations:
        RecommendedProduct.objects.create(user=user, product=product)

In the above code import the required model and functions from lines 1 to 2. It does the following things:

• The purpose of the above code is to get the description of the added product to the cart and use the get_recommendation function to get the recommended or similar products and store them in the table RecommendedProduct table in the database using the function store_recommendations_for_uer.
• The above code is called signal which is triggered whenever the user adds the product to its cart.

If you don’t know “What is signal and how it works”, then refer to the tutorial How to Create Signal in Django

You have done all the required steps of deploying the AI model with Django, Now Let’s create a view called recommended_product_page in the file recommender_views.py in the folder recommendations.

Add the following line of code.

def recommended_product_page(request):
    recommendations = RecommendedProduct.objects.filter(user=request.user)
    return render(request, 'product_recommend.html', { 'recommendations': recommendations})

The above code from lines 38 to 41, retrieves all the products of the currently logged-in user from the table RecommendedProduct and passes to the template ‘product_recommend.html’ as recommendations.

Also, create a new template product_recommend.html in the templates folder of your Django app ‘profile_app’ and add the following codes.

{% extends "home.html" %}
{% load static %}
{% block content %}
<link rel="stylesheet" href="{% static 'css/product_list.css' %}">
<h1>Recommended Products</h1>

<div class="product-container">
    {% for product in recommendations %}
    <div class="product-card">
        <div class="product-image">
            <img src="{{ product.product.image }}" alt="{{ product.name }}">
        </div>
        <div class="product-name">{{ product.product.name }}</div>
        <div class="product-description">{{ product.product.description |truncatechars:50 }}</div>
        <div class="product-price">&#8377; {{ product.product.price }}</div>
        <form action="{% url 'add-to-cart' product.product.id %}" method="post">
            {% csrf_token %}
            <button class="add-to-cart-btn">Add to Cart</button>
        </form>
    </div>
    {% endfor %}
</div>

{% endblock %}

The above code is the template that will show all the recommended products on the user page. Also add URLs for this view, so open urls.py file of your Django app ‘profile_app’ and add the following URLs.

path('recommendations/', recommended_product_page, name='product-page'),

Now test test the deployed model, so run the below command to start the Django server.

python manage.py runserver

After login, you click on the product link and you see a page like this.

Now add any product to the cart by clicking on the Add to Cart button as shown in the below picture.

In the above picture, for example, add the boots to the cart. After adding the boots, let’s go to the recommended page to see the recommended boots. So open the URL http://127.0.0.1:8000/recommendations/ in your terminal and you will see the page like this.

As you can see in the above picture, the model recommends two boots of different types. Well, you have successfully deployed the AI model with Django and it’s working.

But remember for some products it may not recommend the right product but you don’t need to worry. The sole purpose of this tutorial is to show you the steps of how you can deploy any AI model in your Django project.

If a model doesn’t yield the right output, it all depends upon how you train your model, You can again retrain your model with different parameters so that I can predict the accurate result.

If you want to know about TensorFlow or how to build an AI model, then refer to our tutorials about Tensorflow in Python.

Conclusion

In this Python tutorial, you learned how to deploy an AI model with Django, you also learned how to build an AI model using TensorFlow from scratch, save it to your local system, and then load it to make predictions. Finally, you build the recommender system that takes the product description and recommends a similar product to the user.

You may like to read:

• How to Build Chat App in Django
• Routers and ViewSets in Django Rest Framework