PyTorch Linear Regression [With 7 Useful Examples]

In this Python tutorial, we will learn about the PyTorch Linear Regression. The PyTorch Linear Regression is a process that finds the linear relationship between the dependent and independent variables by decreasing the distance. And additionally, we will also cover the different examples related to the PyTorch Linear Regression. And also covers these topics.

• PyTorch linear regression
• PyTorch linear regression from scratch
• PyTorch linear regression dataloaders
• PyTorch linear regression loss
• PyTorch linear regression gradient descent
• PyTorch linear regression with regularization
• PyTorch linear regression accuracy

PyTorch linear regression

In this section, we will learn about the PyTorch linear regression in python.

Linear regression is a supervised machine learning approach that finds the best fit linear line between the dependent and independent variables. It also finds the linear relationship between dependent and independent variables.

The equation of linear regression:

Y = Ax+b

Here Y is the dependent variable, x is the independent variable, b is the y-intercept and A is the coefficient of the slope.

So, with this, we understood the PyTorch linear regression.

Read: Cross Entropy Loss PyTorch

PyTorch linear regression from scratch

In this section, we will learn about the PyTorch linear regression from scratch in python.

The linear regression establishes a linear relationship between the dependent and independent variables.

In Linear regression, we build a model and predict the relationship between the dependent and independent variables.

Code:

In the following code, we firstly import all the necessary libraries such as import torch and import Variables from torch.autograd.

• Xdt = Variable(torch.Tensor([[2.0], [4.0], [6.0]])): Here we are defining the variable Xdt (Xdata). Here the Xdt is the independent variable.
• Ydt = Variable(torch.Tensor([[4.0], [8.0], [12.0]])): Here we are defining the variable Ydt (Ydata). Here the Ydt is the dependent variable and this will be our dataset for now.
• class Linearregressionmodel(torch.nn.Module): The model is a subclass of torch.nn.Module.
• self.linear = torch.nn.Linear(1, 1): Here we have one one input and on output is the argument of torch.nn.Linear() function.
• Model = Linearregressionmodel() is used to create an object for linear regression model.
• criterion = torch.nn.MSELoss(size_average = False) : Here we will use the mean square error as our loss function.
• optimizer = torch.optim.SGD(Model.parameters(), lr = 0.01): Here we are using Stochastic gradient descent as our optimizer and we are arbitrarily fixing a learning rate of 0.01.
• for epoch in range(200): Here we are appearing at our training step where we are performing the following task 200 times during training and performing forward pass by passing our data.
• newvariable = Variable(torch.Tensor([[6.0]])) is used to to test if we get the correct results using the model we define. Test for the unknown values as 6.0.
• print(“predict (After Train the Model)”, Model(newvariable).item()) is used to print the predicted value.

# Importing Libraries
import torch 
from torch.autograd import Variable 

# Describing the variable
Xdt = Variable(torch.Tensor([[2.0], [4.0], [6.0]])) 
Ydt = Variable(torch.Tensor([[4.0], [8.0], [12.0]])) 

# Initializing the model and declaring the forward pass
class Linearregressionmodel(torch.nn.Module):
    def __init__(self): 
        super(Linearregressionmodel, self).__init__()
        # One input and one output 
        self.linear = torch.nn.Linear(1, 1)    

    def forward(self, y): 
        Ypred = self.linear(y) 
        return Ypred 

# Creating an object for linear regression model
Model = Linearregressionmodel() 

# Select optimizerand loss criteria
criterion = torch.nn.MSELoss(size_average = False) 
optimizer = torch.optim.SGD(Model.parameters(), lr = 0.01) 

# Training the model
for epoch in range(200): 
    # Forward pass: Compute predicted y by passing x to the model 
    predy = Model(Xdt) 
    # Compute and print loss 
    loss = criterion(predy, Ydt) 
    # Zero gradients, perform a backward pass, And update the weights. 
    optimizer.zero_grad() 
    loss.backward() 
    optimizer.step() 
    print('epoch {}, loss {}'.format(epoch, loss.item())) 

# Test if getting the correct result using the model
newvariable = Variable(torch.Tensor([[6.0]])) 
predy = Model(newvariable) 
print("predict (After Train the Model)", Model(newvariable).item()) 

Output:

After running the above code, we get the following output in which we can see that our model inherently learns the relationship between the input data and output data without being programmed explicitly.

So with this, we understood about the PyTorch linear regression from scratch.

Read: PyTorch Early Stopping

PyTorch linear regression dataloaders

In this section, we will learn about the PyTorch linear regression dataloaders in python.

In Linear regression output label is indicated as a linear function of input features that uses weights and bias and these weights and bias are the model parameters.

Code:

In the following code, firstly we will import all the necessary libraries such as import torch, and import numpy as np.

• inputvar = np.array([[75, 69, 45], [93, 90, 66],[89, 136, 60],[104, 45, 39]], dtype=’float32′) : Here we are loading the data.
• inputvar = torch.from_numpy(inputvar) is used to convert the numpy array to torch tensor.
• print(inputvar) is used to print the input values.
• dataset = TensorDataset(inputvar, targetvar) is used to create tensor dataset.
• weight = torch.randn(2, 3, requires_grad=True) is used to define the weight variable by using torch.randn() function.
• bias = torch.randn(2, requires_grad=True) is used to define the bias variable by using torch.randn() function.
• print(weight) is used to print the weight.
• print(bias) is used to print the bias.
• preds = model(a) is used to print the model output.

# Importing libraries
import torch
import numpy as np

# Loading the data
inputvar = np.array([[75, 69, 45], 
                   [93, 90, 66], 
                   [89, 136, 60], 
                   [104, 45, 39]],
                  dtype='float32')
targetvar = np.array([[66, 80], 
                    [91, 111], 
                    [129, 143], 
                    [32, 47]],  
                   dtype='float32')
# Convert to the torch tensor
inputvar = torch.from_numpy(inputvar)
targetvar = torch.from_numpy(targetvar)
# Print the inputs and outputs
print(inputvar)
print(targetvar)

# Create a tensordataset
from torch.utils.data import TensorDataset
dataset = TensorDataset(inputvar, targetvar)
dataset[:2]

# Using Dataloaders class
from torch.utils.data import DataLoader

batchsiz = 2
trainingloader = DataLoader(dataset, batch_size=batchsiz, shuffle=True)
# Acess the data from the dataloaders
 a Batch Sample
for input,tar in trainingloader:
    print(input)
    print(tar)
    break

# Define a set of weights and bias
weight = torch.randn(2, 3, requires_grad=True)
bias = torch.randn(2, requires_grad=True)
print(weight)
print(bias)

# Equation of linear regression
def model(y):
    return y @ weight.t() + bias

# Predict the model output
for a,b in trainingloader:
    preds = model(a)
    print("Prediction is :n",preds)
    print("nActual targets is :n",b)
    break

Output:

In the below output, you can see that the PyTorch linear regression datloaders calculate the predictions and actual targets, and the values of the prediction and actual targets are printed on the screen.

This is how we can use the Pytorch linear regression dataloaders for calculating prediction and actual targets.

Read: PyTorch MSELoss

PyTorch linear regression loss

In this section, we will learn about the PyTorch linear regression loss in python.

In Linear regression loss function is used to calculate the performance of the model and how well our model is performing. The loss function is also used to improve the hyperparameters so that the resulting value of the loss will be less.

Code:

In the following code, firstly we will import all the necessary libraries such as import torch, import numpy as np.

• def MSEloss(predict, targ): is used to define the loss function.
• difference = predict – targ : Here mean square error defines the mean of the square of the difference between the actual and predicted values.
• print(“n Loss is: “,MSEloss(preds, b)) is used to print the loss.

# Importing libraries
import torch
import numpy as np

# Loading the data
inputvar = np.array([[75, 69, 45], 
                   [93, 90, 66], 
                   [89, 136, 60], 
                   [104, 45, 39]],
                  dtype='float32')
targetvar = np.array([[66, 80], 
                    [91, 111], 
                    [129, 143], 
                    [32, 47]],  
                   dtype='float32')
# Convert to the torch tensor
inputvar = torch.from_numpy(inputvar)
targetvar = torch.from_numpy(targetvar)
# Print the inputs and outputs
print(inputvar)
print(targetvar)

# Create a tensordataset
from torch.utils.data import TensorDataset
dataset = TensorDataset(inputvar, targetvar)
dataset[:2]

# Using Dataloaders class
from torch.utils.data import DataLoader

batchsiz = 2
trainingloader = DataLoader(dataset, batch_size=batchsiz, shuffle=True)
# Acess the data from the dataloaders
# A Batch Sample
for input,tar in trainingloader:
    print(input)
    print(tar)
    break

# Define a set of weights and bias
weight = torch.randn(2, 3, requires_grad=True)
bias = torch.randn(2, requires_grad=True)
print(weight)
print(bias)

# Equation of linear regression
def model(y):
    return y @ weight.t() + bias

# Predict the model output
for a,b in trainingloader:
    preds = model(a)
    print("Prediction is :n",preds)
    print("nActual targets is :n",b)
    break
# Defining the loss function
def MSEloss(predict, targ):
    difference = predict - targ
    return torch.sum(difference * difference)/ difference.numel()

# Calculate the loss
for a,b in trainingloader:
    preds = model(a)
    print("Predict is :n",preds)
    print("n Actual targets is :n",b)
    print("n Loss is: ",MSEloss(preds, b))
    break

Output:

After running the above code, we get the following output in which we can see that the PyTorch linear regression loss value is printed on the screen.

So, with this, we understood about the PyTorch linear regression loss function.

Read: Keras Vs PyTorch – Key Differences

PyTorch linear regression gradient descent

In this section, we will learn about the PyTorch linear regression gradient descent in python.

In linear regression, gradient descent is defined as an optimization algorithm for finding a local minimum of a differentiable function.

It is an optimization algorithm that computes the gradient of the loss function to update the weights and decrease the loss and also find the minima of the loss function.

Code:

n the following code, firstly we will import all the necessary libraries such as import torch, import numpy as np.

• loss = MSEloss(preds, b) is used to calculate the loss function.
• loss.backward() is used to find the gradient of the loss with respect to the independent variable.
• weight -= weight.grad *1e-6 is used to update the weight.
• bias -= bias.grad * 1e-6 is used to update the bias.

# Importing libraries
import torch
import numpy as np

# Loading the data
inputvar = np.array([[75, 69, 45], 
                   [93, 90, 66], 
                   [89, 136, 60], 
                   [104, 45, 39]],
                  dtype='float32')
targetvar = np.array([[66, 80], 
                    [91, 111], 
                    [129, 143], 
                    [32, 47]],  
                   dtype='float32')
# Convert to the torch tensor
inputvar = torch.from_numpy(inputvar)
targetvar = torch.from_numpy(targetvar)
# Print the inputs and outputs
print(inputvar)
print(targetvar)

# Create a tensordataset
from torch.utils.data import TensorDataset
dataset = TensorDataset(inputvar, targetvar)
dataset[:2]

# Using Dataloaders class
from torch.utils.data import DataLoader

batchsiz = 2
trainingloader = DataLoader(dataset, batch_size=batchsiz, shuffle=True)
# Acess the data from the dataloaders
# A Batch Sample
for input,tar in trainingloader:
    print(input)
    print(tar)
    break

# Define a set of weights and bias
weight = torch.randn(2, 3, requires_grad=True)
bias = torch.randn(2, requires_grad=True)
print(weight)
print(bias)

# Equation of linear regression
def model(y):
    return y @ weight.t() + bias

# Predict the model output
for a,b in trainingloader:
    preds = model(a)
    print("Prediction is :n",preds)
    print("nActual targets is :n",b)
    break
# Defining the loss function
def MSEloss(predict, targ):
    difference = predict - targ
    return torch.sum(difference * difference)/ difference.numel()

# Calculate the loss
for a,b in trainingloader:
    preds = model(a)
    print("Predict is :n",preds)
    print("n Actual targets is :n",b)
    print("n Loss is: ",MSEloss(preds, b))
    break
# Implementing gradient descent for 20 epochs
epochs = 20
for x in range(epochs):
    # Iterate through training dataloader
    for a,b in trainingloader:
        # Generate Prediction
        preds = model(a)
        # Get the loss and perform backpropagation
        loss = MSEloss(preds, b)
        loss.backward()
        # Let's update the weights
        with torch.no_grad():
            weight -= weight.grad *1e-6
            bias -= bias.grad * 1e-6
            # Set the gradients to zero
            weight.grad.zero_()
            bias.grad.zero_()
    print(f"Epoch {x}/{epochs}: Loss: {loss}")

Output:

In the below output, you can see that the PyTorch linear regression gradient descent value is printed on the screen.

This is how we can use the Pytorch linear regression gradient descent for calculating the gradient of loss to update the weight and bias.

Read: PyTorch Load Model + Examples

PyTorch linear regression with regularization

In this section, we will learn about the PyTorch linear regression with regularization in python.

The weight_decay parameter applied regularization during initializing the optimizer and add regularization to the loss.

Code:

In the following code, we will import all the necessary libraries such as import torch, import variable from torch.autograd, and import numpy as num.

• xval = [i for i in range(11)] is used to create dummy data for training.
• class Linearregressionmodel(torch.nn.Module): The model is a subclass of torch.nn.Module.
• model = linearRegression(inputdim, outputdim) is used to initialize the linear regression model.
• criterion = torch.nn.MSELoss() is used to calculate the loss.
• optimizer = torch.optim.SGD(model.parameters(), lr=lR) is used to initialize the optimizer.
• input = Variable(torch.from_numpy(Xtrain).cuda()) is used to convert input to the variable.
• optimizer.zero_grad() is used to clear gradient buffers because we don’t want any gradient from the previous epoch to carry forward, and don’t want to accumulate gradients.
• output = model(input) is used to get the output from the model.
• loss = criterion(output, lbl) is used to get the loss for the predicted output.
• optimizer.step() is used to update the parameters.
• print(‘epoch {}, loss {}’.format(epoch, loss.item())) is used to print the epoch and loss values.

# Importing libraries
import torch
from torch.autograd import Variable
import numpy as num

# create dummy data for training
xval = [i for i in range(11)]
Xtrain = num.array(xval, dtype=np.float32)
Xtrain = Xtrain.reshape(-1, 1)

yval = [2*i + 1 for i in xval]
Ytrain = num.array(yval, dtype=np.float32)
Ytrain = Ytrain.reshape(-1, 1)

# Create a model class
class linearRegression(torch.nn.Module):
    def __init__(self, inputSize, outputSize):
        super(linearRegression, self).__init__()
        self.linear = torch.nn.Linear(inputSize, outputSize)

    def forward(self, y):
        out = self.linear(y)
        return out
# Instantiate the model 
# takes variable 'x' 
inputdim = 1        
# takes variable 'y'
outputdim = 1       
lR = 0.01 
epochs = 80

# Initialize the model
model = linearRegression(inputdim, outputdim)
# For GPU 
if torch.cuda.is_available():
    model.cuda()
# Define the loss function
criterion = torch.nn.MSELoss()
# Initialize the optimizer
optimizer = torch.optim.SGD(model.parameters(), lr=lR)
# Train our model
for epoch in range(epochs):
    # Converting input and label to Variable
    if torch.cuda.is_available():
        input = Variable(torch.from_numpy(Xtrain).cuda())
        label = Variable(torch.from_numpy(Ytrain).cuda())
    else:
        input = Variable(torch.from_numpy(Xtrain))
        lbl = Variable(torch.from_numpy(Ytrain))

    # Clear gradient buffers because we don't want any gradient from previous epoch to carry forward, dont want to cummulate gradients
    optimizer.zero_grad()

    # get output from the model, given the inputs
    output = model(input)

    # get loss for the predicted output
    loss = criterion(output, lbl)
    print(loss)
    # get gradients w.r.t to parameters
    loss.backward()

    # update parameters
    optimizer.step()

    print('epoch {}, loss {}'.format(epoch, loss.item()))

Output:

In the below output, you can see that the PyTorch linear regression with regularization values is printed on the screen.

So, with this, we understood about the PyTorch linear regression with regularization.

Read: Adam optimizer PyTorch

PyTorch linear regression accuracy

In this section, we will learn about the PyTorch linear regression accuracy in python.

The linear regression establishes a linear relationship between the dependent and independent variables.

Accuracy is defined as a process of evaluating the models. Accuracy is the fraction of predictions of the model got correct.

Code:

In the following code, firstly we will import all the necessary libraries such as import torch, import torch.nn as nn, import torchvision.transforms as transforms, import torchvision.datasets as dtset.

• trainds = dtset.MNIST(root=’./data’, train=True, transform=transforms.ToTensor(), download=True) is used to load the dataset.
• len(trainds) is used to find the length of the training dataset.
• numepoch = int(numepoch) is used to get the num epoch integer value.
• class Linearregressionmodel(torch.nn.Module): The model is a subclass of torch.nn.Module.
• models = Linearregressionmodel(inpdim, outdim) is used to initializing the model class.
• criterions = nn.CrossEntropyLoss() is used to calculate the cross entropy loss between the input and target variable.
• optimizers = torch.optim.SGD(models.parameters(), lr=l_r) is used to initialize the optimizer.
• print(len(list(models.parameters()))) is used to print the length of the parameter.
• optimizers.zero_grad() is used to clear the gradient.
• losses = criterions(outps, lbls) is used to calculate the loss.
• totals += lbls.size(0) is used to calculate the total number of labels.
• corrects += (predict == lbls).sum() is used as a total number of correct prediction.
• print(‘Iterations: {}. Loss: {}. Accuracy: {}’.format(iters, losses.item(), accuracy)) is used to print the iteration on the screen.

# Importing libraries
import torch
import torch.nn as nn
import torchvision.transforms as transforms
import torchvision.datasets as dtset

# Loading dataset
trainds = dtset.MNIST(root='./data', 
                            train=True, 
                            transform=transforms.ToTensor(),
                            download=True)
len(trainds)
testds = dtset.MNIST(root='./data', 
                           train=False, 
                           transform=transforms.ToTensor())
len(testds)
# Instantiate the model class
batchsiz = 100
niter = 2000
numepoch = niter / (len(trainds) / batchsiz)
numepoch = int(numepoch)
numepoch

# Dataloaders
trainloadr = torch.utils.data.DataLoader(dataset=trainds, 
                                           batch_size=batchsiz, 
                                           shuffle=True)

testloadr = torch.utils.data.DataLoader(dataset=testds, 
                                          batch_size=batchsiz, 
                                          shuffle=False)

# Create the model class
class Linearregressionmodel(torch.nn.Module):
    def __init__(self,inpdim,outdim): 
        super(Linearregressionmodel, self).__init__()
        # One input and one output 
        self.linear = torch.nn.Linear(inpdim,outdim)    

    def forward(self, y): 
        Ypred = self.linear(y) 
        return Ypred 
inpdim = 28*28
outdim = 10

# Initializing the model
models = Linearregressionmodel(inpdim, outdim)
# Define the loss
criterions = nn.CrossEntropyLoss()  
l_r = 0.001

# Initialize the optimizer
optimizers = torch.optim.SGD(models.parameters(), lr=l_r)  

print(models.parameters())
print(len(list(models.parameters())))
 
print(list(models.parameters())[0].size())

print(list(models.parameters())[1].size())
# Train the model
iters = 0
for epoch in range(numepoch):
    for x, (imgs, lbls) in enumerate(trainloadr):

        imgs = imgs.view(-1, 28*28).requires_grad_()
        label = lbls

       
        optimizers.zero_grad()

        # Forward pass to get output/logits
        outps = models(imgs)


        losses = criterions(outps, lbls)


        losses.backward()

        optimizers.step()

        iters += 1

        if iters % 500 == 0:
                 
            corrects = 0
            totals = 0
            
            for imgs, lbls in testloadr:
               
                imgs = imgs.view(-1, 28*28).requires_grad_()

                
                outps = models(imgs)


                _, predict = torch.max(outps.data, 1)

                totals += lbls.size(0)

 
                corrects += (predict == lbls).sum()

            accuracy = 100 * corrects / totals


            print('Iterations: {}. Loss: {}. Accuracy: {}'.format(iters, losses.item(), accuracy))

Output:

After running the above code, we get the following output in which we can see that the PyTorch linear regression accuracy value is printed on the screen.

This is how we can calculate the PyTorch linear regression accuracy.

Also, take a look at some more Python PyTorch tutorials.

• PyTorch Numpy to Tensor
• PyTorch Activation Function
• PyTorch MNIST Tutorial
• PyTorch Model Summary
• Jax Vs PyTorch [Key Differences]

So, in this tutorial, we discussed the Pytorch Linear Regression and we have also covered different examples related to its implementation. Here is the list of examples that we have covered.

• PyTorch linear regression
• PyTorch linear regression from scratch
• PyTorch linear regression dataloaders
• PyTorch linear regression loss
• PyTorch linear regression gradient descent
• PyTorch linear regression with regularization
• PyTorch linear regression accuracy