In this Python tutorial, we will learn about PyTorch Logistic Regression in python and we will also cover different examples related to PyTorch Logistic Regression. And we will cover these topics.
- PyTorch logistic regression
- PyTorch logistic regression l2
- PyTorch logistic regression mnist
- PyTorch logistic regression accuracy
- PyTorch logistic regression feature importance
- PyTorch logistic regression loss function
- PyTorch logistic regression classifier
PyTorch logistic regression
In this section, we will learn about the PyTorch logistic regression in python.
Logistic regression is defined as a process that expresses data and explains the relationship between one dependent binary variable.
Code:
In the following code, we will import the torch module from which we can do logistic regression.
- models = logistic_regression() is used to define the model.
- criterions = torch.nn.BCELoss(size_average=True) is used to calculate the criterion.
- optimizers = torch.optim.SGD(models.parameters(), lr=0.01) is used to intialize the optimizer.
- optimizers.zero_grad() is used optimize the zero gradient.
- ypred = models(xdata) is used to predic the model.
- losses = criterions(ypred, ydata) is used to calculate the loss.
- losses.backward() is used to calculate the backward loss.
- print(“predicted y value: “, ypred.data[0][0]) is used to print the predicted y value.
import torch
from torch.autograd import Variable
from torch.nn import functional as f
xdata = Variable(torch.Tensor([[12.0], [11.0], [5.0], [4.0]]))
ydata = Variable(torch.Tensor([[92.0], [82.0], [52.0], [32.0]]))
class logistic_regression(torch.nn.Module):
def __init__(self):
super(logistic_regression, self).__init__()
self.linear = torch.nn.Linear(1, 1)
def forward(self, y):
ypred = f.sigmoid(self.linear(y))
return ypred
models = logistic_regression()
criterions = torch.nn.BCELoss(size_average=True)
optimizers = torch.optim.SGD(models.parameters(), lr=0.01)
for epoch in range(20):
models.train()
optimizers.zero_grad()
# Forward pass
ypred = models(xdata)
# Compute Loss
losses = criterions(ypred, ydata)
# Backward pass
losses.backward()
optimizers.step()
x = Variable(torch.Tensor([[6.0]]))
ypred = models(x)
print("predicted y value: ", ypred.data[0][0])Output:
After running the above code, we get the following output in which we can see that the predicted y value is printed on the screen.
Read: PyTorch MSELoss – Detailed Guide
PyTorch logistic regression l2
In this section, we will learn about the PyTorch logistic regression l2 in python.
The weight_decay parameter applied l2 regularization during initializing the optimizer and add regularization to the loss.
Code:
In the following code, we will import the torch module from which we can find logistic regression.
- trainds = dtset.MNIST(root=’./data’,train=True,transform=transforms.ToTensor(), download=True) is used as train dataset.
- len(trainds) is used to find the length of the train dataset.
- numepoch = int(numepoch) is used get the num epoch integer value.
- testloadr = torch.utils.data.DataLoader(dataset=testds,batch_size=batchsiz, shuffle=False) is used to load the test dataset.
- criterions = nn.CrossEntropyLoss() is used to calculate the cross entropy loss between the input and target variable.
- optimizers = torch.optim.SGD(models.parameters(), lr=l_r) is used to initialize the optimizer.
- print(len(list(models.parameters()))) is used to print the length of the parameter.
- imgs = imgs.view(-1, 28*28).requires_grad_() is used to load images as a parameter.
- optimizers.zero_grad() is used to clear the gradient.
- losses = criterions(outps, lbls) is used to calculate the loss.
- totals += lbls.size(0) is used to calculate the total number of labels.
- corrects += (predict == lbls).sum() is used as a total number of correct prediction.
- print(‘Iterations: {}. Loss: {}. Accuracy: {}’.format(iters, losses.item(), accuracy)) is used to print the iteration on the screen.
import torch
import torch.nn as nn
import torchvision.transforms as transforms
import torchvision.datasets as dtset
trainds = dtset.MNIST(root='./data',
train=True,
transform=transforms.ToTensor(),
download=True)
len(trainds)
testds = dtset.MNIST(root='./data',
train=False,
transform=transforms.ToTensor())
len(testds)
batchsiz = 100
niter = 2000
numepoch = niter / (len(trainds) / batchsiz)
numepoch = int(numepoch)
numepoch
trainloadr = torch.utils.data.DataLoader(dataset=trainds,
batch_size=batchsiz,
shuffle=True)
testloadr = torch.utils.data.DataLoader(dataset=testds,
batch_size=batchsiz,
shuffle=False)
class logistic_regression(nn.Module):
def __init__(self, inpdim, outpdim):
super(logistic_regression, self).__init__()
self.linear = nn.Linear(inpdim, outdim)
def forward(self, y):
outp = self.linear(y)
return outp
inpdim = 28*28
outdim = 10
models = logistic_regression(inpdim, outdim)
criterions = nn.CrossEntropyLoss()
l_r = 0.001
optimizers = torch.optim.SGD(models.parameters(), lr=l_r)
print(models.parameters())
print(len(list(models.parameters())))
print(list(models.parameters())[0].size())
print(list(models.parameters())[1].size())
iters = 0
for epoch in range(numepoch):
for x, (imgs, lbls) in enumerate(trainloadr):
imgs = imgs.view(-1, 28*28).requires_grad_()
label = lbls
optimizers.zero_grad()
# Forward pass to get output/logits
outps = models(imgs)
losses = criterions(outps, lbls)
losses.backward()
optimizers.step()
iters += 1
if iters % 500 == 0:
corrects = 0
totals = 0
for imgs, lbls in testloadr:
imgs = imgs.view(-1, 28*28).requires_grad_()
outps = models(imgs)
_, predict = torch.max(outps.data, 1)
totals += lbls.size(0)
corrects += (predict == lbls).sum()
accuracy = 100 * corrects / totals
print('Iterations: {}. Loss: {}. Accuracy: {}'.format(iters, losses.item(), accuracy))Output:
After running the above code, we get the following output in which we can see that the loss and accuracy are printed on the screen.
Read: Keras Vs PyTorch – Key Differences
PyTorch logistic regression mnist
In this section, we will learn about PyTorch logistic regression with mnist data in python.
Logistic regression is used to express the data and also used to clarify the relationship between one dependent binary variable. Here we can use the mnist dataset to do calculate the regression.
Code:
In the following code, we will import the torch module from which we can do the logistic regression.
- datasets = FashionMNIST(root=’D:\PyTorch\data’, train=True, transform=transforms.ToTensor(), download=True) is used as a dataset.
- traindatas, valdatas = random_split(datasets, [50000, 10000]) is used to train and validate the data.
- testdatas = FashionMNIST(root=’D:\PyTorch\data’, train=False, transform=transforms.ToTensor()) is used to test the data.
- trainloadr = DataLoader(traindatas, batchsiz, shuffle=True) is used to load the train data.
- valloadr = DataLoader(valdatas, batchsiz*2) is used to validate the data.
- outp = self(imgs) is used to generate prediction.
- losses = fun.cross_entropy(outp, lbls) is used to calculate the loss.
- accur = accuracy(outp, lbls) is used to calculate the accuracy.
- epochloss = torch.stack(batchlosses).mean() is used to combine losses.
- epochaccu = torch.stack(batchaccus).mean() is used to combine accuracy.
- eval(modl, valloadr) is used to evaluate the model.
import torch
import torchvision
import torch.nn as nn
import matplotlib.pyplot as plot
import torch.nn.functional as fun
import torchvision.transforms as transforms
from torchvision.datasets import FashionMNIST
from torch.utils.data import random_split
from torch.utils.data import DataLoader
import numpy as num
batchsiz = 124
lr = 0.001
inputsiz = 28*28
numclass = 12
datasets = FashionMNIST(root='D:\PyTorch\data', train=True, transform=transforms.ToTensor(), download=True)
traindatas, valdatas = random_split(datasets, [50000, 10000])
testdatas = FashionMNIST(root='D:\PyTorch\data', train=False, transform=transforms.ToTensor())
# Dataloaders
trainloadr = DataLoader(traindatas, batchsiz, shuffle=True)
valloadr = DataLoader(valdatas, batchsiz*2)
testloadr = DataLoader(testdatas, batchsiz*2)
class mnistmodel(nn.Module):
def __init__(self):
super().__init__()
self.linear = nn.Linear(inputsiz, numclass)
def forward(self, ya):
ya = ya.reshape(-1, 784)
outp = self.linear(ya)
return outp
def train(self, batches):
imgs, lbls = batches
outp = self(imgs)
losses = fun.cross_entropy(outp, lbls)
return losses
def validate(self, batches):
imgs, lbls = batches
outp = self(imgs)
losses = fun.cross_entropy(outp, lbls)
accur = accuracy(outp, lbls)
return {'valloss': losses.detach(), 'valacc': accur.detach()}
def validationepoch_end(self, outpts):
batchlosses = [y['valloss'] for y in outpts]
epochloss = torch.stack(batchlosses).mean()
batchaccus = [y['valacc'] for y in outpts]
epochaccu = torch.stack(batchaccus).mean()
return {'valloss': epochloss.item(), 'valacc': epochaccu.item()}
def epochend(self, epoch, result):
print("Epoch [{}], valloss: {:.4f}, valacc: {:.4f}".format(epoch, result['valloss'], result['valacc']))
modl = mnistmodel()
def accuracy(outpts, lbls):
_, pred = torch.max(outpts, dim=1)
return torch.tensor(torch.sum(pred == lbls).item() / len(pred))
def eval(modl, valloadr):
outpts = [modl.validate(batch) for batch in valloadr]
return modl.validationepoch_end(outpts)
def fit(epoch, lr, modl, trainloadr, valloadr, optfunc=torch.optim.SGD):
history = []
optimizers = optfunc(modl.parameters(), lr)
for epoch in range(epoch):
# Training Phase
for batch in trainloadr:
loss = modl.training_step(batch)
loss.backward()
optimizers.step()
optimizers.zero_grad()
# Validation phase
result = eval(modl, valloadr)
modl.epoch_end(epoch, result)
history.append(result)
return history
eval(modl, valloadr)Output:
In the following output, we can see that the validated accuracy score is printed on the screen after evaluating the model.
Read: PyTorch Tensor to Numpy
PyTorch logistic regression accuracy
In this section, we will learn about how to calculate the accuracy of logistic regression in python.
Logistic regression is a statical method for predicting binary classes and computing the probability of an event occurrence.
Accuracy is defined as the proportion of correct prediction over the total prediction and here we can calculate the accuracy of logistic regression.
Code:
In the following code, we will import the torch module from which we can calculate the accuracy of the model.
- traindatast = dsets.MNIST(root =’./data’,train = True,transform = transforms.ToTensor(),download = True) is used as train dataset.
- trainldr = torch.utils.data.DataLoader(dataset = traindatast,batch_size = batchsiz, shuffle = True) is used as a dataset loader.
- modl = logistic_reg(inputsiz, numclsses) is used to define as logistic regression model.
- optimiz = torch.optim.SGD(modl.parameters(), lr = l_r) is used to initialize the optimizer.
- print(‘Accuracy Of The Model on the test images: % d %%’ % (100 * corct / totls)) is used to print the accuracy of the model.
import torch
import torch.nn as nn
import torchvision.datasets as dsets
import torchvision.transforms as transforms
from torch.autograd import Variable
# Hyper Parameters
inputsiz = 784
numclsses = 10
numepoch = 3
batchsiz = 98
l_r = 0.001
traindatast = dsets.MNIST(root ='./data',
train = True,
transform = transforms.ToTensor(),
download = True)
testdatast = dsets.MNIST(root ='./data',
train = False,
transform = transforms.ToTensor())
trainldr = torch.utils.data.DataLoader(dataset = traindatast,
batch_size = batchsiz,
shuffle = True)
testldr = torch.utils.data.DataLoader(dataset = testdatast,
batch_size = batchsiz,
shuffle = False)
class logistic_reg(nn.Module):
def __init__(self, inputsiz, numclsses):
super(logistic_reg, self).__init__()
self.linear = nn.Linear(inputsiz, numclsses)
def forward(self, y):
outp = self.linear(y)
return outp
modl = logistic_reg(inputsiz, numclsses)
critrion = nn.CrossEntropyLoss()
optimiz = torch.optim.SGD(modl.parameters(), lr = l_r)
# Training the Model
for epoch in range(numepoch):
for a, (imgs, lbls) in enumerate(trainldr):
imgs = Variable(imgs.view(-1, 28 * 28))
lbls = Variable(lbls)
optimiz.zero_grad()
output = modl(imgs)
losses = critrion(output, lbls)
losses.backward()
optimiz.step()
# Test the Model
corct = 0
totls = 0
for imgs, lbls in testldr:
imgs = Variable(imgs.view(-1, 28 * 28))
output = modl(imgs)
_, predict = torch.max(output.data, 1)
totls += lbls.size(0)
corct += (predict == lbls).sum()
print('Accuracy Of The Model on the test images: % d %%' % (
100 * corct / totls))Output:
After running the above code, we get the following output in which we can see that the accuracy of the model is printed on the screen.
Read: PyTorch Batch Normalization
PyTorch logistic regression feature importance
In this section, we will learn about the PyTorch logistic regression feature’s importance.
Feature importance in logistic regression is an ordinary way to make a model and also describe an existing model.
Code:
In the following code, we will import some modules from which we can describe the existing model.
- modl = logistic_regr(indim, outdim) is used to instantiate model class.
- optim = torch.optim.SGD(modl.parameters(), lr=l_r) is used to instantiate optimizer.
- critr = nn.CrossEntropyLoss() is used to instantiate loss class.
- imags = imags.view(-1, 28*28).requires_grad_() is used to load images as variable.
- _, predicted = torch.max(outs.data, 1) is used to get predictions from maximum value.
- ttl += labls.size(0) is used to calculate total number of variable.
- crct += (predicted == labls).sum() is used to calculate total number of correct predictions.
import torch
import torch.nn as nn
import torchvision.transforms as transforms
import torchvision.datasets as dtsets
traindt = dtsets.MNIST(root='./data',
train=True,
transform=transforms.ToTensor(),
download=True)
testdt = dtsets.MNIST(root='./data',
train=False,
transform=transforms.ToTensor())
bsize = 100
niterations = 3000
epoch = niterations / (len(traindt) / bsize)
epoch = int(epoch)
trainlr = torch.utils.data.DataLoader(dataset=traindt,
batch_size=bsize,
shuffle=True)
testlr = torch.utils.data.DataLoader(dataset=testdt,
batch_size=bsize,
shuffle=False)
class logistic_regr(nn.Module):
def __init__(self, insize, numclass):
super(logistic_regr, self).__init__()
self.linear = nn.Linear(indim, outdim)
def forward(self, y):
outs = self.linear(y)
return outs
indim = 28*28
outdim = 10
modl = logistic_regr(indim, outdim)
critr = nn.CrossEntropyLoss()
l_r = 0.001
optim = torch.optim.SGD(modl.parameters(), lr=l_r)
iterations = 0
for epoch in range(epoch):
for a, (imags, labls) in enumerate(trainlr):
imags = imags.view(-1, 28*28).requires_grad_()
labls = labls
optim.zero_grad()
outs = modl(imags)
loss = critr(outs, labls)
loss.backward()
optim.step()
iterations += 1
if iterations % 500 == 0:
crct = 0
ttl = 0
for imags, labls in testlr:
imags = imags.view(-1, 28*28).requires_grad_()
outs = modl(imags)
_, predicted = torch.max(outs.data, 1)
ttl += labls.size(0)
crct += (predicted == labls).sum()
accur = 100 * crct.item() / ttl
print(' Accuracy: {}'.format( accur))Output:
After running the above code, we get the following output in which we can see that we can make a model and get the accuracy of the model.
Read: PyTorch Load Model + Examples
PyTorch logistic regression loss function
In this section, we will learn about the PyTorch logistic regression loss function in python.
The loss function for logistic regression is log loss. The loss function is calculated from the target and prediction in sequence to update the weight for the best model selection.
Code:
In the following code, we will import some torch modules from which we can calculate the loss function.
- traindst = dsets.MNIST(root=’./data’, train=True, transform=transforms.ToTensor(), download=True) is used as a train dataset.
- trainldr = torch.utils.data.DataLoader(dataset=traindst, batch_size=batchsiz, shuffle=True) is used to load the train data.
- models = logistic_regression(inputdims, outputdims) is used to create the logistic regression model.
- criterions = torch.nn.CrossEntropyLoss() is used to calculate the loss.
- optimizers = torch.optim.SGD(models.parameters(), lr=l_r) is used to initialize the optimizer.
- optimizers.zero_grad() is used to optimize the zero gradient.
- accuracy = 100 * crrects/totals is used to calculate the accuracy.
- print(“Iteration: {}. Loss: {}.”.format(iter, losses.item())) is used to print the loss.
import torch
from torch.autograd import Variable
import torchvision.transforms as transforms
import torchvision.datasets as dsets
batchsiz = 100
niter = 3000
inputdims = 784
outputdims = 10
l_r = 0.001
traindst = dsets.MNIST(root='./data', train=True, transform=transforms.ToTensor(), download=True)
testdst = dsets.MNIST(root='./data', train=False, transform=transforms.ToTensor())
trainldr = torch.utils.data.DataLoader(dataset=traindst, batch_size=batchsiz, shuffle=True)
testldr = torch.utils.data.DataLoader(dataset=testdst, batch_size=batchsiz, shuffle=False)
epoch = niter / (len(traindst) / batchsiz)
class logistic_regression(torch.nn.Module):
def __init__(self, inputdims, outputdims):
super(logistic_regression, self).__init__()
self.linear = torch.nn.Linear(inputdims, outputdims)
def forward(self, y):
out = self.linear(y)
return out
models = logistic_regression(inputdims, outputdims)
criterions = torch.nn.CrossEntropyLoss()
optimizers = torch.optim.SGD(models.parameters(), lr=l_r)
iter = 0
for epoch in range(int(epoch)):
for x, (imgs, lbls) in enumerate(trainldr):
imgs = Variable(imgs.view(-1, 28 * 28))
lbls = Variable(lbls)
optimizers.zero_grad()
out = models(imgs)
losses = criterions(out, lbls)
losses.backward()
optimizers.step()
iter+=1
if iter%500==0:
crrects = 0
totals = 0
for imgs, lbls in testldr:
imgs = Variable(imgs.view(-1, 28*28))
out = models(imgs)
_, predicted = torch.max(out.data, 1)
totals+= lbls.size(0)
crrects+= (predicted == lbls).sum()
accuracy = 100 * crrects/totals
print("Iteration: {}. Loss: {}.".format(iter, losses.item()))Output:
After running the above code, we get the following output in which we can see that the loss value is printed on the screen.
Read: Adam optimizer PyTorch with Examples
PyTorch logistic regression classifier
In this section, we will learn about the PyTorch logistic regression classifier in python.
A logistic regression classifier is used to explain the data and define the relationship between the independent binary variable.
Code:
In the following code, we will import some modules from which we can calculate the logistic regression classifier.
- models = logistic_regression() is used to create a model.
- criterions = torch.nn.BCELoss(size_average=True) is used to calculate the criterion.
- optimizers = torch.optim.SGD(models.parameters(), lr=0.01) is used to initialize the optimizer.
- optimizers.zero_grad() is used to optimize the zero gradient.
- zpred = models(xdt) is used to predit the model.
- nwx = Variable(torch.Tensor([[6.0]])) is used to create the new variable.
- zpred = models(nwx) is used to predict the new model.
- print(“predicted y value: “, zpred.data[0][0]) is used to predict the y value.
import torch
from torch.autograd import Variable
from torch.nn import functional as f
xdt = Variable(torch.Tensor([[15.0], [12.0]]))
ydt = Variable(torch.Tensor([[92.0], [82.0]]))
class logistic_regression(torch.nn.Module):
def __init__(self):
super(logistic_regression, self).__init__()
self.linear = torch.nn.Linear(1, 1)
def forward(self, y):
zpred = f.sigmoid(self.linear(y))
return zpred
models = logistic_regression()
criterions = torch.nn.BCELoss(size_average=True)
optimizers = torch.optim.SGD(models.parameters(), lr=0.01)
for epoch in range(25):
models.train()
optimizers.zero_grad()
zpred = models(xdt)
losses = criterions(zpred, ydt)
losses.backward()
optimizers.step()
nwx = Variable(torch.Tensor([[6.0]]))
zpred = models(nwx)
print("predicted y value: ", zpred.data[0][0])Output:
In the following output, we can see that the logistic regression classifier is calculated and the predicted value of y is printed on the screen.
You may also like to read the following PyTorch tutorials.
- Cross Entropy Loss PyTorch
- PyTorch Leaky ReLU
- PyTorch Reshape Tensor
- PyTorch nn linear + Examples
- PyTorch Model Summary
- PyTorch Numpy to Tensor
- PyTorch Save Model – Complete Guide
So, in this tutorial, we discussed PyTorch Logistic Regression and we have also covered different examples related to its implementation. Here is the list of examples that we have covered.
- PyTorch logistic regression
- PyTorch logistic regression l2
- PyTorch logistic regression mnist
- PyTorch logistic regression accuracy
- PyTorch logistic regression feature importance
- PyTorch logistic regression loss function
- PyTorch logistic regression classifier
I am Bijay Kumar, a Microsoft MVP in SharePoint. Apart from SharePoint, I started working on Python, Machine learning, and artificial intelligence for the last 5 years. During this time I got expertise in various Python libraries also like Tkinter, Pandas, NumPy, Turtle, Django, Matplotlib, Tensorflow, Scipy, Scikit-Learn, etc… for various clients in the United States, Canada, the United Kingdom, Australia, New Zealand, etc. Check out my profile.