PyTorch nn linear + Examples

In this Python tutorial, we will learn about PyTorch nn linear in Python and we will also cover different examples related to PyTorch nn linear. And, we will cover these topics.

• PyTorch nn linear
• PyTorch nn linear initialization
• PyTorch nn linear example
• PyTorch nn.linear activation
• PyTorch nn.linear source code
• PyTorch nn.linear sigmoid
• PyTorch nn.linear batch
• PyTorch nn.linear in_features
• PyTorch nn.linear module list
• PyTorch nn.linear nan to num

PyTorch nn linear

In this section, we will learn about how PyTorch nn linear works in Python.

• Before moving forward we should have a piece of knowledge about the linear equation.
• The linear equation is in the form of Ax=B where we can define x as input and b as output and A is defined as weight.
• PyTorch nn linear is a module which is used to create a single layer feed-forward network with n inputs and m outputs.

Code:

In the following code, we will import the torch module from which we can create a single layer feed-forward network with n input and m output.

• netofmodel = torch.nn.Linear(2,1); is used as to create a single layer with 2 inputs and 1 output.
• print(‘Network Structure : torch.nn.Linear(2,1) :\n’,netofmodel) is used to print the network structure on the screen.
• print(‘Weight Of Network :\n’,netofmodel.weight) is used to print the weight of the network on the screen.
• print(‘Bias Of Network :\n’,netofmodel.bias) is used to print the basis of the network on the screen.

import torch
netofmodel = torch.nn.Linear(2,1);
 
print('Network Structure : torch.nn.Linear(2,1) :\n',netofmodel)

print('Weight Of Network :\n',netofmodel.weight)

print('Bias Of Network :\n',netofmodel.bias)

In the following output, we can see that the Network structure, Weight of the network, and the basis of the network are printed on the screen.

In the below code, we will create the single layer with the help of 2 inputs and 2 outputs.

• print(‘Network Structure : torch.nn.Linear(2,2) :\n’,netofmodel) is used to print the network structure on the screen.
• print(‘Weight Of The Network :\n’,netofmodel.weight) is used to print the weight of the network on the screen.
• print(‘Bias Of The Network :\n’,netofmodel.bias) is used to print the basis of the network on the screen.

import torch
netofmodel = torch.nn.Linear(2,2);
print('Network Structure : torch.nn.Linear(2,2) :\n',netofmodel)

print('Weight Of The Network :\n',netofmodel.weight)

print('Bias Of The Network :\n',netofmodel.bias)

In the following output, we can see that the Network structure, Weight of the Network, Bais of the Network with 2 inputs and 2 outputs are printed on the screen.

In the below code we will create a single layer with the help of 2 inputs and 3 outputs.

• print(‘Network Structure : torch.nn.Linear(2,3) :\n’,netofmodel) is used to print the network structure on the screen.
• print(‘Weight Of The Network :\n’,netofmodel.weight) is used to print the weight of the network on the screen.
• print(‘Bias Of The Network :\n’,netofmodel.bias) is used to print the basis of the network on the screen.

import torch
netofmodel = torch.nn.Linear(2,3);
 
print('Network Structure : torch.nn.Linear(2,3) :\n',netofmodel)

print('Weight Of The Network :\n',netofmodel.weight)

print('Bias Of The Network :\n',netofmodel.bias)

After running the above code, we get the following out in which we can see that the network structure, weight of the network, bais of the network with 2 input and 3 output is printed on the screen.

In the below code we will create a single layer with the help of 3 inputs and 2 outputs.

• print(‘Network Structure : torch.nn.Linear(3,2) :\n’,netofmodel) is used to print the Network structure on the screen.
• print(‘Weight Of The Network :\n’,netofmodel.weight) is used to print the weight of the network on the screen.
• print(‘Bias Of The Network :\n’,netofmodel.bias) is used to print the bais of the network on the screen.

import torch
netofmodel = torch.nn.Linear(3,2);
print('Network Structure : torch.nn.Linear(3,2) :\n',netofmodel)

print('Weight Of The Network :\n',netofmodel.weight)

print('Bias Of The Network :\n',netofmodel.bias)

After running the above code, we get the following output in which we can see that network structure, Weight of the network, and bais of the network with the help of 3 input and 2 output is printed on the screen.

Read: Pandas in Python

PyTorch nn linear initialization

In this section, we will learn about how PyTorch nn linear initialization is done in python.

• As we know the nn linear is a module which is used to create a single layer feed-forward network with the help of n inputs and m outputs.
• Pytorch nn linear initialization in which we can also create the feed-forward network with the help of the input and output.

Code:

In the following code, we will import the torch library from which we can create a feed-forward network.

• self.linear = nn.Linear(weights.shape[1], weights.shape[0]) is used to give the shape to the weight.
• X = self.linear(X) is used to define the class for the linear regression.
• weight = torch.randn(12, 12) is used to generate the random weights.
• outs = model(torch.randn(1, 12)) is used to return the tensor defined by the variable argument.
• outs.mean().backward() is used to calculate the mean.
• print(name, param.grad) is used to print the name and parameter gradient.

import torch
import torch.nn as nn
class MyModule(nn.Module):
    def __init__(self, weights):
        super(MyModule, self).__init__()

        self.linear = nn.Linear(weights.shape[1], weights.shape[0])
        with torch.no_grad():
            self.linear.weight.copy_(weights)
        
    def forward(self, X):
        X = self.linear(X)
        return X

weight = torch.randn(12, 12)
model = MyModule(weight)
optimizer = torch.optim.SGD(model.parameters(), lr=1e-3)

outs = model(torch.randn(1, 12))
outs.mean().backward()

for name, param in model.named_parameters():
    print(name, param.grad)

w0 = model.linear.weight.clone()
optimizer.step()

Output:

After running the above code, we get the following output in which we can see that the feed-forward network is printed on the screen.

Read: What is NumPy in Python

PyTorch nn linear example

In this section, we will learn about how to implement PyTorch nn linear example in python.

• The nn linear module is used to calculate the linear equation. The linear equation is in the form of Ax = B, where x is input and B is output and A is the weight.
• The nn linear module is also defined as which is used to create the single-layer feed-forward network.

Code:

In the following code, we will import some libraries from which we can create a feed-forward network.

• self.conv1 = nn.Conv2d(3, 8, 7) is used to create a network with 3 inputs and 8 output.
• self.fc1 = nn.Linear(18 * 7 * 7, 140) is used to calculate the linear equation.
• X = f.max_pool2d(f.relu(self.conv1(X)), (4, 4)) is used to create a maxpooling over a window.
• size = x.size()[3:] is used for all the dimension except the batch dimension.
• print(len(params)) is used to print the parameters.

import torch
from torch.autograd import Variable
import torch.nn as nn
import torch.nn.functional as f


class Net(nn.Module):

    def __init__(self):
        super(Net, self).__init__()
    
        self.conv1 = nn.Conv2d(3, 8, 7)
        self.conv2 = nn.Conv2d(3, 18, 7)

        # an affine operation: y = Wx + b
        self.fc1 = nn.Linear(18 * 7 * 7, 140)
        self.fc2 = nn.Linear(140, 86)
        self.fc3 = nn.Linear(86, 12)

    def forward(self, X):
        X = f.max_pool2d(f.relu(self.conv1(X)), (4, 4))

        X = f.max_pool2d(F.relu(self.conv2(X)), 4)

        X = x.view(-3, self.num_flat_features(X))
        X = f.relu(self.fc1(X))
        X = f.relu(self.fc2(X))
        X = self.fc3(X)

        return X

    def num_flat_features(self, x):
        size = x.size()[3:]  
        num_features = 3
        for s in size:       
            num_features *= s
        return num_features


net = Net()
print(net)
params = list(net.parameters())
print(len(params))       

print(params[0].size())

Output:

In the following output, we can see that the feed-forward network is created with the help of nn.linear module.

Read: PyTorch Tensor to Numpy

PyTorch nn.linear activation

In this section, we will learn about how PyTorch nn.linear activation works in python.

• Pytorch nn.linear activation function is defined as the process which takes the input and output attributes and prepares the matrics.
• nn.ReLU is used as an activation function that creates the network and also fits the complex data.

Code:

In the following code, we will import some torch libraries from which we can create a network.

• torch.relu(X) is used as an activation function that creates the network.
• f.leaky_relu(X) is used to allow the small, positive gradient when the unit is not active.
• nn.Linear(input_size, hidden_size) module which takes input and output attributes and prepares the matrix.
• nn.relu is used as an activation function that creates the network.

import torch
import torch.nn as nn
import torch.nn.functional as f

X = torch.tensor([-2.0, 2.0, 3.0, 4.0])

# sofmax
output = torch.softmax(X, dim=0)
print(output)
softmax = nn.Softmax(dim=0)
output = softmax(X)
print(output)

# sigmoid 
output = torch.sigmoid(X)
print(output)
sig = nn.Sigmoid()
output = sig(X)
print(output)

#tanh
output = torch.tanh(X)
print(output)
t = nn.Tanh()
output = t(X)
print(output)

# relu
output = torch.relu(X)
print(output)
relu = nn.ReLU()
output = relu(X)
print(output)


output = f.leaky_relu(X)
print(output)
lrelu = nn.LeakyReLU()
output = lrelu(X)
print(output)


class NeuralNet(nn.Module):
    def __init__(self, input_size, hidden_size):
        super(NeuralNet, self).__init__()
        self.linear1 = nn.Linear(input_size, hidden_size)
        self.relu = nn.ReLU()
        self.linear2 = nn.Linear(hidden_size, 2)
        self.sigmoid = nn.Sigmoid()
    
    def forward(self, X):
        outs = self.linear1(X)
        outs = self.relu(outs)
        outs = self.linear2(outs)
        outs = self.sigmoid(outs)
        return outs

#use activation functions directly in forward pass
class NeuralNet(nn.Module):
    def __init__(self, input_size, hidden_size):
        super(NeuralNet, self).__init__()
        self.linear1 = nn.Linear(input_size, hidden_size)
        self.linear2 = nn.Linear(hidden_size, 2)
    
    def forward(self, X):
        outs = torch.relu(self.linear1(X))
        outs = torch.sigmoid(self.linear2(outs))
        return outs

Output:

In the following output, we can see that the PyTorch nn.linear activation values are printed on the screen.

Read: PyTorch Load Model

PyTorch nn.linear source code

In this section, we will learn about how to implement PyTorch nn.linear source code in python.

• PyTorch nn.linear source code is defined as a process to calculate a linear equation Ax=B.
• The nn.linear module is also used to create the feed-forward network with the help of inputs and outputs.

Code:

In the following code, we will import some libraries from which we can create a network with inputs and outputs.

• torch.nn.Linear(1, 1) is used to create a network with the help of 1 input and 1 output.
• torch.nn.MSELoss(size_average = False) is used as a multiple standard loss function.
• torch.optim.SGD(model.parameters(), lr = 0.01) is used to optimize the parameters.
• pred_y = model(X_data) is used to compute the predicted y data.
• optimizer.zero_grad() zero grad is used perform backward pass and update the weights.
• print(‘epoch {}, loss {}’.format(epoch, loss.item())) is used to print the data.

import torch
from torch.autograd import Variable
 
X_data = Variable(torch.Tensor([[2.0], [3.0], [4.0]]))
y_data = Variable(torch.Tensor([[3.0], [5.0], [7.0]]))
 
 
class LinearRegressionModel(torch.nn.Module):
 
    def __init__(self):
        super(LinearRegressionModel, self).__init__()
        self.linear = torch.nn.Linear(1, 1) 
 
    def forward(self, X):
        y_pred = self.linear(X)
        return y_pred
 

model = LinearRegressionModel()
 
criterion = torch.nn.MSELoss(size_average = False)
optimizer = torch.optim.SGD(model.parameters(), lr = 0.01)
 
for epoch in range(400):
 

    pred_y = model(X_data)
 
    # Compute and print loss
    loss = criterion(pred_y, y_data)
 

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    print('epoch {}, loss {}'.format(epoch, loss.item()))
 
var = Variable(torch.Tensor([[5.0]]))
pred_y = model(var)
print("predict (After training)", 5, model(var).item())

Output:

When we run the above code we get the following output in which we can see that the epoch and loss value is printed on the screen.

Read: Keras Vs PyTorch – Key Differences

PyTorch nn.linear sigmoid

In this section, we will learn about how to implement PyTorch nn.linear sigmoid in python.

Pytorch nn.linear sigmoid is a non-linear function and the activation function for a neuron is the sigmoid function it always gives the output of the unit in between 0 and 1.

Code:

In the following code, we will import some libraries from which we can create a feed-forward network.

• X = torch.randn((4, 4, 4))is used to generate the random numbers.
• Y = torch.sigmoid(x) is used to give the output of the unit always in between 0 and 1.
• Y.min(), Y.max() is used to print the min and max value.
• self.linear = torch.nn.Linear(input_dim, 1) is used to create the feed forward network.
• torch.manual_seed(1) is used to set the seed for generating random numbers.

import torch

torch.manual_seed(1)

X = torch.randn((4, 4, 4))
Y = torch.sigmoid(x)

Y.min(), Y.max()
)
class MyModel(torch.nn.Module):
    def __init__(self, input_dim):
        super().__init__()
        self.linear = torch.nn.Linear(input_dim, 1)
        self.activation = torch.nn.Sigmoid()

    def forward(self, X):
        x = self.linear(X)
        return self.activation(X)


torch.manual_seed(1)

model = MyModel(5)
X = torch.randn((11, 5))
Y = model(X)

Y.min(), Y.max()

Output:

After running the above code, we get the following output in which we can see that the Pytorch nn.linear sigmoid value is printed on the screen.

Read: PyTorch MSELoss – Detailed Guide

PyTorch nn.linear batch

In this section, we will learn about how Pytorch nn.linear batch works in python.

PyTorch nn.linear batch module is defined as a process to create the fully connected weight matrix in which every input is used to create the output value.

Code:

In the following code, we will import some libraries from which we can create nn.linear batches.

• nn.Sequential() is used to run a certain layer sequentially.
• torch.manual_seed(43) is used to set fixed random number seed.
• mlp = MLP() is used to initialize the MLP.
• for epoch in range(0, 6): is used to run the training loop.
• print(f’Starting epoch {epoch+1}’) is used to print the epoch.
• current_loss = 0.0 is used to set the current loss value.
• input, target = data is used to get the inputs.
• output = mlp(input) is used to perform forward pass.

import os
import torch
from torch import nn
from torchvision.datasets import CIFAR10
from torch.utils.data import DataLoader
from torchvision import transforms

class MultilayerPerceptron(nn.Module):
 
  def __init__(self):
    super().__init__()
    self.layers = nn.Sequential(
      nn.Flatten(),
      nn.Linear(34 * 34 * 4, 65),
      nn.BatchNorm1d(65),
      nn.ReLU(),
      nn.Linear(65, 33),
      nn.BatchNorm1d(33),
      nn.ReLU(),
      nn.Linear(33, 11)
    )


  def forward(self, X):
    '''Forward pass'''
    return self.layers(X)
  
  
if __name__ == '__main__':
 
  torch.manual_seed(43)
  
  # Prepare CIFAR-10 dataset
  dataset = CIFAR10(os.getcwd(), download=True, transform=transforms.ToTensor())
  trainloader = torch.utils.data.DataLoader(dataset, batch_size=10, shuffle=True, num_workers=1)
 
  mlp = MLP()
  
  # Define the loss function and optimizer
  loss_function = nn.CrossEntropyLoss()
  optimizer = torch.optim.Adam(mlp.parameters(), lr=1e-4)
  
  for epoch in range(0, 6): 
    
    print(f'Starting epoch {epoch+1}')
    
    current_loss = 0.0
    
    # Iterate over the DataLoader for training data
    for i, data in enumerate(trainloader, 0):
      
      input, target = data
      
      # Zero the gradients
      optimizer.zero_grad()
      
      output = mlp(input)
      
      # Compute loss
      loss = loss_function(output, target)
      
      # Perform backward pass
      loss.backward()
      
      # Perform optimization
      optimizer.step()
      
      # Print statistics
      current_loss += loss.item()
      if i % 500 == 499:
          print('Loss after mini-batch %5d: %.3f' %
                (i + 1, current_loss / 500))
          current_loss = 0.0
  print('Training process has been finished.')

Output:

When we run the above code, we will get the following output in which we can see that the batch is created on the screen.

Read PyTorch Logistic Regression

PyTorch nn.linear in_features

In this section, we will learn about how PyTorch nn.linear in_features works in python.

• PyTorch nn.linear in_features is defined as a process that applies a linear change to incoming data.
• in_feature is a parameter used as the size of every input sample.

Code:

In the following code, we will import some libraries from which we can apply some changes to incoming data.

• torch.tensor() is used as a multidimensional matrix that holds an element of a single data type.
• in_features = X.shape[1] is used to apply some changes to the incoming data.
• nn.Linear(in_features, out_features) is used to create the network.

import torch
import torch.nn as nn

X = torch.tensor([[2.0, -2.0],
                  [0.0,  2.0],
                  [0.0,  0.0]])

in_features = X.shape[1]  
out_features = 3

n = nn.Linear(in_features, out_features) 
n.weight

Output:

After running the above code, we get the following output in which we can see that PyTorch nn.linear in_feature data is printed on the screen.

Read PyTorch Binary Cross Entropy

PyTorch nn.linear module list

In this section, we will learn about how to create the PyTorch nn.linear module list in python.

PyTorch nn.linear module list is defined as a list that can be indexed like a systematic Python list and the modules that are contained by the list are properly registered and it is visible to all the module methods.

Code

In the following code, we will import some libraries from which we can create the PyTorch nn.linear module list.

• m1 = nn.Linear(ninputs, nhidden_unit) is used to create the feed-forward network with n inputs and m outputs.
• nn.Sigmoid() is defined as an activation function for neurons it gives the output of the unit is always in between 0 and 1.
• nn.Softmax(dim=1) is used to apply to all the slices of the input through the particular dim.
• nn.ModuleList(m) is indexed like a systematic Python list and the modules that are contained by the list are properly registered.
• model = neuralnetwork(ninputs=12, nhidden_unit=32, noutput=4) is used to create a model.

import torch
from torch import nn
import torch.nn.functional as f
class neuralnetwork(nn.Module):
    def __init__(self, ninputs, nhidden_unit, noutput):
        super().__init__()
        m1 = nn.Linear(ninputs, nhidden_unit)
        n1 = nn.Sigmoid()
        m2 = nn.Linear(nhidden_unit, noutput)
        s = nn.Softmax(dim=1)
        
        m = [m1, n1, m2, s]
        self.module_list = nn.ModuleList(m)

    def forward(self, X):
        for f in self.module_list:
            X = f(X)
        return X
model = neuralnetwork(ninputs=12, nhidden_unit=32, noutput=4)
model

Output:

After running the above code, we get the following output in which we can see that a neural network module list is created on the screen.

PyTorch nn.linear nan to num

In this section, we will learn about how PyTorch nn.linear nan works in python.

• Before we move forward we should have some piece of knowledge about nan. Nan is known as Not A Number.
• Nan is the way to represent the missing value in the data and also a floating-point value. Here we convert NAN to Num.

Syntax:

torch.nan_to_num(input,nan=0.0,posinf=None,neginf=None,out=None)

Parameter used in the syntax:

• input: input is used as the input tensor.
• nan is used to representing the missing values in the data. The default value of nan is 0.
• posinf: If the number is given the values of a number are replaced by a positive infinite value. If the None is given the positive infinite value is replaced by the greatest finite value that is represented by input dtypes value.
• nehinf: If the number given the value of a number is replaced by a negative infinite value.If the None is given the negative value is represented by the lowest finite value that is represented by input dtypes value.
• out: out is used as the output tensor.

Example:

In this example, we import torch library from which we can convert the nan value to num.

• X = torch.tensor([float(‘nan’), -float(‘inf’), float(‘inf’), 3.16]) is used to give the x value.
• torch.nan_to_num(X) is used to convert the nan to num.

import torch
X = torch.tensor([float('nan'), -float('inf'), float('inf'), 3.16])
torch.nan_to_num(X)

Output:

After running the above code, we get the following output in which we can see that the nan to num value is printed on the screen.

So, in this tutorial, we discussed PyTorchh nn linear and we have also covered different examples related to its implementation. Here is the list of examples that we have covered.

• PyTorch nn linear
• PyTorch nn linear initialization
• PyTorch nn linear example
• PyTorch nn.linear activation
• PyTorch nn.linear source code
• PyTorch nn.linear sigmoid
• PyTorch nn.linear batch
• PyTorch nn.linear in_features
• PyTorch nn.linear module list
• PyTorch nn.linear nan to num