Scikit learn Ridge Regression

In this Python tutorial, we will learn How to create scikit learn ridge regression in Python and we will also cover different examples related to ridge regression. And we will cover these topics.

• Scikit learn ridge regression
• Scikit learn ridge regression coefficient
• Scikit learn kernel ridge regression
• Scikit learn bayesian ridge regression

Scikit learn ridge regression

In this section, we will learn about how to solve the Scikit learn ridge regression in python.

Ridge regression is used to solve this regression model and modify the loss function by adding some penalty equivalent to the square of the magnitude of the coefficients.

Code:

In the following code, we will import some libraries from which we can solve the ridge regression in python.

• n_samples, n_features = 15, 10 is used to add samples and features in this ridge function.
• rng = np.random.RandomState(0) is used for the random states.
• rdg.fit(X, y) is used to fit the values.

from sklearn.linear_model import Ridge
import numpy as np
n_samples, n_features = 15, 10
range = np.random.RandomState(0)
y = range.randn(n_samples)
X = range.randn(n_samples, n_features)
ridge = Ridge(alpha = 0.5)
ridge.fit(X, y)
ridge.score(X,y)

Output:

After running the above code, we get the following output in which we can see that the ridge regression score is shown on the screen.

Also, check: Scikit-learn Vs Tensorflow

Scikit learn ridge regression coefficient

In this section, we will learn about how to create scikit learn ridge regression coefficient in python.

Code:

In the following code, we will import the ridge library from sklearn.learn and also import numpy as np.

• n_samples, n_features = 15, 10 is used to add samples and features in the ridge function.
• ridge.score(X,y) is used give get the score of ridge function.
• ridge.coef_ is used to get the coefficient of the ridge function.

from sklearn.linear_model import Ridge
import numpy as np
n_samples, n_features = 15, 10
range = np.random.RandomState(0)
y = range.randn(n_samples)
X = range.randn(n_samples, n_features)
ridge = Ridge(alpha = 0.5)
ridge.fit(X, y)
ridge.score(X,y)
ridge.coef_

Output:

After running the above code, we get the following output in which we can see that the ridge coefficient is printed on the screen.

Read: Scikit-learn logistic regression

Scikit learn kernel ridge regression

In this section, we will learn about how to create a scikit learn kernel ridge regression in Python.

Kernel Ridge Regression merges the ridge regression with their own trick. The kernel includes the space in a linear function.

Code:

In the following code, we will import KernelRidge from sklearn.kernel_ridge from which we can get the kernel ridge value.

• n_samples, n_features = 10, 5 is used to create the sample and feature of kernel ridge.
• KernelRidge(alpha=1.0) is used to get the kernel ridge value.

from sklearn.kernel_ridge import KernelRidge
import numpy as np
n_samples, n_features = 10, 5
range = np.random.RandomState(0)
y = rng.randn(n_samples)
X = rng.randn(n_samples, n_features)
kernel = KernelRidge(alpha=1.0)
kernel.fit(X, y)

Output:

After running the above code, we get the following output in which we can see that the kernel ridge value is shown on the screen.

Read: Scikit learn Decision Tree

Scikit learn bayesian ridge regression

In this section, we will learn about how to create a scikit learn Bayesian ridge regression in Python.

Bayesian regression permits the natural mechanism to hold on to insufficient distributed data by formulating linear regression using probability.

Code:

In the following code, we will import some libraries from which we can create a Bayesian ridge regression.

• n_samples, n_features = 100, 100 is used to generating data.
• X = np.random.randn(n_samples, n_features) is used to create gaussian data.
• lambda_ = 4.0 is used to create weights with lambda value.
• relevant_features = np.random.randint(0, n_features, 10) is used to keep the 10 weight of interest.
• clf = BayesianRidge(compute_score=True) is used to fit the bayesian regression.
• plot.figure(figsize=(6, 5)) is used to plot the graph.

import numpy as np
import matplotlib.pyplot as plot
from scipy import stats

from sklearn.linear_model import BayesianRidge, LinearRegression
np.random.seed(0)
n_samples, n_features = 100, 100
X = np.random.randn(n_samples, n_features)  # Create Gaussian data
lambda_ = 4.0
w = np.zeros(n_features)
relevant_features = np.random.randint(0, n_features, 10)
for i in relevant_features:
    w[i] = stats.norm.rvs(loc=0, scale=1.0 / np.sqrt(lambda_)).
alpha_ = 50.0
noise = stats.norm.rvs(loc=0, scale=1.0 / np.sqrt(alpha_), size=n_samples)
y = np.dot(X, w) + noise

clf = BayesianRidge(compute_score=True)
clf.fit(X, y)

ols = LinearRegression()
ols.fit(X, y)

lw = 2
plot.figure(figsize=(6, 5))
plot.title("Weights of model")
plot.plot(clf.coef_, color="green", linewidth=lw, label="Bayesian Ridge estimate")
plot.plot(w, color="yellow", linewidth=lw, label="Ground truth")
plot.plot(ols.coef_, color="red", linestyle="--", label="OLS estimate")
plot.xlabel("Features")
plot.ylabel("Values of  weights")
plot.legend(loc="best", prop=dict(size=12))

plot.figure(figsize=(6, 5))
plot.title("Histogram of weights")
plot.hist(clf.coef_, bins=n_features, color="gold", log=True, edgecolor="black")
plot.scatter(
    clf.coef_[relevant_features],
    np.full(len(relevant_features), 5.0),
    color="navy",
    label="Relevant features",
)
plot.ylabel("Features")
plot.xlabel("Values of weights")
plot.legend(loc="upper left")

plot.figure(figsize=(6, 5))
plot.title("Marginal log-likelihood")
plot.plot(clf.scores_, color="navy", linewidth=lw)
plot.ylabel("Score")
plot.xlabel("Iterations")
def f(x, noise_amount):
    y = np.sqrt(x) * np.sin(x)
    noise = np.random.normal(0, 1, len(x))
    return y + noise_amount * noise


degree = 10
X = np.linspace(0, 10, 100)
y = f(X, noise_amount=0.1)
clf_poly = BayesianRidge()
clf_poly.fit(np.vander(X, degree), y)

X_plot = np.linspace(0, 11, 25)
y_plot = f(X_plot, noise_amount=0)
y_mean, y_std = clf_poly.predict(np.vander(X_plot, degree), return_std=True)
plot.figure(figsize=(6, 5))
plot.errorbar(
    X_plot,
    y_mean,
    y_std,
    color="navy",
    label="Polynomial Bayesian Ridge Regression",
    linewidth=lw,
)
plot.plot(X_plot, y_plot, color="yellow", linewidth=lw, label="Ground Truth")
plot.ylabel("Output y")
plot.xlabel("Feature X")
plot.legend(loc="lower left")
plot.show()

Output:

After running the above code, we get the following output in which we can see that the bayesian ridge regression is plotted on the screen.

You may also like to read the following tutorials on scikit learn.

• Scikit learn Hierarchical Clustering
• Scikit learn Genetic algorithm
• Scikit learn Classification Tutorial
• Scikit learn Hidden Markov Model
• Scikit learn accuracy_score

So, in this tutorial we discussed scikit learn ridge regression and we have also covered different examples related to its implementation. Here is the list of examples that we have covered.

• Scikit learn ridge regression
• Scikit learn ridge regression coefficient
• Scikit learn kernel ridge regression
• Scikit learn bayesian ridge regression