# Scikit learn Ridge Regression

In this Python tutorial, we will learn How to create scikit learn ridge regression in Python and we will also cover different examples related to ridge regression. And we will cover these topics.

• Scikit learn ridge regression
• Scikit learn ridge regression coefficient
• Scikit learn kernel ridge regression
• Scikit learn bayesian ridge regression

## Scikit learn ridge regression

In this section, we will learn about how to solve the Scikit learn ridge regression in python.

Ridge regression is used to solve this regression model and modify the loss function by adding some penalty equivalent to the square of the magnitude of the coefficients.

Code:

In the following code, we will import some libraries from which we can solve the ridge regression in python.

• n_samples, n_features = 15, 10 is used to add samples and features in this ridge function.
• rng = np.random.RandomState(0) is used for the random states.
• rdg.fit(X, y) is used to fit the values.
``````from sklearn.linear_model import Ridge
import numpy as np
n_samples, n_features = 15, 10
range = np.random.RandomState(0)
y = range.randn(n_samples)
X = range.randn(n_samples, n_features)
ridge = Ridge(alpha = 0.5)
ridge.fit(X, y)
ridge.score(X,y)``````

Output:

After running the above code, we get the following output in which we can see that the ridge regression score is shown on the screen.

Also, check: Scikit-learn Vs Tensorflow

## Scikit learn ridge regression coefficient

In this section, we will learn about how to create scikit learn ridge regression coefficient in python.

Code:

In the following code, we will import the ridge library from sklearn.learn and also import numpy as np.

• n_samples, n_features = 15, 10 is used to add samples and features in the ridge function.
• ridge.score(X,y) is used give get the score of ridge function.
• ridge.coef_ is used to get the coefficient of the ridge function.
``````from sklearn.linear_model import Ridge
import numpy as np
n_samples, n_features = 15, 10
range = np.random.RandomState(0)
y = range.randn(n_samples)
X = range.randn(n_samples, n_features)
ridge = Ridge(alpha = 0.5)
ridge.fit(X, y)
ridge.score(X,y)
ridge.coef_``````

Output:

After running the above code, we get the following output in which we can see that the ridge coefficient is printed on the screen.

## Scikit learn kernel ridge regression

In this section, we will learn about how to create a scikit learn kernel ridge regression in Python.

Kernel Ridge Regression merges the ridge regression with their own trick. The kernel includes the space in a linear function.

Code:

In the following code, we will import KernelRidge from sklearn.kernel_ridge from which we can get the kernel ridge value.

• n_samples, n_features = 10, 5 is used to create the sample and feature of kernel ridge.
• KernelRidge(alpha=1.0) is used to get the kernel ridge value.
``````from sklearn.kernel_ridge import KernelRidge
import numpy as np
n_samples, n_features = 10, 5
range = np.random.RandomState(0)
y = rng.randn(n_samples)
X = rng.randn(n_samples, n_features)
kernel = KernelRidge(alpha=1.0)
kernel.fit(X, y)``````

Output:

After running the above code, we get the following output in which we can see that the kernel ridge value is shown on the screen.

## Scikit learn bayesian ridge regression

In this section, we will learn about how to create a scikit learn Bayesian ridge regression in Python.

Bayesian regression permits the natural mechanism to hold on to insufficient distributed data by formulating linear regression using probability.

Code:

In the following code, we will import some libraries from which we can create a Bayesian ridge regression.

• n_samples, n_features = 100, 100 is used to generating data.
• X = np.random.randn(n_samples, n_features) is used to create gaussian data.
• lambda_ = 4.0 is used to create weights with lambda value.
• relevant_features = np.random.randint(0, n_features, 10) is used to keep the 10 weight of interest.
• clf = BayesianRidge(compute_score=True) is used to fit the bayesian regression.
• plot.figure(figsize=(6, 5)) is used to plot the graph.
``````
import numpy as np
import matplotlib.pyplot as plot
from scipy import stats

from sklearn.linear_model import BayesianRidge, LinearRegression
np.random.seed(0)
n_samples, n_features = 100, 100
X = np.random.randn(n_samples, n_features)  # Create Gaussian data
lambda_ = 4.0
w = np.zeros(n_features)
relevant_features = np.random.randint(0, n_features, 10)
for i in relevant_features:
w[i] = stats.norm.rvs(loc=0, scale=1.0 / np.sqrt(lambda_)).
alpha_ = 50.0
noise = stats.norm.rvs(loc=0, scale=1.0 / np.sqrt(alpha_), size=n_samples)
y = np.dot(X, w) + noise

clf = BayesianRidge(compute_score=True)
clf.fit(X, y)

ols = LinearRegression()
ols.fit(X, y)

lw = 2
plot.figure(figsize=(6, 5))
plot.title("Weights of model")
plot.plot(clf.coef_, color="green", linewidth=lw, label="Bayesian Ridge estimate")
plot.plot(w, color="yellow", linewidth=lw, label="Ground truth")
plot.plot(ols.coef_, color="red", linestyle="--", label="OLS estimate")
plot.xlabel("Features")
plot.ylabel("Values of  weights")
plot.legend(loc="best", prop=dict(size=12))

plot.figure(figsize=(6, 5))
plot.title("Histogram of weights")
plot.hist(clf.coef_, bins=n_features, color="gold", log=True, edgecolor="black")
plot.scatter(
clf.coef_[relevant_features],
np.full(len(relevant_features), 5.0),
color="navy",
label="Relevant features",
)
plot.ylabel("Features")
plot.xlabel("Values of weights")
plot.legend(loc="upper left")

plot.figure(figsize=(6, 5))
plot.title("Marginal log-likelihood")
plot.plot(clf.scores_, color="navy", linewidth=lw)
plot.ylabel("Score")
plot.xlabel("Iterations")
def f(x, noise_amount):
y = np.sqrt(x) * np.sin(x)
noise = np.random.normal(0, 1, len(x))
return y + noise_amount * noise

degree = 10
X = np.linspace(0, 10, 100)
y = f(X, noise_amount=0.1)
clf_poly = BayesianRidge()
clf_poly.fit(np.vander(X, degree), y)

X_plot = np.linspace(0, 11, 25)
y_plot = f(X_plot, noise_amount=0)
y_mean, y_std = clf_poly.predict(np.vander(X_plot, degree), return_std=True)
plot.figure(figsize=(6, 5))
plot.errorbar(
X_plot,
y_mean,
y_std,
color="navy",
label="Polynomial Bayesian Ridge Regression",
linewidth=lw,
)
plot.plot(X_plot, y_plot, color="yellow", linewidth=lw, label="Ground Truth")
plot.ylabel("Output y")
plot.xlabel("Feature X")
plot.legend(loc="lower left")
plot.show()``````

Output:

After running the above code, we get the following output in which we can see that the bayesian ridge regression is plotted on the screen.