In this Python tutorial, we will learn How to create scikit learn ridge regression in Python and we will also cover different examples related to ridge regression. And we will cover these topics.
- Scikit learn ridge regression
- Scikit learn ridge regression coefficient
- Scikit learn kernel ridge regression
- Scikit learn bayesian ridge regression
Scikit learn ridge regression
In this section, we will learn about how to solve the Scikit learn ridge regression in python.
Ridge regression is used to solve this regression model and modify the loss function by adding some penalty equivalent to the square of the magnitude of the coefficients.
Code:
In the following code, we will import some libraries from which we can solve the ridge regression in python.
- n_samples, n_features = 15, 10 is used to add samples and features in this ridge function.
- rng = np.random.RandomState(0) is used for the random states.
- rdg.fit(X, y) is used to fit the values.
from sklearn.linear_model import Ridge
import numpy as np
n_samples, n_features = 15, 10
range = np.random.RandomState(0)
y = range.randn(n_samples)
X = range.randn(n_samples, n_features)
ridge = Ridge(alpha = 0.5)
ridge.fit(X, y)
ridge.score(X,y)
Output:
After running the above code, we get the following output in which we can see that the ridge regression score is shown on the screen.
Also, check: Scikit-learn Vs Tensorflow
Scikit learn ridge regression coefficient
In this section, we will learn about how to create scikit learn ridge regression coefficient in python.
Code:
In the following code, we will import the ridge library from sklearn.learn and also import numpy as np.
- n_samples, n_features = 15, 10 is used to add samples and features in the ridge function.
- ridge.score(X,y) is used give get the score of ridge function.
- ridge.coef_ is used to get the coefficient of the ridge function.
from sklearn.linear_model import Ridge
import numpy as np
n_samples, n_features = 15, 10
range = np.random.RandomState(0)
y = range.randn(n_samples)
X = range.randn(n_samples, n_features)
ridge = Ridge(alpha = 0.5)
ridge.fit(X, y)
ridge.score(X,y)
ridge.coef_
Output:
After running the above code, we get the following output in which we can see that the ridge coefficient is printed on the screen.
Read: Scikit-learn logistic regression
Scikit learn kernel ridge regression
In this section, we will learn about how to create a scikit learn kernel ridge regression in Python.
Kernel Ridge Regression merges the ridge regression with their own trick. The kernel includes the space in a linear function.
Code:
In the following code, we will import KernelRidge from sklearn.kernel_ridge from which we can get the kernel ridge value.
- n_samples, n_features = 10, 5 is used to create the sample and feature of kernel ridge.
- KernelRidge(alpha=1.0) is used to get the kernel ridge value.
from sklearn.kernel_ridge import KernelRidge
import numpy as np
n_samples, n_features = 10, 5
range = np.random.RandomState(0)
y = rng.randn(n_samples)
X = rng.randn(n_samples, n_features)
kernel = KernelRidge(alpha=1.0)
kernel.fit(X, y)
Output:
After running the above code, we get the following output in which we can see that the kernel ridge value is shown on the screen.
Read: Scikit learn Decision Tree
Scikit learn bayesian ridge regression
In this section, we will learn about how to create a scikit learn Bayesian ridge regression in Python.
Bayesian regression permits the natural mechanism to hold on to insufficient distributed data by formulating linear regression using probability.
Code:
In the following code, we will import some libraries from which we can create a Bayesian ridge regression.
- n_samples, n_features = 100, 100 is used to generating data.
- X = np.random.randn(n_samples, n_features) is used to create gaussian data.
- lambda_ = 4.0 is used to create weights with lambda value.
- relevant_features = np.random.randint(0, n_features, 10) is used to keep the 10 weight of interest.
- clf = BayesianRidge(compute_score=True) is used to fit the bayesian regression.
- plot.figure(figsize=(6, 5)) is used to plot the graph.
import numpy as np
import matplotlib.pyplot as plot
from scipy import stats
from sklearn.linear_model import BayesianRidge, LinearRegression
np.random.seed(0)
n_samples, n_features = 100, 100
X = np.random.randn(n_samples, n_features) # Create Gaussian data
lambda_ = 4.0
w = np.zeros(n_features)
relevant_features = np.random.randint(0, n_features, 10)
for i in relevant_features:
w[i] = stats.norm.rvs(loc=0, scale=1.0 / np.sqrt(lambda_)).
alpha_ = 50.0
noise = stats.norm.rvs(loc=0, scale=1.0 / np.sqrt(alpha_), size=n_samples)
y = np.dot(X, w) + noise
clf = BayesianRidge(compute_score=True)
clf.fit(X, y)
ols = LinearRegression()
ols.fit(X, y)
lw = 2
plot.figure(figsize=(6, 5))
plot.title("Weights of model")
plot.plot(clf.coef_, color="green", linewidth=lw, label="Bayesian Ridge estimate")
plot.plot(w, color="yellow", linewidth=lw, label="Ground truth")
plot.plot(ols.coef_, color="red", linestyle="--", label="OLS estimate")
plot.xlabel("Features")
plot.ylabel("Values of weights")
plot.legend(loc="best", prop=dict(size=12))
plot.figure(figsize=(6, 5))
plot.title("Histogram of weights")
plot.hist(clf.coef_, bins=n_features, color="gold", log=True, edgecolor="black")
plot.scatter(
clf.coef_[relevant_features],
np.full(len(relevant_features), 5.0),
color="navy",
label="Relevant features",
)
plot.ylabel("Features")
plot.xlabel("Values of weights")
plot.legend(loc="upper left")
plot.figure(figsize=(6, 5))
plot.title("Marginal log-likelihood")
plot.plot(clf.scores_, color="navy", linewidth=lw)
plot.ylabel("Score")
plot.xlabel("Iterations")
def f(x, noise_amount):
y = np.sqrt(x) * np.sin(x)
noise = np.random.normal(0, 1, len(x))
return y + noise_amount * noise
degree = 10
X = np.linspace(0, 10, 100)
y = f(X, noise_amount=0.1)
clf_poly = BayesianRidge()
clf_poly.fit(np.vander(X, degree), y)
X_plot = np.linspace(0, 11, 25)
y_plot = f(X_plot, noise_amount=0)
y_mean, y_std = clf_poly.predict(np.vander(X_plot, degree), return_std=True)
plot.figure(figsize=(6, 5))
plot.errorbar(
X_plot,
y_mean,
y_std,
color="navy",
label="Polynomial Bayesian Ridge Regression",
linewidth=lw,
)
plot.plot(X_plot, y_plot, color="yellow", linewidth=lw, label="Ground Truth")
plot.ylabel("Output y")
plot.xlabel("Feature X")
plot.legend(loc="lower left")
plot.show()
Output:
After running the above code, we get the following output in which we can see that the bayesian ridge regression is plotted on the screen.
You may also like to read the following tutorials on scikit learn.
- Scikit learn Hierarchical Clustering
- Scikit learn Genetic algorithm
- Scikit learn Classification Tutorial
- Scikit learn Hidden Markov Model
- Scikit learn accuracy_score
So, in this tutorial we discussed scikit learn ridge regression and we have also covered different examples related to its implementation. Here is the list of examples that we have covered.
- Scikit learn ridge regression
- Scikit learn ridge regression coefficient
- Scikit learn kernel ridge regression
- Scikit learn bayesian ridge regression
I am Bijay Kumar, a Microsoft MVP in SharePoint. Apart from SharePoint, I started working on Python, Machine learning, and artificial intelligence for the last 5 years. During this time I got expertise in various Python libraries also like Tkinter, Pandas, NumPy, Turtle, Django, Matplotlib, Tensorflow, Scipy, Scikit-Learn, etc… for various clients in the United States, Canada, the United Kingdom, Australia, New Zealand, etc. Check out my profile.