Recently, I was working on a data analysis project that required dealing with noisy sensor data. The issue is that raw data often contains random fluctuations, which make it difficult to identify underlying patterns. So we need effective smoothing techniques.
In this article, I’ll cover several simple ways you can use SciPy to smooth your data in Python (from basic moving averages to advanced filters).
So let’s dive in!
Data Smoothing Methods
Data smoothing helps us reduce noise and random variations in our data while preserving important patterns and trends. Think of it as removing the static from a radio signal to hear the music more clearly.
For example, if you’re analyzing daily temperature readings from weather stations across California, the raw data might show random spikes due to sensor errors or temporary weather anomalies. Smoothing this data would help reveal the actual temperature trends.
SciPy provides several powerful tools that make it easy to implement various smoothing techniques without having to code everything from scratch.
Read SciPy Stats
1 – Moving Average with SciPy’s uniform_filter1d
A moving average is one of the simplest and most intuitive smoothing techniques. It replaces each data point with the average of neighboring points.
Here’s how to implement it using SciPy:
import numpy as np
import matplotlib.pyplot as plt
from scipy.ndimage import uniform_filter1d
# Generate some noisy data
x = np.linspace(0, 10, 200)
y_clean = np.sin(x)
noise = np.random.normal(0, 0.2, size=len(x))
y_noisy = y_clean + noise
# Apply moving average smoothing
window_size = 15
y_smoothed = uniform_filter1d(y_noisy, size=window_size)
# Plot the results
plt.figure(figsize=(10, 6))
plt.plot(x, y_noisy, 'gray', alpha=0.5, label='Noisy Data')
plt.plot(x, y_clean, 'g', label='Original Signal')
plt.plot(x, y_smoothed, 'r', linewidth=2, label=f'Moving Average (window={window_size})')
plt.legend()
plt.title('Moving Average Smoothing with SciPy')
plt.xlabel('Time')
plt.ylabel('Value')
plt.grid(True, alpha=0.3)
plt.show()I executed the above example code and added the screenshot below.
The uniform_filter1d function applies a uniform moving average filter with the specified window size. Larger window sizes produce smoother results but might remove important features of your data.
Check out SciPy Stats
2 – Gaussian Smoothing with SciPy’s gaussian_filter1d
Gaussian smoothing applies a weighted average where points closer to the center of the window have more influence than those at the edges. This often produces more natural-looking results than simple moving averages.
from scipy.ndimage import gaussian_filter1d
# Apply Gaussian smoothing
sigma = 3 # Standard deviation for Gaussian kernel
y_smoothed_gaussian = gaussian_filter1d(y_noisy, sigma=sigma)
# Plot the results
plt.figure(figsize=(10, 6))
plt.plot(x, y_noisy, 'gray', alpha=0.5, label='Noisy Data')
plt.plot(x, y_clean, 'g', label='Original Signal')
plt.plot(x, y_smoothed_gaussian, 'b', linewidth=2, label=f'Gaussian Smoothing (sigma={sigma})')
plt.legend()
plt.title('Gaussian Smoothing with SciPy')
plt.xlabel('Time')
plt.ylabel('Value')
plt.grid(True, alpha=0.3)
plt.show()I executed the above example code and added the screenshot below.
The sigma parameter controls the width of the Gaussian kernel. Higher values result in stronger smoothing effects.
3 – Savitzky-Golay Filter with SciPy
The Savitzky-Golay filter is particularly effective for preserving the shape of peaks in your data while reducing noise. It fits a polynomial to a moving window of data points and uses that to estimate the smoothed value.
from scipy.signal import savgol_filter
# Apply Savitzky-Golay filter
window_length = 15 # Must be odd
poly_order = 3 # Polynomial order
y_smoothed_savgol = savgol_filter(y_noisy, window_length, poly_order)
# Plot the results
plt.figure(figsize=(10, 6))
plt.plot(x, y_noisy, 'gray', alpha=0.5, label='Noisy Data')
plt.plot(x, y_clean, 'g', label='Original Signal')
plt.plot(x, y_smoothed_savgol, 'purple', linewidth=2,
label=f'Savitzky-Golay (window={window_length}, order={poly_order})')
plt.legend()
plt.title('Savitzky-Golay Smoothing with SciPy')
plt.xlabel('Time')
plt.ylabel('Value')
plt.grid(True, alpha=0.3)
plt.show()I executed the above example code and added the screenshot below.
The window_length must be odd and the poly_order must be less than the window length. This filter is excellent for preserving features like peaks in spectroscopic data.
Read SciPy Integrate
4 – Spline Smoothing with SciPy
Spline smoothing fits a smooth curve through your data points. It’s particularly useful when you want a continuous function that represents your smoothed data.
from scipy.interpolate import UnivariateSpline
# Apply spline smoothing
smoothing_factor = 0.5
spline = UnivariateSpline(x, y_noisy, s=smoothing_factor)
y_smoothed_spline = spline(x)
# Plot the results
plt.figure(figsize=(10, 6))
plt.plot(x, y_noisy, 'gray', alpha=0.5, label='Noisy Data')
plt.plot(x, y_clean, 'g', label='Original Signal')
plt.plot(x, y_smoothed_spline, 'orange', linewidth=2,
label=f'Spline Smoothing (s={smoothing_factor})')
plt.legend()
plt.title('Spline Smoothing with SciPy')
plt.xlabel('Time')
plt.ylabel('Value')
plt.grid(True, alpha=0.3)
plt.show()The s parameter controls the smoothing factor. Higher values produce smoother curves, while lower values follow the original data more closely.
Read SciPy Signal
Compare Different Smoothing Methods
Each smoothing method has its strengths and weaknesses. Let’s compare them side by side:
plt.figure(figsize=(12, 8))
plt.plot(x, y_noisy, 'gray', alpha=0.3, label='Noisy Data')
plt.plot(x, y_clean, 'k--', label='Original Signal')
plt.plot(x, y_smoothed, 'r', linewidth=1.5, label=f'Moving Average')
plt.plot(x, y_smoothed_gaussian, 'b', linewidth=1.5, label=f'Gaussian')
plt.plot(x, y_smoothed_savgol, 'purple', linewidth=1.5, label=f'Savitzky-Golay')
plt.plot(x, y_smoothed_spline, 'orange', linewidth=1.5, label=f'Spline')
plt.legend()
plt.title('Comparison of Different Smoothing Methods')
plt.xlabel('Time')
plt.ylabel('Value')
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()Practical Application: Smoothing Stock Market Data
Let’s look at a practical example using real stock market data for a company like Apple:
import pandas as pd
import yfinance as yf
# Download Apple stock data for the past year
apple = yf.download('AAPL', period='1y')
# Extract the closing prices
close_prices = apple['Close'].values
dates = np.array(range(len(close_prices)))
# Apply different smoothing techniques
ma_smoothed = uniform_filter1d(close_prices, size=10)
gaussian_smoothed = gaussian_filter1d(close_prices, sigma=3)
savgol_smoothed = savgol_filter(close_prices, 15, 3)
# Plot the results
plt.figure(figsize=(12, 8))
plt.plot(dates, close_prices, 'gray', alpha=0.5, label='Original Prices')
plt.plot(dates, ma_smoothed, 'r', linewidth=2, label='Moving Average')
plt.plot(dates, gaussian_smoothed, 'b', linewidth=2, label='Gaussian Smoothing')
plt.plot(dates, savgol_smoothed, 'purple', linewidth=2, label='Savitzky-Golay')
plt.legend()
plt.title('Apple Stock Price Smoothing')
plt.xlabel('Trading Days')
plt.ylabel('Price ($)')
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()I hope you found this article helpful. SciPy’s smoothing functions provide powerful tools that make it easy to clean up noisy data and reveal underlying patterns. Each method has its strengths – moving averages are simple and intuitive, Gaussian filters provide natural smoothing, Savitzky-Golay preserves peak shapes, and splines offer continuous functional representations.
The choice of smoothing method depends on your specific data and what features you want to preserve. When working with real-world data, I recommend trying multiple approaches and comparing the results to find the best fit for your analysis.
Other Python articles you may also like:
I am Bijay Kumar, a Microsoft MVP in SharePoint. Apart from SharePoint, I started working on Python, Machine learning, and artificial intelligence for the last 5 years. During this time I got expertise in various Python libraries also like Tkinter, Pandas, NumPy, Turtle, Django, Matplotlib, Tensorflow, Scipy, Scikit-Learn, etc… for various clients in the United States, Canada, the United Kingdom, Australia, New Zealand, etc. Check out my profile.
