How to Remove NaN Values from a List in Python [6 Ways]

In this Python tutorial, we will learn multiple ways to remove NaN values from a list in Python using inbuilt methods and different ways.

In Python, NaN stands for Not a Number. NaN value indicates missing or undefined values of the dataset. Let’s see how we will remove NaN values using the loop, list comprehension, filter method, the isnan() method from the math library, and the inbuilt method of the Numpy and Pandas libraries.

Different Ways to Remove NaN values from a list in Python

  • using for loop
  • using list comprehension
  • using filter method
  • using isnan() of math library
  • using Numpy
  • using Pandas

Remove NaN Values from a List in Python using the For loop

The for loop in Python is a common and effective way to remove NaN values from a list.

Here’s an example demonstrating how to remove NaN values from a list in Python using a for loop:

original_data = [1, 2, 3, 4, float('nan'), 6]
cleaned_data = []
for x in original_data:
    if x == x:
        cleaned_data.append(x)
print(cleaned_data)

In this code, we iterate through each element in the original_data list in Python. The condition if x == x checks if the element is not NaN (since NaN does not equal itself)

If the condition is true, we append the element to the cleaned_data list in Python.

# When it iterates to NaN == NaN, then it will return False 
if x == x:
        cleaned_data.append(x)

Output

[1, 2, 3, 4, 6]

Below is a screenshot that captures the outcome after the code was implemented in the Python editor.

How to remove nan from a list in python

Remove NaN values using List Comprehension in Python

List comprehension is a concise and elegant way to create lists in Python. It offers a more compact syntax compared to loops. In the context of removing NaN values from a list.

Let’s see an example to remove null values in a Python list:

Code

employee_age = [42, 35, float('nan'), 26, float('nan'), 28]
filtered_list = [x for x in employee_age if x == x]
print(filtered_list)

We use the same logic we used in the previous example, using list comprehension in Python with fewer lines of code and a faster approach.

Output

[42, 35, 26, 28]

The resulting output is displayed in the screenshot below after running the code in Python editor.

remove NaN value in python

Remove NaN values from the list using the filter method in Python

The filter() method is a built-in function of Python that applies a specified function to each item of the collection and returns an iterator containing only the items for which the function evaluates to True.

Let’s see an instance,

Code

week_sales = [5000, 8000, float('nan'), 2000, float('nan'), 3000, 9000]
filtered_list = list(filter(lambda x: x == x, week_sales ))
print(filtered_list)

The filter() method in Python removes NaN values from the week_sales list by applying a lambda function that checks each element for equality with itself.

Since NaN does not equal itself, the lambda function in Python filters out NaN values. Then, filtered_list is converted back to a list for further processing.

Output

[5000, 8000, 2000, 3000, 9000]

Below is a screenshot that reveals the output after the code has been implemented in the Python editor.

drop nan values from the list in python

Remove NaN from list using the isnan() math module Python

To remove NaN values from the Python list, we can use the isnan() function, which is an inbuilt method of the math module in Python.

It will check all the elements and return True, where it will find NaN. Let’s create a program to remove nan values using the isnan() method in Python.

from math import isnan

original_list = [1, 2, float('nan'), 4, float('nan'), 6]
cleaned_list = []
for i in original_list:
    if not isnan(i):
        cleaned_list+=[i]

print(cleaned_list)

I’ve initialized the original_list list in Python with numerical values, including NaN values that float(‘nan’) represents.

Then, iterate through each element i in original_list. the isnan() function in Python is used to check if i is not a NaN value.

Output

[1, 2, 4, 6]

The screenshot below presents the output obtained after the code execution in the Python editor.

isnan method to Remove NaN Values from a List in Python

Remove NaN values from the Numpy array

To remove NaN values from a list in Python from the NumPy array, we will use the isnan() method of the numpy library to check whether the element is NaN or not and will return True or False based on whether the element is NaN.

If an element is NaN in the array, it will return True.

Here is an example of using the isnan() function.

Code

import numpy as np
from numpy import nan
data = np.array([5, 12, nan, 7,nan,9])
 
filtered_data = data[np.logical_not(np.isnan(data))]
print(filtered_data)

We used NumPy’s isnan() function in Python to identify NaN values within the array data. The np.logical_not() function is used to negate this result.

Output

[ 5. 12.  7.  9.]

Note: Python NumPy library should be installed in your system to use its inbuilt functions.

How to remove NaN values from Numpy array in Python

How to remove NaN from the list in Pandas

To remove NaN from a list using Pandas Python, there is one inbuilt function called dropna(), which will directly remove the NaN values from the series in Python, and then you can convert it to the list using the tolist() method.

Here is an instance to remove NaN values from a list in Python using the pandas library:

Code

import pandas as pd
original_list = [1, 2, float('nan'), 4, float('nan'), 6]
series = pd.Series(original_list)
cleaned_list = series.dropna().tolist()
print(cleaned_list)

We’ve used the dropna() function in Python Pandas to remove any NaN values from the series of original lists, resulting in a cleaned_list. Finally, the tolist() function in Python converts the cleaned series into a list format.

Output

[1.0, 2.0, 4.0, 6.0]

Here is the screenshot taken after the code was executed in Python editor.

How to remove NaN from the list using pandas in python

Conclusion

Here, We’ve covered all the different ways to remove the NaN value from the list in Python using for loop, list comprehension, filter method, isnan() method from math module, isnan() method from numpy, and dropna() method from Pandas. Each method offers its advantages and can be chosen based on specific requirements.

I hope you understand everything and will use this in your codes.

You may also like to read: