Pandas DataFrame filter() method

In this Python Pandas tutorial, you will learn the Pandas DataFrame filter() method.

In addition, you will learn how to use the filter() method, with its inbuilt parameters it will accept, and how to use all the parameters like items, like, and regex in the Python Pandas filter method.

The filter() method is an inbuilt method of Pandas Library. This method allows you to filter rows or columns based on the labels or particular conditions. It is used for selecting and manipulating the data based on specific criteria.

Data Filteration using Pandas DataFrame filter() method

There are three different parameters that you can pass to the Python Pandas DataFrame.filter() method to filter the data.

  • using items
  • using like
  • combine with regex

1. Filter DataFrame using items parameter in the filter() method

In Pandas, the items parameter is the part of filter method and it is used to filter the DataFrame based on the specific column names or index labels. items will accept a list of column names or index label names as values.

Let me give you some practical examples of using item parameters in the Pandas filter() method.

Example 1: Giving column names in items parameter of Pandas filter() method.

import pandas as pd
Employee_data = {'Name': ['Grace', 'Henry', 'George', 'John', 'Lisa'],
        'Age': [25, 30, 22, 26, 24],
        'Salary': [50000, 60000, 45000, 42000, 30000]}
df = pd.DataFrame(Employee_data)
filtered_data = df.filter(items = ['Name' , 'Salary'])
print(filtered_data)

Output

     Name  Salary
0   Grace   50000
1   Henry   60000
2  George   45000
3    John   42000
4    Lisa   30000

Screenshot: using item parameters with column names as values.

Filter DataFrame with filter() method using items parameter in pandas

Example 2: Giving label names in items parameter of Pandas filter() method.

import pandas as pd
Employee_data = {'Name': ['Grace', 'Henry', 'George', 'John', 'Lisa'],
        'Age': [25, 30, 22, 26, 24],
        'Salary': [50000, 60000, 45000, 42000, 30000]}
index_labels = ['EMP1', 'EMP2', 'EMP3', 'EMP4', 'EMP5']
df = pd.DataFrame(Employee_data, index=index_labels)
filtered_data = df.filter(items=['EMP1', 'EMP3', 'EMP5'], axis=0)
print(filtered_data)

Note: In this example, I have used the axis parameter also, because when we want to filter DataFrame based on label names, we have to set axis = 0. axis = 0 refers to a row, and axis = 1 refers to a column.

Output

         Name  Age  Salary
EMP1   Grace   25   50000
EMP3  George   22   45000
EMP5    Lisa   24   30000

Screenshot: using items parameter with label names with axis parameter as values.

filter() method with items as label name in pandas

Example 3: Giving index values in items parameter of Pandas filter() method.

import pandas as pd
Employee_data = {'Name': ['Grace', 'Henry', 'George', 'John', 'Lisa'],
        'Age': [25, 30, 22, 26, 24],
        'Salary': [50000, 60000, 45000, 42000, 30000]}
df = pd.DataFrame(Employee_data)
filtered_data = df.filter(items=[0,2,4], axis=0)
print(filtered_data)

Output

     Name  Age  Salary
0   Grace   25   50000
2  George   22   45000
4    Lisa   24   30000

Screenshot: using item parameters with index values.

filter() method with items parameter using row index in python pandas

2. Filter DataFrame using a like parameter in the filter() method in Pandas

In Python Pandas, the like parameter is used to filter columns based on the given string as the value of the like parameter.

It will match the column names with the given string pattern in a like parameter. If you want to filter the row, then you have to mention axis = 0 which refers to the row and then you can give the index name as a string.

Let me give you some practical examples of using like parameters in the Pandas filter() method.

Example 1: Let’s try to filter columns of DataFrame using like parameter of filter() method in Python Pandas

import pandas as pd
Employee_data = {'Name': ['Grace', 'Henry', 'George', 'John', 'Lisa'],
        'Age': [25, 30, 22, 26, 24],
        'Salary_2023': [50000, 60000, 45000, 42000, 30000],
        'Salary_2024': [70000, 80000, 65000, 62000, 50000]}
df = pd.DataFrame(Employee_data)
filtered_data = df.filter(like='Sal')
print(filtered_data)

Output

It will match the pattern “Sal” with the column names of the DataFrame.

   Salary_2023  Salary_2024
0        50000        70000
1        60000        80000
2        45000        65000
3        42000        62000
4        30000        50000

Screenshot: using like parameter to filter columns.

filter method using like parameter in pandas

Example 2: Giving label names in like parameter of Pandas filter() method.

import pandas as pd
Employee_data = {'Name': ['Grace', 'Henry', 'George', 'John', 'Lisa'],
        'Age': [25, 30, 22, 26, 24],
        'Salary_2023': [50000, 60000, 45000, 42000, 30000],
        'Salary_2024': [70000, 80000, 65000, 62000, 50000]}
index_labels = ['EMP1', 'EMP2', 'EMP3', 'EMP4', 'EMP5']
df = pd.DataFrame(Employee_data, index=index_labels)
filtered_data = df.filter(like='P3', axis=0)
print(filtered_data)

Output

In this example, we gave the index name manually and filtered the data by matching the pattern(“P3”), so it selected the EMP3 index.

        Name  Age  Salary_2023  Salary_2024
EMP3  George   22        45000        65000

Screenshot: using like parameter with label names with axis parameter as values.

filter dataframe using filter method with like parameter in pandas

Example 3: Giving index values in like parameter of Pandas filter() method.

import pandas as pd
Employee_data = {'Name': ['Grace', 'Henry', 'George', 'John', 'Lisa'],
        'Age': [25, 30, 22, 26, 24],
        'Salary_2023': [50000, 60000, 45000, 42000, 30000],
        'Salary_2024': [70000, 80000, 65000, 62000, 50000]}
df = pd.DataFrame(Employee_data)
filtered_data = df.filter(like='2', axis=0)
print(filtered_data)

Output

In this example, we’ve filtered the row index data, it will check for the “2” index in DataFrame and filter out that row.

     Name  Age  Salary_2023  Salary_2024
2  George   22        45000        65000

Screenshot: using like parameter with the index value.

Pandas DataFrame filter() method

3. Combine Regex in filter() method to Filter DataFrame in Pandas

When you want to perform advanced filtering using regular expressions then you have to use regex. We will use regex as a parameter of the filter() method for DataFrame Filteration in Python Pandas. Regex is a powerful tool for pattern matching.

Here are some practical examples of regex with filter() method in Pandas:\

Example 1: Using “^” as the value of the regex in the filter() method, which will match the starting pattern of the line.

import pandas as pd
Employee_data = {'Name': ['Grace', 'Henry', 'George', 'John', 'Lisa'],
        'Age': [25, 30, 22, 26, 24],
        'Salary_2023': [50000, 60000, 45000, 42000, 30000],
        'Salary_2024': [70000, 80000, 65000, 62000, 50000]}
df = pd.DataFrame(Employee_data)
filtered_data = df.filter(regex="^S")
print(filtered_data)

Output

It will select the column names which start with “S”.

   Salary_2023  Salary_2024
0        50000        70000
1        60000        80000
2        45000        65000
3        42000        62000
4        30000        50000

Screenshot: Using “^” as the value of the regex in filter()

filter dataframe using filter method with regex in python pandas

Example 2: Using “$” as the value of the regex in the filter() method, which will match the ending pattern of the line.

import pandas as pd
Employee_data = {'Name': ['Grace', 'Henry', 'George', 'John', 'Lisa'],
        'Age': [25, 30, 22, 26, 24],
        'Salary_2023': [50000, 60000, 45000, 42000, 30000],
        'Salary_2024': [70000, 80000, 65000, 62000, 50000]}
df = pd.DataFrame(Employee_data)
filtered_data = df.filter(regex="e$")
print(filtered_data)

Output

It will select the columns which end with “e”.

     Name  Age
0   Grace   25
1   Henry   30
2  George   22
3    John   26
4    Lisa   24

Screenshot: Using “$” as the value of the regex in filter()

filter dataframe using filter method with regex in python pandas

Conclusion

In this Pandas tutorial, we’ve covered Python Pandas DataFrame Filtration using the filter() method with different parameters called items, like, and regex. Also, we’ve explained the use of axis parameters with practical examples.

You may like to read: