Get Unique Values from a List in Python | Add only unique values to list in Python [7 Methods]

In this Python blog, I will explain how to get unique values from a list in Python, We will see different methods present in Python to add only unique values to list in Python. Additionally, we will also see some examples to illustrate all the methods. And at last, we will see the pros and cons of each method to get Unique values from a list in Python.

In Python, a list is an ordered collection of items, stored within [] brackets. Each element present in the list has an index associated with it. So, a Python List allows duplicate values within it. But, sometimes we only need unique elements from the list as another Python list.

Methods to Get Unique Values From a List in Python

There are various different methods present in Python to add only unique values to list in Python:

  • Using the set Data Type
  • Using List Comprehension
  • Using dict.fromkeys()
  • Using a Custom Function
  • Using Generators
  • Using Filter Function
  • Using collections.Counter

Let’s see them one by one with some demonstrative examples:

Method 1: Unique Elements in List Python using set() function

The set() function is used to create a set object in Python. This method involves converting a Python list to a Python set to remove duplicates. Converting the list to a set automatically removes duplicates, since a set can’t contain repeated items. However, the original order might not be maintained.

Example: Let’s Consider a situation where a poll is conducted during the US Presidential elections where attendees could indicate which candidates they’d consider voting for through Python. Some people may choose the same candidate more than once due to confusion or error. So, we only need to get unique values from a list in Python.

By using the set data type, the list would have repeated votes removed, giving us a list of unique candidates.

candidates = ['Biden', 'Trump', 'Biden', 'Sanders', 'Trump']
unique_candidates = list(set(candidates))
print(unique_candidates)

Output: In this Python code, we have the ‘candidates‘ list in which we have some duplicate values. we are using the set() function that converts this Python list to a Python set and then we are using the list() function that will again convert that set back into a Python list.

['Sanders', 'Biden', 'Trump']
python list unique

This way we can use the set() with the list() function to get unique values from a list in Python.

Method 2: Python Get Unique Values in List Using List Comprehension

Through list comprehension, we create a new list by iterating over the original and appending elements that haven’t been added before in Python. This way we can easily get unique values from a list in Python and this also preserves order but can be slower for large lists.

Example: Consider a Python list of US states that tourists would like to visit, but some states are entered multiple times.

List comprehension can generate a new list in Python, where each state is added only if it hasn’t been included before, preserving the order of preference.

states = ['California', 'Texas', 'Nevada', 'California', 'Texas']
unique_states = []
[unique_states.append(state) for state in states if state not in unique_states]
print(unique_states)

Output: In this example, the list comprehension in Python helps in iterating over the “states” list, filtering the elements with the if condition, and adding only the unique elements in the new empty list in Python.

['California', 'Texas', 'Nevada']
list comprehension unique values

This way we can use List comprehension in to get unique values from a list in Python.

Method 3: Python Unique List of Objects using dict.fromkeys()

Dictionaries can’t have duplicate keys, so we can leverage this to our advantage. The fromkeys() method of dictionaries can be utilized to eliminate duplicates. This is a concise method that preserves the order of first occurrence.

The dict.fromkeys() converts the elements of the Python list into Python dictionary keys, and then again we will convert them back into a Python list using list().

Scenario: A survey asks people to list their favorite American football teams through Python. Some participants list the same team multiple times and now, we want only unique values in the Python list.

teams = ['Cowboys', 'Patriots', '49ers', 'Patriots', 'Cowboys']
unique_teams = list(dict.fromkeys(teams))
print(unique_teams)

Output: Using the dict.fromkeys() Python method, the original order of ‘teams‘ appearance is preserved while removing duplicates.

['Cowboys', 'Patriots', '49ers']
unique list python

This way we can use the dict.fromkeys() method to get the unique values from a list in Python.

Method 4: Python Unique Elements in List by a Custom Function

Implementing a custom function in Python allows flexibility in preserving order and handling duplicates, typically using a set in Python to track seen items as we iterate over the Python list.

Scenario: In a list of American authors that people enjoy reading in Python, some authors appear multiple times. We need to create a new list to only get unique values from a list in Python.

def get_unique_items(input_list):
    seen = set()
    result = []
    for item in input_list:
        if item not in seen:
            seen.add(item)
            result.append(item)
    return result

authors = ['Hemingway', 'Fitzgerald', 'Hemingway', 'Twain']
print(get_unique_items(authors))

Output: In this Python code, we have defined a function “get_unique_items” in which first we will iterate over the ‘input_list‘, with the help of the if conditional statement. Then, we will first add the item (if not added previously) to the empty set, and then, we add that element after this many operations to an empty list called “result“.

['Hemingway', 'Fitzgerald', 'Twain']
python distinct list

This way we can create our own Python function to get unique values from a list in Python.

Method 5: Unique Strings in List Python using Generator.

In Python, a generator is a special type of iterator that allows you to iterate over items one and generate each item on the fly without storing them in memory. Generators provide a memory-efficient way to work with large data streams or sequences.

Scenario: A music survey asks participants for their favorite American music genres through Python. Some genres are listed multiple times. So, we need to get a list that contains only unique values from an existing list in Python.

def unique_gen(input_list):
    seen = set()
    for item in input_list:
        if item not in seen:
            seen.add(item)
            yield item

genres = ['Rock', 'Jazz', 'Country', 'Rock']
unique_genres = list(unique_gen(genres))
print(unique_genres)

Output: In this example, we have again defined a Python function “unique_gen” to accept a list “input_list” as its argument. The function iterates over each ‘item‘ in the Python List.

For each “item“, the function checks if it’s already in the seen Python set. If the item hasn’t been seen (it’s not in the set), it’s added to the Python set and yielded with the yield keyword.

The yield keyword indicates that the function is a generator and allows it to produce the item as part of its output sequence.

['Rock', 'Jazz', 'Country']
distinct list python

This way we can use Generator to get the unique values from a list in Python.

Method 6: Python unique objects in list using filter function()

The filter() function in Python is a built-in function used to filter a sequence (like a list, tuple, or any iterable) based on a specific condition. This function returns an iterator (specifically, a filter object) that contains the elements from the sequence for which the applied function returns True.

The filter function, combined with a lambda function, can be applied to extract unique items from a List in Python. It processes the Python list and keeps items that haven’t been seen before, preserving their order.

Example: A Python list contains popular American holidays. However, some holidays are repeated. We need to remove all the duplicates and only get unique values from a list in Python.

holidays = ['Christmas', 'Thanksgiving', 'Halloween', 'Christmas']
seen = set()
unique_holidays = list(filter(lambda x: x not in seen and not seen.add(x), holidays))
print(unique_holidays)

Output: The code initializes a ‘seen‘ Python set to track holidays encountered. It then filters the ‘holidays‘ list, retaining only the first occurrence of each holiday by checking against and updating the Python set. The result is a Python list of unique_holidays in their original order.

['Christmas', 'Thanksgiving', 'Halloween']
unique python

This is the way, how we use the filter() function with the lambda() function to get unique values from a list in Python.

Method 7: Python List with only Unique Values using collections.Counter() function

The Counter() is a subclass of the dictionary in the collections module of Python’s standard library. It is used to count the occurrences of elements in an iterable.

Example: A list in Python contains favorite American cars among a group of enthusiasts, with some cars repeated. Let’s get a Python list with only a group of cars that are unique.

from collections import Counter

cars = ['Ford', 'Chevrolet', 'Ford', 'Dodge']
unique_cars = list(Counter(cars).keys())
print(unique_cars)

Output: The Python code imports the Counter class from the collections module to count occurrences of car brands in the cars list. It then extracts the unique car brands by getting the keys from the Counter object and converts them to a Python list, which is printed out.

['Ford', 'Chevrolet', 'Dodge']
Get Unique Values From a List in Python

This way we can use the Counter class from the collections module to get all unique values in the list Python.

Python Unique Values in List Examples

Case 1: Get unique values from a list of dictionaries Python

  • In this section, we will discuss how to get the unique values from a Python list of dictionaries by using Python.
  • To perform this particular task we are going to use the list function and within this method, we are going to apply the list comprehension method as an argument.

Source Code:

new_dict = [{'Micheal' : 432, 'George' : 675}, {'William' : 432, 'John' : 178}, {'William' : 432, 'Smith':105}]

final_output = list(set(value for dic in new_dict for value in dic.values()))
print("Unique values:",final_output)

Output: In the above code, we created a list of dictionaries and then initialized a variable ‘final_output‘ in which we have assigned the list values that can iterate over the dictionary values. Now convert to a list by using the dic.values() method.

Unique values: [432, 105, 178, 675]

Here is the implementation of the following given code:

keep only unique values in list python

Case 2: Python gets unique values from a list of objects

  • In this section, we will discuss how to get the unique values from a list of objects by using Python.
  • By using the Python set() method we can easily perform this particular task. In Python the set method stores only unique values it will automatically remove duplicate values from the list.

Example:

new_values = [67,117,67,'b','z','b',25]

result = set(new_values)
z=list(result)

print("Unique values:",z)

Output: We will create a list and assign integer and string values to it in Python. After that declare a variable ‘result’ in which we are going to apply the set() method and within this function, we will pass list ‘new_values’ as an argument.

Unique values: ['b', 67, 117, 25, 'z']

You can refer to the below Screenshot:

unique list in python

Case 3: Get unique values from two lists of Python

  • Here we can see how to get unique values from two lists by using Python.
  • To solve this problem we are going to apply the set() function. In Python, the set() function is used to get the unique values from given lists.

Example:

lis1 = [23,41,23,78,94,41]
lis2 = [78,109,78,56,234,109]

z=list(set(lis1) - set(lis2))
m=list(set(lis2) - set(lis1))
print("Unique values from two lists:",z,m)

Output: Here, we will create two lists ‘lis1’ and ‘lis2’ in Python. After that, we are going to declare a variable ‘z’ and ‘m’ in which we have assigned a Python list and set a function in Python for getting the unique values.

Unique values from two lists: [41, 94, 23] [56, 234, 109]

Here is the Screenshot of the following code:

list unique python

Case 4: Get unique value from list Python using index

To get the unique value from a Python list with an index number, we can easily use the concept of a Python list.index() method.

Example:

new_list = [2,6,9,2,9,4]

final_output = [new_list.index(i) for i in set(new_list)]
print("Unique values get index from list:",final_output)

Output: Here, we will create a list in Python and assign integer values to it. After that, we are going to use the list comprehension method and it will iterate the list values.

Unique values get index from list: [2, 0, 5, 1]
python list distinct

Get all Unique Values in List Python Methods Pros and Cons

NameProsCons
The set() Data Type1. Slower for larger lists because of the repeated checks.1. Original order might not be preserved.
2. Only works for hashable elements.
List Comprehension1. Preserve the original order.
2. Works for both hashable and non-hashable items.
1. Slower for larger lists because of the repeated checks.
The dict.fromkeys()1. Preserve the original order.
2. Faster than list comprehension for larger lists.
1. Slightly less intuitive than the set method.
The Custom Function1. Customizable.
2. Preserve the original order.
1. Requires more code than other methods.
The Generator1. Memory efficient for large lists.
2. Lazy evaluation: processes items only as they’re needed.
1. Might be less intuitive for beginners.
Filter Function1. Functional approach.
2. Preserve the original order.
1. A bit tricky due to the use of side effects in lambda.
The collections.Counter1. Preserve the original order.
2. Can also get a count of each unique item.
1. A bit overkill if you only need unique values.
Pros and cons for all methods to get unique values from a List in Python.

Conclusion

To get unique values from a list in Python is a frequently encountered task in data processing and programming.

From leveraging built-in Python constructs like the set, a custom function, filter() with lambda function, generators, the dict.fromkeys(), and list comprehensions to utilizing robust libraries like collections, there’s a wide range of tools to make us get unique values from a list in Python.

We have seen illustrative examples for each of the mentioned methods and some additional use cases to better understand the concept. Additionally, we have also seen the pros and cons of each method present to get unique values from a list in Python.

Related Python tutorials: