In this Python blog, I will explain how to get unique values from a list in Python, We will see different methods present in Python to add only unique values to list in Python. Additionally, we will also see some examples to illustrate all the methods. And at last, we will see the pros and cons of each method to get Unique values from a list in Python.
In Python, a list is an ordered collection of items, stored within [] brackets. Each element present in the list has an index associated with it. So, a Python List allows duplicate values within it. But, sometimes we only need unique elements from the list as another Python list.
Methods to Get Unique Values From a List in Python
There are various different methods present in Python to add only unique values to list in Python:
- Using the set Data Type
- Using List Comprehension
- Using dict.fromkeys()
- Using a Custom Function
- Using Generators
- Using Filter Function
- Using collections.Counter
Let’s see them one by one with some demonstrative examples:
Method 1: Unique Elements in List Python using set() function
The set() function is used to create a set object in Python. This method involves converting a Python list to a Python set to remove duplicates. Converting the list to a set automatically removes duplicates, since a set can’t contain repeated items. However, the original order might not be maintained.
Example: Let’s Consider a situation where a poll is conducted during the US Presidential elections where attendees could indicate which candidates they’d consider voting for through Python. Some people may choose the same candidate more than once due to confusion or error. So, we only need to get unique values from a list in Python.
By using the set data type, the list would have repeated votes removed, giving us a list of unique candidates.
candidates = ['Biden', 'Trump', 'Biden', 'Sanders', 'Trump']
unique_candidates = list(set(candidates))
print(unique_candidates)Output: In this Python code, we have the ‘candidates‘ list in which we have some duplicate values. we are using the set() function that converts this Python list to a Python set and then we are using the list() function that will again convert that set back into a Python list.
['Sanders', 'Biden', 'Trump']
This way we can use the set() with the list() function to get unique values from a list in Python.
Method 2: Python Get Unique Values in List Using List Comprehension
Through list comprehension, we create a new list by iterating over the original and appending elements that haven’t been added before in Python. This way we can easily get unique values from a list in Python and this also preserves order but can be slower for large lists.
Example: Consider a Python list of US states that tourists would like to visit, but some states are entered multiple times.
List comprehension can generate a new list in Python, where each state is added only if it hasn’t been included before, preserving the order of preference.
states = ['California', 'Texas', 'Nevada', 'California', 'Texas']
unique_states = []
[unique_states.append(state) for state in states if state not in unique_states]
print(unique_states)Output: In this example, the list comprehension in Python helps in iterating over the “states” list, filtering the elements with the if condition, and adding only the unique elements in the new empty list in Python.
['California', 'Texas', 'Nevada']
This way we can use List comprehension in to get unique values from a list in Python.
Method 3: Python Unique List of Objects using dict.fromkeys()
Dictionaries can’t have duplicate keys, so we can leverage this to our advantage. The fromkeys() method of dictionaries can be utilized to eliminate duplicates. This is a concise method that preserves the order of first occurrence.
The dict.fromkeys() converts the elements of the Python list into Python dictionary keys, and then again we will convert them back into a Python list using list().
Scenario: A survey asks people to list their favorite American football teams through Python. Some participants list the same team multiple times and now, we want only unique values in the Python list.
teams = ['Cowboys', 'Patriots', '49ers', 'Patriots', 'Cowboys']
unique_teams = list(dict.fromkeys(teams))
print(unique_teams)Output: Using the dict.fromkeys() Python method, the original order of ‘teams‘ appearance is preserved while removing duplicates.
['Cowboys', 'Patriots', '49ers']
This way we can use the dict.fromkeys() method to get the unique values from a list in Python.
Method 4: Python Unique Elements in List by a Custom Function
Implementing a custom function in Python allows flexibility in preserving order and handling duplicates, typically using a set in Python to track seen items as we iterate over the Python list.
Scenario: In a list of American authors that people enjoy reading in Python, some authors appear multiple times. We need to create a new list to only get unique values from a list in Python.
def get_unique_items(input_list):
seen = set()
result = []
for item in input_list:
if item not in seen:
seen.add(item)
result.append(item)
return result
authors = ['Hemingway', 'Fitzgerald', 'Hemingway', 'Twain']
print(get_unique_items(authors))Output: In this Python code, we have defined a function “get_unique_items” in which first we will iterate over the ‘input_list‘, with the help of the if conditional statement. Then, we will first add the item (if not added previously) to the empty set, and then, we add that element after this many operations to an empty list called “result“.
['Hemingway', 'Fitzgerald', 'Twain']
This way we can create our own Python function to get unique values from a list in Python.
Method 5: Unique Strings in List Python using Generator.
In Python, a generator is a special type of iterator that allows you to iterate over items one and generate each item on the fly without storing them in memory. Generators provide a memory-efficient way to work with large data streams or sequences.
Scenario: A music survey asks participants for their favorite American music genres through Python. Some genres are listed multiple times. So, we need to get a list that contains only unique values from an existing list in Python.
def unique_gen(input_list):
seen = set()
for item in input_list:
if item not in seen:
seen.add(item)
yield item
genres = ['Rock', 'Jazz', 'Country', 'Rock']
unique_genres = list(unique_gen(genres))
print(unique_genres)Output: In this example, we have again defined a Python function “unique_gen” to accept a list “input_list” as its argument. The function iterates over each ‘item‘ in the Python List.
For each “item“, the function checks if it’s already in the seen Python set. If the item hasn’t been seen (it’s not in the set), it’s added to the Python set and yielded with the yield keyword.
The yield keyword indicates that the function is a generator and allows it to produce the item as part of its output sequence.
['Rock', 'Jazz', 'Country']
This way we can use Generator to get the unique values from a list in Python.
Method 6: Python unique objects in list using filter function()
The filter() function in Python is a built-in function used to filter a sequence (like a list, tuple, or any iterable) based on a specific condition. This function returns an iterator (specifically, a filter object) that contains the elements from the sequence for which the applied function returns True.
The filter function, combined with a lambda function, can be applied to extract unique items from a List in Python. It processes the Python list and keeps items that haven’t been seen before, preserving their order.
Example: A Python list contains popular American holidays. However, some holidays are repeated. We need to remove all the duplicates and only get unique values from a list in Python.
holidays = ['Christmas', 'Thanksgiving', 'Halloween', 'Christmas']
seen = set()
unique_holidays = list(filter(lambda x: x not in seen and not seen.add(x), holidays))
print(unique_holidays)Output: The code initializes a ‘seen‘ Python set to track holidays encountered. It then filters the ‘holidays‘ list, retaining only the first occurrence of each holiday by checking against and updating the Python set. The result is a Python list of unique_holidays in their original order.
['Christmas', 'Thanksgiving', 'Halloween']
This is the way, how we use the filter() function with the lambda() function to get unique values from a list in Python.
Method 7: Python List with only Unique Values using collections.Counter() function
The Counter() is a subclass of the dictionary in the collections module of Python’s standard library. It is used to count the occurrences of elements in an iterable.
Example: A list in Python contains favorite American cars among a group of enthusiasts, with some cars repeated. Let’s get a Python list with only a group of cars that are unique.
from collections import Counter
cars = ['Ford', 'Chevrolet', 'Ford', 'Dodge']
unique_cars = list(Counter(cars).keys())
print(unique_cars)Output: The Python code imports the Counter class from the collections module to count occurrences of car brands in the cars list. It then extracts the unique car brands by getting the keys from the Counter object and converts them to a Python list, which is printed out.
['Ford', 'Chevrolet', 'Dodge']
This way we can use the Counter class from the collections module to get all unique values in the list Python.
Python Unique Values in List Examples
Case 1: Get unique values from a list of dictionaries Python
- In this section, we will discuss how to get the unique values from a Python list of dictionaries by using Python.
- To perform this particular task we are going to use the list function and within this method, we are going to apply the list comprehension method as an argument.
Source Code:
new_dict = [{'Micheal' : 432, 'George' : 675}, {'William' : 432, 'John' : 178}, {'William' : 432, 'Smith':105}]
final_output = list(set(value for dic in new_dict for value in dic.values()))
print("Unique values:",final_output)Output: In the above code, we created a list of dictionaries and then initialized a variable ‘final_output‘ in which we have assigned the list values that can iterate over the dictionary values. Now convert to a list by using the dic.values() method.
Unique values: [432, 105, 178, 675]Here is the implementation of the following given code:
Case 2: Python gets unique values from a list of objects
- In this section, we will discuss how to get the unique values from a list of objects by using Python.
- By using the Python set() method we can easily perform this particular task. In Python the set method stores only unique values it will automatically remove duplicate values from the list.
Example:
new_values = [67,117,67,'b','z','b',25]
result = set(new_values)
z=list(result)
print("Unique values:",z)Output: We will create a list and assign integer and string values to it in Python. After that declare a variable ‘result’ in which we are going to apply the set() method and within this function, we will pass list ‘new_values’ as an argument.
Unique values: ['b', 67, 117, 25, 'z']You can refer to the below Screenshot:
Case 3: Get unique values from two lists of Python
- Here we can see how to get unique values from two lists by using Python.
- To solve this problem we are going to apply the set() function. In Python, the set() function is used to get the unique values from given lists.
Example:
lis1 = [23,41,23,78,94,41]
lis2 = [78,109,78,56,234,109]
z=list(set(lis1) - set(lis2))
m=list(set(lis2) - set(lis1))
print("Unique values from two lists:",z,m)Output: Here, we will create two lists ‘lis1’ and ‘lis2’ in Python. After that, we are going to declare a variable ‘z’ and ‘m’ in which we have assigned a Python list and set a function in Python for getting the unique values.
Unique values from two lists: [41, 94, 23] [56, 234, 109]Here is the Screenshot of the following code:
Case 4: Get unique value from list Python using index
To get the unique value from a Python list with an index number, we can easily use the concept of a Python list.index() method.
Example:
new_list = [2,6,9,2,9,4]
final_output = [new_list.index(i) for i in set(new_list)]
print("Unique values get index from list:",final_output)Output: Here, we will create a list in Python and assign integer values to it. After that, we are going to use the list comprehension method and it will iterate the list values.
Unique values get index from list: [2, 0, 5, 1]
Get all Unique Values in List Python Methods Pros and Cons
| Name | Pros | Cons |
|---|---|---|
| The set() Data Type | 1. Slower for larger lists because of the repeated checks. | 1. Original order might not be preserved. 2. Only works for hashable elements. |
| List Comprehension | 1. Preserve the original order. 2. Works for both hashable and non-hashable items. | 1. Slower for larger lists because of the repeated checks. |
| The dict.fromkeys() | 1. Preserve the original order. 2. Faster than list comprehension for larger lists. | 1. Slightly less intuitive than the set method. |
| The Custom Function | 1. Customizable. 2. Preserve the original order. | 1. Requires more code than other methods. |
| The Generator | 1. Memory efficient for large lists. 2. Lazy evaluation: processes items only as they’re needed. | 1. Might be less intuitive for beginners. |
| Filter Function | 1. Functional approach. 2. Preserve the original order. | 1. A bit tricky due to the use of side effects in lambda. |
| The collections.Counter | 1. Preserve the original order. 2. Can also get a count of each unique item. | 1. A bit overkill if you only need unique values. |
Conclusion
To get unique values from a list in Python is a frequently encountered task in data processing and programming.
From leveraging built-in Python constructs like the set, a custom function, filter() with lambda function, generators, the dict.fromkeys(), and list comprehensions to utilizing robust libraries like collections, there’s a wide range of tools to make us get unique values from a list in Python.
We have seen illustrative examples for each of the mentioned methods and some additional use cases to better understand the concept. Additionally, we have also seen the pros and cons of each method present to get unique values from a list in Python.
Related Python tutorials:
- Add string to list Python
- Python concatenate list with examples
- Check if a list exists in another list Python
- How to Prepend to a List in Python
- numpy.unique
- Python list comprehension using if-else
I am Bijay Kumar, a Microsoft MVP in SharePoint. Apart from SharePoint, I started working on Python, Machine learning, and artificial intelligence for the last 5 years. During this time I got expertise in various Python libraries also like Tkinter, Pandas, NumPy, Turtle, Django, Matplotlib, Tensorflow, Scipy, Scikit-Learn, etc… for various clients in the United States, Canada, the United Kingdom, Australia, New Zealand, etc. Check out my profile.