How to Get Unique Values from a List in Python

Last Modified Date: by

To get unique values from a list in Python, use set(my_list) when order doesn’t matter, or list(dict.fromkeys(my_list)) to keep the order in which values first appear. If “unique” means values that occur only once, count them with collections.Counter:

list(dict.fromkeys(["TX", "CA", "TX", "NY"]))   # ['TX', 'CA', 'NY']

Set ordering can differ between runs, which is one reason I prefer dict.fromkeys; all output below is from Python 3.12.5.

You’ll see both distinct-value methods, counting unique values, values that appear exactly once, ignoring case, unique values of a key in a list of dicts, and lists of lists.

Get unique values from a list with set() or dict.fromkeys()

Both drop repeats. A set has no order, so the printout can look shuffled; a dict keeps insertion order, so dict.fromkeys preserves the first-seen order:

states = ["TX", "CA", "TX", "NY", "CA", "FL", "TX"]

print(set(states))                       # unique, but in no particular order
print(list(dict.fromkeys(states)))       # unique, first-seen order kept
print(sorted(set(states)))               # unique and alphabetical
print(len(set(states)), "unique of", len(states))

Output:

{'NY', 'TX', 'FL', 'CA'}
['TX', 'CA', 'NY', 'FL']
['CA', 'FL', 'NY', 'TX']
4 unique of 7
Command Prompt output of Python getting unique state codes from a list with set, dict.fromkeys and sorted set
The set shows four states in its own order, dict.fromkeys keeps TX, CA, NY, FL, the sorted version is alphabetical, and the count is 4 of 7.

len(set(lst)) is the quick count of distinct values. If your goal is to clean the list itself, see removing duplicates from a list in Python.

How do I find values that appear only once in a list?

Sometimes “unique” means the values with no repeats at all. Counter counts every value in one pass, so you can filter by count:

from collections import Counter

orders = ["Mug", "Beans", "Mug", "Filters", "Kettle", "Beans", "Mug"]
counts = Counter(orders)

print(counts)
print("distinct items:   ", list(counts))
print("ordered only once:", [item for item, n in counts.items() if n == 1])
print("repeat items:     ", [item for item, n in counts.items() if n > 1])
print("most common:      ", counts.most_common(1))

Output:

Counter({'Mug': 3, 'Beans': 2, 'Filters': 1, 'Kettle': 1})
distinct items:    ['Mug', 'Beans', 'Filters', 'Kettle']
ordered only once: ['Filters', 'Kettle']
repeat items:      ['Mug', 'Beans']
most common:       [('Mug', 3)]
Command Prompt output of Python Counter finding list items ordered only once, repeated items and the most common item
Filters and Kettle were ordered once, Mug and Beans repeat, and Mug is the most common with 3 orders.

The same counts answer “which values repeat”, as covered in finding duplicates in a Python list.

Get unique values ignoring case and spaces

“Austin” and “austin” are different strings, so a set keeps both. Normalize to build the key, but keep the original text for display:

cities = ["Austin", "austin", "Denver ", "DENVER", "Boise"]

seen = {}
for c in cities:
    seen.setdefault(c.strip().lower(), c.strip())   # keep the first spelling
print(list(seen.values()))

Output:

['Austin', 'Denver', 'Boise']

setdefault only stores the first spelling it sees for each key. For more on the counting side, see counting the frequency of elements in a list.

Get unique values of a key in a list of dictionaries

For records, collect one field in a set comprehension. To keep one whole record per value, key a dict by that field:

customers = [
    {"name": "Emily Carter", "state": "TX"},
    {"name": "James Wilson", "state": "CO"},
    {"name": "Olivia Brown", "state": "TX"},
    {"name": "Liam Davis", "state": "FL"},
]

print(sorted({c["state"] for c in customers}))          # unique values of one key

unique_by_state = list({c["state"]: c for c in customers}.values())
print([c["name"] for c in unique_by_state], "<- last customer per state wins")

Output:

['CO', 'FL', 'TX']
['Olivia Brown', 'James Wilson', 'Liam Davis'] <- last customer per state wins
Command Prompt output of Python getting unique state values from a list of customer dictionaries and one customer per state
The customers come from three states, and keeping one record per state leaves Olivia for TX because the last customer wins.

Reverse the list first if you want the first record to win. Turning the dicts into other shapes is covered in converting a dictionary to a list.

Fix TypeError: unhashable type ‘list’

Sets and dict keys need hashable values, and lists aren’t hashable. Convert inner lists to tuples, deduplicate, then convert back if needed:

routes = [["Austin", "Dallas"], ["Denver", "Boise"], ["Austin", "Dallas"]]

try:
    set(routes)
except TypeError as err:
    print("TypeError:", err)

unique_routes = [list(r) for r in dict.fromkeys(tuple(r) for r in routes)]
print(unique_routes)

Output:

TypeError: unhashable type: 'list'
[['Austin', 'Dallas'], ['Denver', 'Boise']]
Command Prompt output of Python TypeError unhashable type list when calling set on a list of lists, fixed with tuples
set() on the routes fails with unhashable type: ‘list’, and the tuple round-trip keeps the two distinct routes.

The same fix works for dicts inside lists: use tuple(sorted(d.items())) as the key. Converting lists to sets is explained further in converting a list to a set in Python.

Unique value methods compared

GoalCodeOrder kept?
Distinct valuesset(lst)No
Distinct, first-seen orderlist(dict.fromkeys(lst))Yes
Number of distinct valueslen(set(lst))–
Values that appear once[x for x, n in Counter(lst).items() if n == 1]Yes
Unique values of a key{d[k] for d in records}No
Lists inside the listdict.fromkeys(tuple(x) for x in lst)Yes

Keep working with lists using these guides:

Frequently asked questions

How do I get unique values from a list in Python?

Use set(lst), or list(dict.fromkeys(lst)) to keep the original order.

Why is the order different after using set()?

Sets are unordered. Use dict.fromkeys when order matters, or sorted(set(lst)) for alphabetical order.

How do I count unique values in a list?

len(set(lst)) gives the number of distinct values.

How do I get values that appear only once?

Count with Counter(lst) and keep the items whose count is 1.

Why do I get unhashable type: ‘list’?

The list contains lists. Convert them to tuples before putting them in a set.

Where is Counter documented?

See collections.Counter in the Python docs.