How To Split A Python List Into Evenly Sized Chunks?

As a data scientist working with large datasets, I often encounter situations where I need to divide a list into smaller, more manageable pieces. This is especially useful when processing data in batches or performing parallel computing. In this tutorial, I will explain how to split a Python list into evenly sized chunks with suitable examples.

Table of Contents

Split a Python List Into Evenly Sized Chunks

Let us get into the various methods and examples to split a Python list into chunks of equal size.

Read How to Add Tuples to Lists in Python?

Method 1: Use a Loop with List Slicing

One simple approach to split a list into evenly sized chunks is to use a loop along with Python list slicing. Here’s an example:

def split_list(lst, chunk_size):
    chunks = []
    for i in range(0, len(lst), chunk_size):
        chunks.append(lst[i:i + chunk_size])
    return chunks

# Example usage
names = ["John", "Emily", "Michael", "Emma", "William", "Olivia", "James", "Sophia", "Benjamin", "Isabella"]
chunk_size = 3
result = split_list(names, chunk_size)
print(result)

Output:

[['John', 'Emily', 'Michael'], ['Emma', 'William', 'Olivia'], ['James', 'Sophia', 'Benjamin'], ['Isabella']]

I executed the above example code and added the screenshot below.

Split a Python List Into Evenly Sized Chunks

In this example, we define a function called split_list that takes a list (lst) and the desired chunk size (chunk_size) as parameters. We initialize an empty list called chunks to store the resulting chunks.

Inside the function, we use a for loop with the range() function to iterate over the indices of the list with a step size equal to chunk_size. In each iteration, we use list slicing to extract a chunk of elements from the list and append it to the chunks list. Finally, we return the chunks list containing the evenly sized chunks.

Check out How to Convert Dictionary to List of Tuples in Python?

Method 2: Use List Comprehension

We can also use Python list comprehension to split a list into chunks in a more concise way. Here’s an example:

def split_list(lst, chunk_size):
    return [lst[i:i + chunk_size] for i in range(0, len(lst), chunk_size)]

# Example usage
cities = ["New York", "Los Angeles", "Chicago", "Houston", "Phoenix", "Philadelphia", "San Antonio", "San Diego", "Dallas", "San Jose"]
chunk_size = 4
result = split_list(cities, chunk_size)
print(result)

Output:

[['New York', 'Los Angeles', 'Chicago', 'Houston'], ['Phoenix', 'Philadelphia', 'San Antonio', 'San Diego'], ['Dallas', 'San Jose']]

I executed the above example code and added the screenshot below.

How to Split a Python List Into Evenly Sized Chunks

In this approach, we use list comprehension to generate the chunks in a single line. The list comprehension iterates over the indices of the list using range() with a step size of chunk_size and creates chunks using list slicing.

Check out How to Write a List to a File in Python?

Method 3: Use the yield Keyword

Another way to split a list into chunks is by using the yield keyword to create a generator function. This method is memory-efficient as it generates chunks on the fly instead of storing them all in memory at once. Here’s an example:

def chunk_generator(lst, chunk_size):
    for i in range(0, len(lst), chunk_size):
        yield lst[i:i + chunk_size]

# Example usage
states = ["California", "Texas", "Florida", "New York", "Pennsylvania", "Illinois", "Ohio", "Georgia", "North Carolina", "Michigan"]
chunk_size = 5

for chunk in chunk_generator(states, chunk_size):
    print(chunk)

Output:

['California', 'Texas', 'Florida', 'New York', 'Pennsylvania']
['Illinois', 'Ohio', 'Georgia', 'North Carolina', 'Michigan']

I executed the above example code and added the screenshot below.

Split a Python List Into Evenly Sized Chunks yield keyword

In this example, we define a generator function called chunk_generator that takes a list (lst) and the desired chunk size (chunk_size) as parameters. Inside the function, we use a Python for loop with range() to iterate over the indices of the list with a step size equal to chunk_size. In each iteration, we use the yield keyword to generate a chunk of elements from the list.

When we iterate over the generator function using a for loop, it yields one chunk at a time, allowing us to process the chunks efficiently without loading the entire list into memory at once.

Read How to Iterate Through a List in Python?

Method 4: Use NumPy’s array_split()

If you’re working with NumPy arrays, you can use the array_split() function to split an array into evenly sized chunks. Here’s an example:

import numpy as np

# Example usage
ages = np.array([25, 32, 18, 45, 29, 36, 41, 22, 27, 33])
chunk_size = 3
result = np.array_split(ages, chunk_size)
print(result)

Output:

[array([25, 32, 18]), array([45, 29, 36]), array([41, 22, 27]), array([33])]

In this example, we use NumPy’s array_split() function to split the ages array into chunk_size number of evenly sized chunks. The resulting chunks are returned as a list of NumPy arrays.

Read Convert String to List in Python Without Using Split

Handle Uneven Chunks

When the length of the list is not evenly divisible by the chunk size, the last chunk may contain fewer elements than the others. In such cases, you can choose to distribute the remaining elements among the chunks or keep them as a separate smaller chunk.

The methods discussed above handle uneven chunks differently:

Method 1 and Method 2 will create a smaller last chunk if the list length is not evenly divisible by the chunk size.
Method 3 (using the yield keyword) will also create a smaller last chunk.
Method 4 (using NumPy’s array_split()) will distribute the remaining elements among the chunks to make them as even as possible.

Check out How to Convert String to List in Python?

Conclusion

In this tutorial, I helped you to understand how to split a Python list into evenly sized chunks. I explained four important methods such as using a loop with list slicing, using list comprehension, using the yield keyword, and using NumPy's Chunks.