10 Game-Changing Python Performance Hacks You Should Know

Python is praised for its simplicity and readability, but improper use can lead to slow execution. Over the years I have discovered ten game‑changing performance optimization tricks that make Python code faster and more efficient.

1. Use list comprehensions instead of loops

Loops are useful but can be slow on large datasets. List comprehensions are faster and more Pythonic.

<code>numbers = []
for i in range(1000000):
    numbers.append(i * 2)
</code>

Recommended version using a list comprehension:

<code>numbers = [i * 2 for i in range(1000000)]
</code>

Even better with a generator expression (saves memory):

<code>numbers = (i * 2 for i in range(1000000))
</code>

2. Use built‑in functions and libraries

Python’s built‑in functions are implemented in C and therefore much faster than hand‑written equivalents.

<code>def custom_sum(numbers):
    total = 0
    for num in numbers:
        total += num
    return total

custom_sum([1, 2, 3, 4, 5])
</code>

Recommended version using sum() :

<code>sum([1, 2, 3, 4, 5])
</code>

Using NumPy for numeric operations:

<code>import numpy as np
arr = np.array([1, 2, 3, 4, 5])
arr.sum()
</code>

Built‑in functions like sum() , max() and min() are C‑optimized and far faster than manual implementations.

3. Use join() instead of string concatenation

Concatenating strings with + inside a loop creates a new string each time and is very slow.

<code>words = ["Python", "is", "fast"]
sentence = ""
for word in words:
    sentence += word + " "
</code>

Recommended version using join() :

<code>sentence = " ".join(words)
</code>

join() is implemented in C with O(n) complexity, while repeated + concatenation has O(n²) complexity.

4. Use generators for large data

Generators produce values lazily, saving memory compared to storing large lists.

<code>def get_numbers():
    return [i for i in range(1000000)]
</code>

Recommended version using a generator:

<code>def get_numbers():
    for i in range(1000000):
        yield i
</code>

Generators do not store data in memory, making them ideal for processing massive datasets.

5. Use set() for fast membership tests

Checking membership in a list takes O(n) time, while a set provides average O(1) lookup.

<code>items = [1, 2, 3, 4, 5]
if 3 in items:
    print("Found")
</code>

Recommended version using a set:

<code>items = {1, 2, 3, 4, 5}
if 3 in items:
    print("Found")
</code>

Sets are implemented with hash tables, giving much faster lookups than lists.

6. Use multiprocessing for CPU‑bound tasks

The Global Interpreter Lock (GIL) prevents multiple threads from executing Python bytecode simultaneously. For CPU‑intensive work, multiprocessing is preferred over multithreading.

<code>from threading import Thread

def compute():
    print(sum(i * i for i in range(10**7)))

threads = [Thread(target=compute) for _ in range(4)]
for thread in threads:
    thread.start()
for thread in threads:
    thread.join()
</code>

Recommended version using multiprocessing:

<code>from multiprocessing import Pool

def compute(_):
    return sum(i * i for i in range(10**7))

with Pool(4) as p:
    p.map(compute, range(4))
</code>

Each process has its own Python interpreter, bypassing the GIL.

7. Use lru_cache() to cache expensive function calls

<code>def slow_function(n):
    print(f"Computing {n}...")
    return n * n

print(slow_function(10))
print(slow_function(10))
</code>

Recommended version with caching:

<code>from functools import lru_cache

@lru_cache(maxsize=1000)
def slow_function(n):
    print(f"Computing {n}...")
    return n * n

print(slow_function(10))
print(slow_function(10))
</code>

lru_cache() stores results, avoiding repeated calculations.

8. Use itertools for efficient iteration

The itertools module provides high‑performance iterator functions.

<code>from itertools import combinations
items = ["a", "b", "c"]
for combo in combinations(items, 2):
    print(combo)
</code>

Operations in itertools are implemented in optimized C code, making them faster than manual loops.

9. Accelerate code with Cython or Numba

For performance‑critical tasks, compile Python code to C using Cython or Numba.

<code>from numba import jit

@jit(nopython=True)
def fast_function(n):
    return sum(i * i for i in range(n))

print(fast_function(10**7))
</code>

Numba compiles Python to machine code, yielding speedups of 10‑100×.

10. Profile code with cProfile

Use cProfile to locate bottlenecks and guide optimizations.

<code>python -m cProfile -s time my_script.py
</code>

Profiling identifies slow functions, enabling targeted improvements.

Final Thoughts

These ten optimization techniques can dramatically improve the speed and efficiency of Python code, whether you are handling large datasets, CPU‑intensive tasks, or API calls.