Python Code Optimization Techniques for Faster Execution

Python is a scripting language that often lags behind compiled languages like C/C++ in raw speed, but many optimizations can narrow the gap; this article gathers practical techniques to accelerate Python code.

0. Code Optimization Principles

Do not premature optimize; ensure code works correctly before tuning. Consider the cost of optimization, balancing time versus space and development effort. Focus on hot spots rather than optimizing every line.

1. Avoid Global Variables

# Not recommended – 26.8 s
import math
size = 10000
for x in range(size):
    for y in range(size):
        z = math.sqrt(x) + math.sqrt(y)

Placing code in a function reduces lookup overhead, typically yielding a 15‑30% speed gain.

# Recommended – 20.6 s
import math

def main():
    size = 10000
    for x in range(size):
        for y in range(size):
            z = math.sqrt(x) + math.sqrt(y)

main()

2. Avoid Attribute Access

2.1 Avoid Module and Function Attribute Access

# Not recommended – 14.5 s
import math

def computeSqrt(size: int):
    result = []
    for i in range(size):
        result.append(math.sqrt(i))
    return result

Using from math import sqrt brings the function into the local namespace, eliminating the attribute lookup cost.

# First optimization – 10.9 s
from math import sqrt

def computeSqrt(size: int):
    result = []
    for i in range(size):
        result.append(sqrt(i))
    return result

# Second optimization – 9.9 s
import math

def computeSqrt(size: int):
    result = []
    sqrt = math.sqrt
    for i in range(size):
        result.append(sqrt(i))
    return result

2.2 Avoid Class Attribute Access

# Not recommended – 10.4 s
class DemoClass:
    def __init__(self, value: int):
        self._value = value
    def computeSqrt(self, size: int):
        result = []
        for _ in range(size):
            result.append(math.sqrt(self._value))
        return result

Cache frequently accessed attributes in local variables.

# Recommended – 8.0 s
class DemoClass:
    def __init__(self, value: int):
        self._value = value
    def computeSqrt(self, size: int):
        result = []
        sqrt = math.sqrt
        value = self._value
        for _ in range(size):
            result.append(sqrt(value))
        return result

3. Avoid Unnecessary Abstraction

# Not recommended – 0.55 s
class DemoClass:
    @property
    def value(self) -> int:
        return self._value
    @value.setter
    def value(self, x: int):
        self._value = x

Direct attribute access is faster than property getters/setters.

# Recommended – 0.33 s
class DemoClass:
    def __init__(self, value: int):
        self.value = value

4. Avoid Unnecessary Data Copying

4.1 Eliminate Meaningless Copies

# Not recommended – 6.5 s
value = range(size)
value_list = [x for x in value]
square_list = [x * x for x in value_list]

# Recommended – 4.8 s
value = range(size)
square_list = [x * x for x in value]

4.2 Swap Values Without Temporary Variable

# Not recommended – 0.07 s
a = 3
b = 5
temp = a
a = b
b = temp

# Recommended – 0.06 s
a, b = b, a

4.3 Use join for String Concatenation

# Not recommended – 2.6 s
result = ''
for s in string_list:
    result += s

# Recommended – 0.3 s
result = ''.join(string_list)

5. Leverage Short‑Circuit Evaluation

# Not recommended – 0.05 s
if str_i in abbreviations:
    result += str_i

# Recommended – 0.03 s
if str_i[-1] == '.' and str_i in abbreviations:
    result += str_i

6. Loop Optimizations

6.1 Replace while with for

# Not recommended – 6.7 s
i = 0
while i < size:
    sum_ += i
    i += 1

# Recommended – 4.3 s
for i in range(size):
    sum_ += i

6.2 Use Implicit for Loops

# Recommended – 1.7 s
def computeSum(size: int) -> int:
    return sum(range(size))

6.3 Reduce Inner‑Loop Computations

# Not recommended – 12.8 s
for x in range(size):
    for y in range(size):
        z = sqrt(x) + sqrt(y)

# Recommended – 7.0 s
for x in range(size):
    sqrt_x = sqrt(x)
    for y in range(size):
        z = sqrt_x + sqrt(y)

7. Use numba.jit

# Recommended – 0.62 s
import numba

@numba.jit
def computeSum(size: int) -> int:
    sum = 0
    for i in range(size):
        sum += i
    return sum

8. Choose Appropriate Data Structures

Built‑in structures (list, dict, set, tuple) are implemented in C and are highly efficient; custom structures rarely match their speed. For frequent insert/delete operations, collections.deque offers O(1) performance. Use bisect for ordered list searches and heapq for fast min/max retrieval.

Source: https://zhuanlan.zhihu.com/p/439210443