Speeding Up Python List and Dictionary Operations by Avoiding Re‑evaluation
This article demonstrates Python performance optimization techniques by showing how moving list appends and dictionary lookups outside loops reduces re‑evaluation overhead, resulting in measurable speed improvements of 200‑300 ms, and provides a timing decorator to benchmark the changes.
In this tutorial we explore simple Python optimization techniques that avoid repeated evaluation of list methods and dictionary lookups inside loops, thereby speeding up code execution.
First, a timeit decorator is defined to measure the runtime of any function:
<code>import functools
import time
def timeit(func):
@functools.wraps(func)
def newfunc(*args, **kwargs):
startTime = time.time()
func(*args, **kwargs)
elapsedTime = time.time() - startTime
print('function - {}, took {} ms to complete'.format(func.__name__, int(elapsedTime * 1000)))
return newfunc
</code>Avoid re‑evaluating list appends inside the loop
Inside‑loop version (appending directly):
<code>@timeit
def append_inside_loop(limit):
nums = []
for num in limit:
nums.append(num)
append_inside_loop(list(range(1, 9999999)))
</code>Output: function - append_inside_loop, took 529 ms to complete
Outside‑loop version (binding append once):
<code>@timeit
def append_outside_loop(limit):
nums = []
append = nums.append
for num in limit:
append(num)
append_outside_loop(list(range(1, 9999999)))
</code>Output: function - append_outside_loop, took 328 ms to complete
The outside‑loop approach saves roughly 201 ms.
Avoid re‑evaluating dictionary lookups inside the loop
Inside‑loop version (using dict.get each iteration):
<code>@timeit
def inside_evaluation(limit):
data = {}
for num in limit:
data[num] = data.get(num, 0) + 1
inside_evaluation(list(range(1, 9999999)))
</code>Output: function - inside_evaluation, took 1400 ms to complete
Outside‑loop version (binding get once):
<code>@timeit
def outside_evaluation(limit):
data = {}
get = data.get
for num in limit:
data[num] = get(num, 0) + 1
outside_evaluation(list(range(1, 9999999)))
</code>Output: function - outside_evaluation, took 1189 ms to complete
Binding the get method outside the loop reduces runtime by about 211 ms.
These simple changes—moving method lookups out of hot loops—demonstrate how minor refactoring can yield noticeable performance gains in Python programs.
Python Programming Learning Circle
A global community of Chinese Python developers offering technical articles, columns, original video tutorials, and problem sets. Topics include web full‑stack development, web scraping, data analysis, natural language processing, image processing, machine learning, automated testing, DevOps automation, and big data.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.