Optimizing Locust HTTP Requests with PycURL: Experiments and Findings

Locust is one of the most popular load‑testing frameworks for protocol interfaces, and this article continues the discussion by focusing on optimizing the efficiency of its HTTP requests.

Although Locust offers strong platform extensibility and low development cost, its performance lags behind tools like Tsung, prompting an exploration of ways to enhance HTTP request efficiency.

Locust sends HTTP requests using the requests library, which internally relies on urllib3 . Because pycurl demonstrates higher performance than urllib3 , the author attempted to replace urllib3 with pycurl , referencing a performance comparison from a StackOverflow discussion.

Locust provides a protocol‑extension mechanism, so a custom PycURL‑based client was implemented to replace the native HttpLocust . Since the pycurl.Curl class cannot be subclassed, the method for obtaining the function was modified accordingly.

Performance tests of the new PycurlLocust showed roughly a 20% improvement for a single user compared to the original HttpLocust . However, when increasing the number of concurrent users, the requests‑per‑second (RPS) increase was minimal, leading to further investigation.

The investigation turned to Gevent, which Locust uses internally. In core.py , monkey.patch_all(thread=False) replaces blocking I/O functions (e.g., socket, time.sleep) with their non‑blocking Gevent counterparts. Because pycurl is a compiled C extension, Gevent's monkey patching does not affect it, resulting in blocking behavior.

An example code snippet (shown in the accompanying image) demonstrates the performance difference when monkey.patch_all is enabled versus disabled for asynchronous requests.

Conclusion: Replacing urllib3 with pycurl does not provide a practical performance boost for Locust, but the optimization process reveals how Locust leverages Gevent to accelerate standard requests and underscores the importance of understanding the framework's internal mechanisms.