Diagnosing /tmp Disk Space Exhaustion in a Flask File Transfer Service: Understanding Werkzeug Form Parsing and Temporary File Handling

We operate an independently deployed file‑transfer service built with Flask that receives files from clients and forwards them to RabbitMQ. One day a disk‑space alarm appeared, showing a bizarre wave‑like growth pattern rather than a gradual increase.

Initial inspection suggested a simple cleanup, but the rapid, large spikes hinted at a deeper issue. The error log revealed a massive traceback originating from werkzeug/formparser.py , indicating that the request handling code was exhausting disk space.

Although the service itself does not write to /tmp , Werkzeug’s WSGI request parsing does. The relevant functions are FormDataParser._parse_multipart , MultiPartParser.parse , parse_parts , and parse_lines . These functions separate the incoming HTTP body into form and files structures.

The core of the temporary‑file mechanism is the default_stream_factory defined in werkzeug/formparser.py :

# werkzeug/formparser.py
from tempfile import TemporaryFile

def default_stream_factory(total_content_length, filename, content_type,
                           content_length=None):
    """The stream factory that is used per default."""
    if total_content_length > 1024 * 500:  # 500 KB threshold
        return TemporaryFile('wb+')
    return BytesIO()

If the uploaded payload exceeds 500 KB, Werkzeug creates a TemporaryFile (usually under /tmp ); otherwise it keeps the data in memory. In our production environment, only about 2.75 % of uploads were smaller than 500 KB, meaning almost every file upload triggered the temporary‑file path.

Monitoring the /tmp directory with inotifywait confirmed that large file uploads generated a flurry of create/modify/delete events, while small uploads produced none. Two test cases were performed:

Case 1 : Upload a file > 500 KB using the files (multipart) field – temporary files were created.

Case 2 : Upload the same large file using the data (form) field – the data stayed in memory and no temporary files appeared.

These experiments proved that the upload method determines whether the 500 KB threshold leads to disk usage.

Conclusions :

File‑type uploads larger than 500 KB invoke /tmp via TemporaryFile ; the directory can be redirected by setting TMPDIR , TEMP or TMP .

Form‑type uploads always reside in memory, regardless of size, which may cause memory pressure if payloads are huge.

Read/write performance differs significantly between the two methods, suggesting a trade‑off between speed and resource consumption.

To avoid similar incidents, we limited the maximum upload size, which immediately restored normal operation, and we recommend allocating a larger /tmp space on production servers.