Improving Third‑Party Service Performance in Kubernetes with Nginx Caching

The article discusses a method to improve the performance and stability of third‑party service access in a Kubernetes environment by configuring Nginx as a caching gateway. The approach requires no code changes, but it lacks some customisation, does not support write‑through caching, and the cache is not highly available across pods.

Modern tech companies increasingly rely on external services as part of their application stack. In the Back Market scenario, a product catalog is served by a third‑party API, and the team needed a fast, reliable way to access this data from their own Kubernetes cluster. They therefore placed Nginx in front of the API to cache all internal requests.

After a few days of operation, only about 1 % of read requests needed to wait for the third‑party response. The cache hit‑rate was high, and even when the third‑party service was down for 12 hours, 96 % of requests were still served from cache, dramatically reducing latency compared with direct calls from the US to a European API.

Nginx cache configuration

proxy_cache_path ... max_size=1g inactive=1w;
proxy_ignore_headers Cache-Control Expires Set-Cookie;
proxy_cache_valid 1m;
proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
proxy_cache_background_update on;

The goal of the configuration is to minimise read (GET) requests to the third‑party service. When a cache miss occurs, the request must wait for the upstream response. The cache is limited by size (1 GB) and inactivity (1 week). With proxy_cache_background_update on , cached responses older than one minute are still returned while Nginx refreshes them in the background.

Additional directives are used to avoid parallel background refreshes ( proxy_cache_lock on ) and to rewrite absolute URLs returned by the third‑party API so they point to the gateway:

sub_filter 'https://$proxy_host' '$scheme://$http_host';
sub_filter_last_modified on;
sub_filter_once off;
sub_filter_types application/json;

Because sub_filter is incompatible with gzip, the gateway disables gzip on those responses:

# Required because sub_filter is incompatible with gzip responses:
proxy_set_header Accept-Encoding "";

To ensure that client requests never wait for a background cache update, the Nginx keepalive_timeout is set to 0, forcing a new connection for each request.

Kubernetes deployment

The Nginx configuration is packaged in a non‑privileged container image and deployed like any other web service. Since Nginx cache lives on the local filesystem, persisting it across pod restarts is a challenge.

To achieve shared persistent cache, each pod mounts an /mnt/cache emptyDir volume that is synchronised with an S3 bucket. An init container pulls the bucket contents before Nginx starts:

aws s3 sync s3://thirdparty-gateway-cache /mnt/cache/complete

A sidecar container continuously pushes local cache updates back to S3:

while true; do
  sleep 600
  aws s3 sync /mnt/cache/complete s3://thirdparty-gateway-cache
done

The cache path is configured to use a temporary directory to avoid partially written entries being uploaded:

proxy_cache_path /mnt/cache/complete ... use_temp_path=on;
proxy_temp_path /mnt/cache/tmp;

Bucket lifecycle rules can be added to delete old entries after a set number of days.

Limitations

The solution works well for read‑heavy workloads that can tolerate eventual consistency, but it does not support write‑through caching, low‑traffic back‑ends, or custom request handling. All clients share the same cache unless authentication information is incorporated into the cache key. Security considerations include keeping the S3 bucket private.

Because the cache is centrally stored in S3, it is not a high‑availability shared cache as recommended by Nginx; future work may explore a primary/backup cache architecture.