Observability and Data Collection Strategies in Cloud‑Native Environments

Observability is not a brand‑new concept, but observable systems have rapidly emerged in recent years as a necessary response to the complexity and scale of cloud‑native applications.

An observability platform aggregates data from all parts of a system—cloud infrastructure, containers, middleware, business frameworks, and services—into a unified platform, resulting in massive data volumes.

Collecting such massive data poses significant technical challenges, requiring both complete and timely reporting to monitor system health and respond to anomalies.

Figure 1: Data sources covered by an observability system.

Data collection can follow two main schemes. The first scheme pushes data directly from business services to the observability platform (Figure 2). The second scheme writes data locally and uses a collector component to forward it to the platform (Figure 3).

Figure 2: Direct push architecture.

Figure 3: File‑based collection architecture.

Direct push offers real‑time data and a simple architecture but consumes resources on the business container and may fail when the service is overloaded or crashes. File‑based collection preserves data integrity during service failures but adds a collector component, increasing system complexity and maintenance effort.

To optimize data collection in high‑volume scenarios, both instrumentation (the “point of collection”) and reporting should be refined, for example by adjusting sampling rates.

Using Spring Cloud Sleuth to collect trace data and report it to a Zipkin server, a benchmark shows that a sampling rate of 0.1 has negligible performance impact, whereas sampling at 1.0 (full collection) incurs about a 16% performance loss under heavy load.

Sampling techniques are essential to reduce overhead while preserving valuable data. Three main types are discussed:

1. Head‑based sampling decides at the start of a request whether the entire trace will be sampled. It is simple and reduces data volume, but may discard useful data from later services if the initial decision is negative.

2. Tail‑based sampling evaluates a trace after completion, allowing selective sampling based on latency, error status, or other criteria, thus retaining valuable long‑running or error traces, though it requires buffering and adds load to the backend.

3. Local (unit) sampling lets each service independently decide whether to report its own span, resulting in non‑coherent traces but offering low overhead and the ability to focus on spans of interest.

Considering these options, the article recommends the file‑based approach with a dedicated collector (e.g., Filebeat) deployed in a separate container. This separation isolates collection overhead from business services, ensures at‑least‑once delivery, and allows independent scaling and monitoring of the collector.

In cloud‑native environments where services run as containers, mounting a shared directory for logs enables the collector to continue operating even if a business container crashes, while the collector’s performance impact remains isolated.

Finally, the article promotes the book “Best Practices for Observability Systems in the Cloud‑Native Era” and includes a limited‑time discount offer.