Design and Implementation of SLA for Object Storage Services

1. Introduction With the rapid development of cloud computing and storage, object storage has become a fundamental IaaS capability for storing massive unstructured data. Service providers typically guarantee availability through SLA (Service-Level Agreement) expressed as a number of nines. This article shares practical experience on measuring and evaluating object storage stability.

2. Object Storage and the S3 Protocol Object storage is a key‑value store accessed via REST APIs (GET, PUT, DELETE, etc.). Common implementations include Amazon S3, OpenStack Swift, Ceph RGW, Alibaba OSS, Huawei OBS, and Qiniu. The S3 API defines five main groups: general resources (buckets, hosts), authentication & ACL, service operations, bucket operations, and object operations.

Interface

Method

URI

Create Object

POST

{api_uri}/objects

Delete Object

DELETE

{api_uri}/{bucketName}/{objectName}

Download Object

GET

{api_uri}/{bucketName}/{objectName}

List Objects

GET

{api_uri}/{bucketName}

3. SLA Metric Design SLA consists of SLA (contract), SLO (objective), and SLI (indicator). The primary indicator for object storage is service availability, calculated as (total time – downtime) / total time × 100%. Downtime is counted when the service is unavailable for more than one minute, excluding short network glitches and certain planned interruptions.

Target availability varies by environment:

Environment

Availability Target

Notes

Production

>=99.99%

≤52.6 minutes annual downtime (3 replicas, carrier‑hosted data center)

Pre‑production / Staging

>=99.9%

≤8.76 hours annual downtime (2 replicas)

Integration Test

>=99%

≤3.65 days annual downtime (2 replicas, self‑hosted)

Additional SLA features cover data durability (up to 99.999999999% for public clouds), privacy (access‑key isolation, bucket/object ACLs), migration, functional limits (unlimited file count, 5 GB single‑file limit, 5 TB multipart), performance baselines, scalability (3‑day expansion), and operational support (24/7 phone & WeChat, unified monitoring, ELK/APM logs).

4. Availability Probing Implementation

Probing (dial‑testing) periodically performs real CRUD operations against the storage endpoint. Each probe uploads a 520 KB random file, then downloads and deletes it, requiring all three steps to succeed (2xx response). Failures trigger a single retry; timeouts after 5 s are considered failures. Probes run every minute from at least two nodes on different racks.

4.2.1 s3cmd based script

Install s3cmd via yum or pip, configure ~/.s3cfg , and use shell scripts to combine commands such as:

s3cmd mb s3://prod_test
s3cmd put 1.txt s3://prod_test
s3cmd get s3://prod_test/1.txt ./
s3cmd del s3://prod_test/1.txt

4.2.2 Python boto implementation

Use the lightweight boto SDK to programmatically create a client, upload, download, and delete objects. Example steps:

import boto, json, os
client = boto.connect_s3(host='endpoint', aws_access_key_id='...', aws_secret_access_key='...')
bucket = client.get_bucket('prod_test')
key = bucket.new_key('1.txt')
key.set_contents_from_filename('1.txt')
key.get_contents_to_filename('downloaded.txt')
key.delete()

4.2.3 Java SDK implementation

Add the S3 SDK dependency to pom.xml and use the client to put and get objects:

<dependency>
  <groupId>com.xx.xx</groupId>
  <artifactId>s3-sdk</artifactId>
  <version>2.1.0</version>
</dependency>

S3Client client = new S3Client("ACCESS_KEY", "SECRET_KEY", new S3Client.Endpoint("http://endpoint"));
client.putObject("bucket", "a2.dat", new File("/tmp/a1.dat"));
S3Object obj = client.getObject("bucket", "a2.dat");
obj.getObjectContent();
obj.close();

For SLA probing, Python is recommended for its lightweight nature and ease of integration with automation tools (SaltStack, Ansible). Deploy the script via cron (once per minute) or through monitoring agents.

5. Monitoring and Alerting Probe results (status and latency) are packaged as JSON and pushed to a unified monitoring platform (e.g., Zabbix, Prometheus + Grafana, Open‑Falcon). Alerts are configured for:

status = 0 for 1 minute → Critical (service unavailable)

status = -1 for 1 minute → Critical (probe node unreachable)

latency > 1000 ms for 3 minutes → Warning (performance degradation)

6. Value and Benefits Implementing SLA monitoring for object storage clarifies reliability targets for infrastructure teams, reduces incident frequency and duration, improves tenant satisfaction, provides reusable probing capabilities for other services, and enhances overall observability beyond basic host‑level metrics.

References: Chinese Open‑Source Cloud Alliance white‑paper, Ceph documentation, Amazon S3 documentation, Open‑Falcon project.