Unify Object Storage via S3 Protocol for Multi-Cloud and MinIO

Background

In today's Internet era, almost all cloud vendors provide "Object Storage Service", a massive, secure, low‑cost, highly reliable cloud storage suitable for any file type. Capacity and processing power scale elastically, with multiple storage classes to optimize cost.

When using a vendor's product, you normally import its SDK and follow the documentation, but when dealing with many vendors—or when you need to guarantee interface portability—you must handle "breaking changes" in the target vendor's API.

Below are SDK upload examples for several vendors:

Alibaba Cloud

<code>// Endpoint example for Hangzhou; adjust for other regions.
String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
String accessKeyId = "<yourAccessKeyId>";
String accessKeySecret = "<yourAccessKeySecret>";

// Create OSSClient instance.
OSS ossClient = new OSSClientBuilder().build(endpoint, accessKeyId, accessKeySecret);

// Create PutObjectRequest.
String content = "Hello OSS";
PutObjectRequest putObjectRequest = new PutObjectRequest("<yourBucketName>", "<yourObjectName>", new ByteArrayInputStream(content.getBytes()));

// Upload string.
ossClient.putObject(putObjectRequest);

// Shut down client.
ossClient.shutdown();
</code>

Huawei Cloud

<code>String endPoint = "https://your-endpoint";
String ak = "*** Provide your Access Key ***";
String sk = "*** Provide your Secret Key ***";
// Create ObsClient instance
ObsClient obsClient = new ObsClient(ak, sk, endPoint);

obsClient.putObject("bucketname", "objectname", new File("localfile")); // localfile is the path to the file to upload
</code>

Qiniu Cloud

<code>Configuration cfg = new Configuration(Region.region0());
UploadManager uploadManager = new UploadManager(cfg);
String accessKey = "your access key";
String secretKey = "your secret key";
String localFilePath = "/home/qiniu/test.png";
String key = null;

Auth auth = Auth.create(accessKey, secretKey);
String upToken = auth.uploadToken(bucket);
Response response = uploadManager.put(localFilePath, key, upToken);
</code>

Solution

Amazon S3 Protocol

Amazon was the first to offer object storage and defined industry standards; implementing the S3 protocol allows you to connect to any compatible storage vendor or middleware, with additional requirements for availability and other criteria.

Domestic Cloud Vendors Supporting S3 Protocol

Alibaba Cloud – https://www.aliyun.com

Huawei Cloud – https://www.huaweicloud.com

Tencent Cloud – https://cloud.tencent.com

Qiniu Cloud – https://www.qiniu.com

Kingsoft Cloud – https://www.ksyun.com

How to Use

Import the dependency; no need to import each vendor's SDK.

<code><dependency>
    <groupId>com.pig4cloud.plugin</groupId>
    <artifactId>oss-spring-boot-starter</artifactId>
    <version>0.0.1</version>
</dependency>
</code>

Configure storage properties.

<code>oss:
  path-style-access: false    # request path format: XXX/{bucketName}
  endpoint: s3-cn-east-1.qiniucs.com
  access-key: xxx             # key provided by the cloud vendor
  secret-key: xxx             # secret provided by the cloud vendor
  bucketName: pig4cloud       # bucket created earlier
</code>

Operate via OssTemplate.

<code>@Autowire
private final OssTemplate ossTemplate;

ossTemplate.putObject(CommonConstants.BUCKET_NAME, fileName, file.getInputStream());
</code>

Support for MinIO and Other Self‑Hosted Storage

Create MinIO container.

<code>docker run -p 9000:9000 --name minio1 \
  -e "MINIO_ACCESS_KEY=lengleng" \
  -e "MINIO_SECRET_KEY=lengleng" \
  minio/minio server /data
</code>

Configure MinIO parameters.

<code># File system
oss:
  path-style-access: true
  endpoint: http://IP:9000
  access-key: lengleng
  secret-key: lengleng
  bucketName: lengleng
</code>

Use OssTemplate to upload.

The main difference from cloud vendors is the path‑style‑access parameter: Alibaba Cloud requires requests in the form bucketname.region.aliyuncs.com , while self‑hosted storage uses http://domain/bucketname . Adjusting this parameter enables seamless switching.