Best Practices for Large File Upload with Chunking, Hashing, and Concurrent Requests

In the era of the internet, uploading large files has become a common requirement; this article shares practical experience for efficiently handling large file uploads from a front‑end page to an OSS backend.

The main goals are to quickly upload a 1.5 GB file, support breakpoint‑resume when the network is poor, and enable instant (秒传) uploads when the same file already exists on the server.

First, the file’s content hash is calculated using spark-md5 to uniquely identify the file. The hash is sent to the server to query the file’s storage status, which can be “not uploaded”, “partially uploaded”, or “already uploaded”. Based on the response, different upload strategies are chosen.

Hash calculation and file slicing are implemented as follows: import SparkMD5 from 'spark-md5' const CHUNK_SIZE = 1024 * 1024 * 5 // 5M function sliceFile2Chunk(file) { /* ... */ } function getFileHash(file) { /* ... */ }

After receiving the server status, a list of chunks that still need to be uploaded is built: function createWait2UploadChunks(res) { /* ... */ } function formateChunk(item, index) { /* ... */ }

Chunks are uploaded concurrently with a maximum of five HTTP requests. Progress is tracked and, once all chunks are uploaded, a merge request is sent to the server to assemble the file: async function uploadFileChunk(chunkFormData) { /* ... */ } async function mergeChunks() { /* ... */ }

Optimizations include using a Web Worker to compute the hash without blocking the UI, adjusting the chunk size (e.g., 5 MB) according to network bandwidth and server resources, and exploring multi‑client simultaneous uploads to further reduce total upload time.

The article concludes that by splitting a large file into smaller chunks, uploading them concurrently, and leveraging hash‑based status checks, developers can achieve fast, reliable, and resumable large file uploads.