FolderBsdp: An Efficient Incremental Update Solution for React Native Hybrid Apps

Background

Shopee’s mobile applications are hybrid apps that combine native code with React Native (RN). When a user opens the app, the client checks whether newer RN resources are available. If so, the backend silently downloads the new resources and reloads RN, allowing updates without requiring the user to reinstall the app from Google Play/App Store.

Although RN resources are updated frequently, the differences between successive versions are usually small. Downloading the entire RN bundle wastes user bandwidth and degrades the user experience because the silent background update still consumes network resources.

Therefore, an incremental (diff) update strategy is needed: only the differences between the old and new RN resources are packaged into an incremental patch file. After the patch is downloaded and validated, it is merged with the old version to produce the new version, saving traffic.

Given Shopee’s target markets with limited network conditions, reducing data traffic is crucial. This article analyses various incremental‑update schemes, discusses their pros and cons, and finally presents the FolderBsdp algorithm, which performs diff directly on the raw resources.

2. Evolution of Incremental‑Package Practices

2.1 Direct Transmission of Raw Files

RN resources (transpiled JavaScript, translation JSON files, images, configuration files) are referred to as “direct resources”. If these raw files are stored on a CDN with both old and new versions, the client could download only the newly added or modified files.

Drawbacks:

1) Distributed transmission is error‑prone

Multiple HTTP requests are required for new images, modified translation files, etc., complicating error handling (e.g., partial download recovery).

2) Large traffic consumption

Transmitting raw images and JSON files wastes bandwidth because they could be compressed. JSON files are often only slightly changed, yet the whole file would be sent.

2.2 File‑Level Diff Algorithm – BSDiff

To avoid the problems of 2.1, a diff algorithm is introduced.

2.2.1 Diff Algorithm

A file‑level diff takes an old file and a new file as input and outputs a Patch (the incremental package). A corresponding patch‑apply algorithm takes the old file and the Patch to reconstruct the new file. Different diff algorithms vary in time/space complexity, Patch size, and readability.

2.2.2 Bsdp (BSDiff & BSPatch) Algorithm

BSDiff focuses on generating the smallest possible Patch and is widely used (e.g., in Google Chrome). It is open‑source, written in C, and can run on servers as well as Android/iOS clients. BSPatch applies the Patch to the old file. Together they are referred to as the Bsdp algorithm.

2.2.3 Pros and Cons of Directly Using BSDiff

While BSDiff reduces transmitted data compared with raw‑file transmission, it does not solve the problem of multiple HTTP requests for many files. A method to bundle resources for a single download is still needed.

2.3 BSDiff on Compressed Resources

To address the distributed‑transfer issue, all RN resources are first placed into a folder and compressed into a ZIP archive. The initial installation uses the full ZIP package; subsequent updates download a Patch that, when applied with BSPatch, produces a new ZIP package.

Advantages: fewer HTTP requests, reduced total size (e.g., 14 MB uncompressed → ~4 MB compressed).

However, ZIP compression changes the byte‑level similarity between old and new resources, causing the generated Patch to be larger than ideal.

压缩不利差分性质

Compression algorithms (LZ sliding‑window, Huffman coding) make small modifications in the original file produce large changes in the compressed output, inflating Patch size.

2.4 Store‑File BSDiff

To avoid the “compression harms diff” problem, an uncompressed archive (Store file) is used. ZIP’s Store mode or a TAR archive can hold the raw files without compression, preserving byte‑level similarity.

ZIP Store mode is easy to generate (set the store flag to true) and is supported by Android SDK, but cross‑platform incompatibilities arise: a Store file created with Node.js’s archiver library cannot be read by Android’s native ZIP implementation.

Tar archives also carry platform‑specific metadata and would require additional dependencies on mobile clients, conflicting with the “minimal dependency” principle.

3. Best‑Practice Solution – FolderBsdp

FolderBsdp builds on Bsdp to diff entire folder hierarchies. It consists of FolderBSDiff (diff) and FolderBSPatch (apply).

3.1 FolderBSDiff Algorithm Details

The algorithm compares the directory trees of the old and new folders, detecting five change types: added directories, deleted directories, added files, deleted files, and modified files.

For added/deleted directories and files, simple operations are recorded. For modified files, BSDiff is invoked on the raw file contents, producing tiny Patch files and avoiding the “compression harms diff” issue. Added files are copied as‑is. All operations are serialized into a JSON manifest, and the resulting Patch files plus new files are bundled into a ZIP archive (the FolderBSDiff Patch).

Advantage 1 – Low Memory Usage on Client

FolderBSPatch processes each resource file sequentially, keeping only the old file, the Patch, and the new file in memory at any time. This reduces peak memory consumption by about 40 % compared with applying a monolithic ZIP Patch.

Advantage 2 – Storage Savings

Because the client no longer needs to retain the full ZIP or Store archive, storage space is reclaimed. In a Shopee app example, the total app size is 202 MB; the RN ZIP package (~8 MB) is eliminated after adopting FolderBsdp.

Advantage 3 – No ZIP Library Required

FolderBsdp eliminates the need for a ZIP decompression library on iOS and reduces dependency weight on Android, aligning with the “minimal dependency” goal.

Advantage 4 – Precise MD5 Verification

MD5 hashes of each added, deleted, or modified file are stored in the JSON manifest, enabling strict integrity checks during patch application.

4. Comparison with Industry Solutions

4.1 Google’s archive‑patcher

Google’s solution also extracts original byte streams before diffing, solving the compression‑diff problem, but it still requires retaining the ZIP package on the client due to cross‑platform ZIP nondeterminism, which conflicts with storage‑saving goals.

4.2 Other Approaches

Some open‑source projects merge all files into an uncompressed archive before diffing, similar to the Store‑file idea, but they often consume more memory during patching. FolderBsdp’s per‑file patching offers better memory efficiency, which is critical for low‑end devices.

5. Conclusion

FolderBsdp reduces memory peaks during patching, eliminates the need to keep large ZIP archives on the client, provides per‑file MD5 verification, and requires no extra native dependencies. It therefore represents the best practice for incremental RN resource updates at Shopee.