Improving WebGL Engine Performance with IndexedDB: Data Retrieval and Update Strategies

The Crystal WebGL engine, a core component of the Zhengtong Crystal platform, has powered hundreds of dashboards and business systems since its launch in 2018, continuously enhancing visual effects, spatiotemporal analysis, and product performance.

To further boost performance, the team adopted several techniques such as texture merging and compression, geometry simplification, LOD paging, data compression, mesh merging, and instanced rendering, all aimed at reducing network transfer and rendering overhead.

In addition, a browser‑side storage solution based on IndexedDB was introduced to cache historical data, minimizing network requests and improving user interaction.

Background of introducing a frontend database: Data acquisition speed is heavily impacted by both network transmission efficiency and data processing efficiency. Because browsers limit concurrent HTTP/1.1 requests (e.g., Chrome allows six per domain), loading large WebGL assets (JS, CSS, images, geojson, 3DTiles, etc.) becomes a bottleneck even after compression.

By avoiding network requests and retrieving data directly from local storage, the engine can bypass these constraints.

1. Introduction to IndexedDB

IndexedDB (idb) is a browser‑provided local database offering key‑value storage with binary support, no practical size limit, asynchronous transactional operations, query capabilities, persistence across sessions, and same‑origin restrictions.

Key‑value storage, binary support, unlimited size.

Asynchronous API with transactions and query support.

Persistent data that survives browser cache clearing.

Same‑origin security model.

Because of its virtually unlimited size, IndexedDB is ideal for caching images and can be extended to other file types.

2. Loading Efficiency Comparison

The chart shows two groups of tests: loading with and without browser cache versus loading from IndexedDB. Results indicate that IndexedDB dramatically improves loading speed for uncached online data and also provides noticeable gains for already cached data.

3. IndexedDB Read Acceleration Scheme

Data are stored as key‑value pairs; to speed up massive reads, data are categorized into separate databases and tables (e.g., image cache, 3DTiles, JSON). This reduces the amount of data queried per request.

An array capacity limit is set, and a combined FIFO and hit‑rate policy is used for cache eviction, further lowering read time complexity.

4. IndexedDB Data Update Scheme

To avoid stale data, a version‑control database records the version of each cached item. Data are refreshed only when the version changes.

For low‑frequency updates, a timestamp‑based cleanup automatically removes outdated entries. For high‑frequency updates, the backend can push notifications to trigger immediate cache refreshes.

5. Conclusion

Utilizing IndexedDB substantially reduces network consumption during repeated data loads, enhances loading efficiency, and, together with structured storage strategies and multiple update mechanisms, greatly strengthens the engine’s practical capabilities.