Understanding the Dynamic Freeze Threshold in OceanBase: Source Code Analysis and Debugging

1 Background

During load testing with a small tenant, we observed that the memory freeze threshold displayed in monitoring changes dynamically. In non‑load‑test conditions the threshold is about 322 MB, calculated as memstore_limit_percentage * freeze_trigger_percentage , while during load testing the threshold continuously drops below this value.

Why does the threshold vary under load? The following analysis explains the cause.

Environment Information

Version

OceanBase_CE 4.1.0.0

Tenant Specification

unit 8C4G

memstore_limit_percentage

50 (default)

freeze_trigger_percentage

20 (default)

sysbench Command

sysbench /usr/share/sysbench/oltp_read_only.lua --mysql-host=x.x.x.x --mysql-port=42881 --mysql-db=sysbenchdb --mysql-user="sysbench@sysbench_tenant2" --mysql-password=sysbench --tables=10 --table_size=1000000 --threads=1024 --time=120 --report-interval=10 --db-driver=mysql --db-ps-mode=disable --skip-trx=on --mysql-ignore-errors=6002,6004,4012,2013,4016,1062 run

2 Investigation Process

Verify Monitoring Data Collection

We executed the monitoring SQL from the official documentation and confirmed that the freeze threshold indeed changes during the test, indicating that the monitoring data collection and display are correct.

select /* MONITOR_AGENT */ con_id tenant_id, stat_id, value from v$sysstat, DBA_OB_TENANTS where stat_id IN (130002) and (con_id > 1000 or con_id = 1) and class < 1000

Inspect Source Code for Threshold Calculation

In the source tree ${path_to_oceanbase}/src/share/inner_table/ob_inner_table_schema_def.py we found the definition of the virtual view that maps to the virtual table __all_virtual_sysstat . The view v$sysstat is therefore a query interface for this virtual table.

The virtual table is handled by the class ObAllVirtualSysStat defined in ob_all_virtual_sys_stat.h/.cpp . The method inner_get_next_row() eventually calls ObTenantFreezer::get_freeze_trigger_() , which contains the actual freeze‑threshold calculation.

Dynamic Debugging Verification

Using VSCode + gdb we set a breakpoint in ObTenantFreezer::get_freeze_trigger_() , opened a connection to an OBServer, started debug mode, and manually executed the monitoring SQL. Because multiple call paths may reach get_freeze_trigger_() , we verified the stack to ensure the breakpoint was hit by the intended SQL.

oceanbase::storage::ObTenantFreezer::get_freeze_trigger_(oceanbase::storage::ObTenantFreezeCtx & ctx) (\opt\oceanbase\src\storage\tx_storage\ob_tenant_freezer.cpp:838)
... (stack trace omitted for brevity) ...

The stack shows that after generic SQL processing, execution enters ObVirtualTableIterator to handle the virtual table, and finally reaches the freeze‑trigger calculation.

Understanding the Calculation in Source

int ObTenantFreezer::get_freeze_trigger_(ObTenantFreezeCtx &ctx)
{
  int ret = OB_SUCCESS;
  // obtain tenant ID, memstore limit, etc.
  const uint64_t tenant_id = MTL_ID();
  const int64_t mem_memstore_limit = ctx.mem_memstore_limit_;
  int64_t kv_cache_mem = 0;
  int64_t memstore_freeze_trigger = 0;
  int64_t max_mem_memstore_can_get_now = 0;
  // ... obtain resource manager, calculate various memory variables ...
  // determine if overflow and compute min value
  int64_t min = mem_memstore_limit;
  if (!is_overflow) {
    min = MIN(mem_memstore_limit, max_mem_memstore_can_get_now);
  }
  if (min < 100) {
    memstore_freeze_trigger = get_freeze_trigger_percentage_() * min / 100;
  } else {
    memstore_freeze_trigger = min / 100 * get_freeze_trigger_percentage_();
  }
  ctx.max_mem_memstore_can_get_now_ = max_mem_memstore_can_get_now;
  ctx.memstore_freeze_trigger_ = memstore_freeze_trigger;
  ctx.kvcache_mem_ = kv_cache_mem;
  return ret;
}

Calculation Summary

The method uses seven memory‑related variables: tenant_mem_limit, tenant_mem_hold, mem_memstore_limit, tenant_memstore_hold, kv_cache_mem, max_mem_memstore_can_get_now, and memstore_freeze_trigger.

If the resource manager cannot be obtained, the fallback formula freeze_trigger_percentage / 100 * memstore_limit is used (currently always yields 0 due to a bug).

When the resource manager is available, the final freeze threshold is based on the smaller of mem_memstore_limit and max_mem_memstore_can_get_now , where the latter equals unallocated memory + memstore hold + kv_cache hold .

The chosen minimum is multiplied by freeze_trigger_percentage using one of two equivalent formulas depending on whether the minimum is below or above 100.

3 Summary

In OceanBase the freeze threshold is not a fixed constant; it is computed in real time according to the current memory situation. When memory modules other than MemStore and kv_cache occupy a large portion of memory, they squeeze MemStore and cause the threshold to drop.

In the observed scenario the growth of fuse_row_cache (blue line) inversely mirrors the freeze‑threshold curve, indicating that fuse_row_cache consumption reduces the available MemStore memory and thus lowers the threshold.

Operations Recommendation

When operators notice the freeze threshold moving, it signals that other memory modules are encroaching on MemStore. They should closely monitor overall memory usage and evaluate the impact on the workload.

Note that the max_mem_memstore_can_get_now formula only includes the KvstorCacheMb module, not all dynamically expandable memory such as fuse_row_cache or user_row_cache . These will be covered in a future article.

References

PR: https://github.com/oceanbase/oceanbase/pull/1440

Issue: https://github.com/oceanbase/oceanbase/issues/1439