ByteDance Service Framework: Compile‑Time Merged Microservices – Practices, Challenges, and Solutions

ByteDance operates over 300,000 microservices, causing significant overhead in encoding/decoding, serialization, networking, and service governance; to address this, the company introduced a compile‑time merging approach that combines multiple services into a single binary, reducing RPC overhead to zero.

How merge compilation works : two or more services are compiled together so that calls between them become in‑process function calls instead of RPCs. This eliminates network latency and serialization costs while preserving independent development responsibilities until deployment.

Advantages include performance gains (zero RPC overhead), maintained development autonomy, flexible merging and splitting, and improved stability because network‑related failures disappear.

Drawbacks involve loss of runtime isolation, version‑management complexity, and the need to recompile dependent services when updates occur.

Challenges and solutions :

Dependency isolation – using Go modules and replace directives to place each service’s dependencies in separate directories, e.g. import ( "namespaceA/github.com/cloudweGo/kitex" "namespaceB/github.com/cloudweGo/kitex" ) and replace github.com/cloudweGo/kitex => /tmp/kitex .

Call conversion – exposing the server’s main function as an exportable function that returns server metadata, then injecting it into a custom ServiceInlineClient so that Call executes a local function instead of an RPC. Example code: type Client interface { Call(ctx context.Context, method string, request, response interface{}) error } serverInfo := server.Main() kc, err := client.NewServiceInlineClient(serviceInfo(), serverInfo, options…)

Version management – integrating merge compilation with the internal release platform to automatically propagate upstream/downstream version information, allowing downstream teams to set default upstream versions and enabling coordinated upgrades.

Service selection criteria focus on services with high resource consumption, strong upstream/downstream traffic affinity, and encoding overhead above 3 %. Additional filters exclude cache‑heavy services, highly loaded containers, and memory‑constrained services.

Case study : a pair of services with a 96 % upstream traffic ratio and ~10 % encoding cost yielded over 40 k core‑CPU savings and latency reductions of 2–15 ms after merging.

Scale : Roughly 1.8 w merge‑eligible links (≈2.6 × 10⁸ cores) exist; a 10 % merge benefit would save about 67 k cores. To date, more than 300 k cores have been saved.

Summary and outlook : Compile‑time merging has proven viable at ByteDance, but current point‑to‑point adoption lacks a global view. Future work aims for platform‑wide, top‑down automation, integrating merge compilation with business‑domain governance to reduce architectural complexity without code changes.