Design and Implementation of Shopee Games Unified API Gateway

The Shopee Games team has built several casual e‑commerce games (e.g., Shopee Candy, Shopee Farm, Shopee Shake) to create a more enjoyable shopping experience and increase traffic for merchants. As the number of backend services grew into a micro‑service architecture, the team needed a unified API gateway to handle both external user traffic and internal service calls.

Background

Shopee Games adopts a studio‑plus‑platform organization: each studio owns a set of games, while a central Game Platform provides shared services such as points, lives, and exchange stores. Some scenarios require internal service calls (e.g., cross‑game interactions or admin platform integration), which makes a single HTTP server insufficient for handling multiple authentication logics and traffic sources.

To solve this, the team decided to standardize on RPC (via Protobuf) for internal communication and to place a Web Server in front of each RPC service for user‑facing traffic. However, this introduced repetitive work: manual HTTP‑to‑RPC conversion code, duplicated deployment pipelines, and resource waste when each game runs its own proxy.

Goals

The API gateway should:

Hide external traffic details and let developers focus on business logic.

Enforce a unified interface specification (HTTP status codes, error handling, documentation).

Provide cross‑cutting capabilities such as authentication, logging, device info collection, and monitoring.

Improve resource utilization by sharing gateway capacity among services.

Open‑source solutions evaluated

1. gRPC‑Gateway – a Go‑based code generator that creates HTTP server scaffolding from Protobuf annotations.

syntax = "proto3";
package helloworld;
import "google/api/annotations.proto";
option go_package = "gen/pb";
service Hello {
  rpc SayHello(HelloRequest) returns (HelloResponse) {
    option (google.api.http) = {
      post: "/v1/example/echo"
      body: "*"
    };
  };
}
message HelloRequest {
  string str = 1;
}
message HelloResponse {
  string str = 1;
}

Running protoc with the gRPC‑Gateway plugin generates the HTTP server code.

protoc ./proto/hello.proto -I=./proto -I=${GOPATH}/googleapis \
    --go_out=${PWD}/gen/pb --go_opt paths=source_relative \
    --go-grpc_out=${PWD}/gen/pb --go-grpc_opt paths=source_relative \
    --grpc-gateway_out=${PWD}/gen/pb --grpc-gateway_opt paths=source_relative

2. Envoy – a C++‑based reverse proxy that supports a gRPC‑JSON transcoder.

Both solutions lack multi‑service support and require additional effort for dynamic configuration, so the team designed a custom gateway.

Technical design and implementation

Overall architecture

Backend RPC services register themselves to a service registry. The API gateway watches registry changes, maintains an in‑memory routing table, and is fronted by an Nginx L7 load balancer. The gateway parses the HTTP path to locate the target service and forwards the request after protocol conversion.

Protobuf‑based control plane

The gateway uses Protobuf files not only to describe RPC interfaces but also to embed control rules via custom options. By loading the FileDescriptorSet (FD) at runtime, the gateway can read service‑level, method‑level, and field‑level options to drive routing, authentication, logging, rate‑limiting, etc.

Example of a custom option in a proto file:

service bookService {
  option (xproto.svcTimeoutMs) = 3000; // service timeout in ms

  rpc GetBook(GetBookRequest) returns (GetBookResponse) {
    option (google.api.http) = {
      get: "/get_book"
    };
  };
}

message GetBookRequest {
  int32 id = 1;
  string from = 2 [(xproto.queryKey) = "from"];
}

At runtime, the gateway parses the FD to obtain the option values and applies them automatically.

Descriptor handling

Using Go’s protobuf descriptor library:

import "google.golang.org/protobuf/types/descriptorpb"

var fds descriptorpb.FileDescriptorSet
_ = proto.Unmarshal(fileDescriptor, &fds)
svc := fds.GetFile()[0].GetService()[0]
method := svc.GetMethod()[0]
fmt.Println(method.GetInputType())

This enables dynamic message creation:

msg := dynamic.NewMessage(methodDescriptor.GetInputType())
msg.SetFieldByName("age", 1)
msg.SetFieldByName("name", "username")

Routing and metadata

HTTP paths are matched to RPC methods using both a map and a trie structure for fast lookup. Example routing definition in proto:

rpc SayHello(HelloRequest) returns (HelloResponse) {
  option (google.api.http) = {
    post: "/v1/example/echo"
  };
}

rpc GetBook(GetBookRequest) returns (GetBookResponse) {
  option (google.api.http) = {
    get: "/v1/example/book/:id"
  };
}

Metadata such as forwarded headers can be defined at the service level:

service exampleService {
  option (xproto.forwardHeaderKeys) = "User-Agent";

  rpc SayHello(HelloRequest) returns (HelloResponse) {
    option (google.api.http) = { post: "/v1/example/echo" };
  };
}

In Go, the gateway extracts the header via metadata.FromIncomingContext and can inject response headers back to the client.

Aspect-oriented features

By declaring options, developers get built‑in cross‑cutting concerns without writing extra code, e.g., authentication, blacklist filtering, timeout configuration, logging rules, A/B testing, and rate limiting.

Usability enhancements

Automatic Swagger generation from Protobuf annotations.

Docker image that bundles the gateway, protoc plugins, and development tools for one‑click local debugging.

Scaffolding CLI to bootstrap new projects with Makefile commands.

Stability guarantees

The gateway implements service governance (rate limiting, timeout, circuit breaking, load balancing, health checks), multi‑dimensional monitoring, alerting, fault‑tolerance playbooks, and performance optimizations such as removing reflection from the web framework, using sync.Pool for object reuse, and unsafe string‑byte conversions to avoid copies.

func b2s(b []byte) string {
  // Direct conversion without allocation
  return *(*string)(unsafe.Pointer(&b))
}

These measures have increased throughput by ~20% and have been validated during several large‑scale promotional events.

Conclusion

The Shopee Games API gateway demonstrates how dynamic FD retrieval, Protobuf‑driven control planes, and a unified gateway can simplify micro‑service integration, enforce consistent API contracts, and provide robust operational characteristics. Future work includes priority‑based resource governance and dynamic configuration overrides.