Instrumentation of gRPC in OpenTelemetry: Adding Request Size Metrics via Byte‑Buddy

Recently a pull request was submitted to opentelemetry-java-instrumentation that adds four new gRPC metrics: rpc.client.request.size (client request size), rpc.client.response.size (client response size), rpc.server.request.size (server request size) and rpc.server.response.size (server response size). The main goal of the PR is to make the size of RPC payloads observable in monitoring systems.

gRPC is the most widely used RPC framework in the cloud‑native ecosystem, and the same approach can be applied to other RPC implementations.

OpenTelemetry creates a span for each gRPC call by using the framework’s interceptor interfaces. The client side uses io.grpc.ClientInterceptor and the server side uses io.grpc.ServerInterceptor . By implementing these interceptors, custom logic such as metric collection and attribute extraction can be injected without modifying business code.

The Java agent provided by OpenTelemetry relies on Byte‑Buddy to enhance bytecode at runtime. The agent adds the tracing interceptor to the gRPC channel builder, which would otherwise require manual code changes.

@Override
public
ClientCall
interceptCall(
    MethodDescriptor
method,
    CallOptions callOptions,
    Channel next) {
  GrpcRequest request = new GrpcRequest(method, null, null, next.authority());
  Context parentContext = Context.current();
  if (!instrumenter.shouldStart(parentContext, request)) {
    return next.newCall(method, callOptions);
  }
  Context context = instrumenter.start(parentContext, request);
  ClientCall
result;
  try (Scope ignored = context.makeCurrent()) {
    try {
      // call other interceptors
      result = next.newCall(method, callOptions);
    } catch (Throwable e) {
      instrumenter.end(context, request, Status.UNKNOWN, e);
      throw e;
    }
  }
  return new TracingClientCall<>(result, parentContext, context, request);
}

To inject the interceptor without changing application code, Byte‑Buddy matches the ManagedChannelBuilder class and its interceptors field, then applies advice that inserts TracingClientInterceptor as the first interceptor.

var managedChannel = ManagedChannelBuilder.forAddress(host, port)
    .intercept(new TracingClientInterceptor()) // add interceptor
    .usePlaintext()
    .build();

public class GrpcClientBuilderBuildInstrumentation {
  @Override
  public ElementMatcher
typeMatcher() {
    return extendsClass(named("io.grpc.ManagedChannelBuilder"))
        .and(declaresField(named("interceptors")));
  }

  @Override
  public void transform(TypeTransformer transformer) {
    transformer.applyAdviceToMethod(
        isMethod().and(named("build")),
        GrpcClientBuilderBuildInstrumentation.class.getName() + "$AddInterceptorAdvice");
  }

  @SuppressWarnings("unused")
  public static class AddInterceptorAdvice {
    @Advice.OnMethodEnter(suppress = Throwable.class)
    public static void addInterceptor(
        @Advice.This ManagedChannelBuilder
builder,
        @Advice.FieldValue("interceptors") List
interceptors) {
      VirtualField
, Boolean> instrumented =
          VirtualField.find(ManagedChannelBuilder.class, Boolean.class);
      if (!Boolean.TRUE.equals(instrumented.get(builder))) {
        interceptors.add(0, GrpcSingletons.CLIENT_INTERCEPTOR);
        instrumented.set(builder, true);
      }
    }
  }
}

The instrumentation also extracts span attributes grouped into three categories:

net.* – network‑related attributes (IP, port, etc.)

rpc.* – gRPC‑specific attributes (service, method, status code)

thread.* – thread information

OpenTelemetry provides an InstrumenterBuilder where custom attribute extractors can be registered:

clientInstrumenterBuilder
    .setSpanStatusExtractor(GrpcSpanStatusExtractor.CLIENT)
    .addAttributesExtractors(additionalExtractors)
    .addAttributesExtractor(RpcClientAttributesExtractor.create(rpcAttributesGetter))
    .addAttributesExtractor(ServerAttributesExtractor.create(netClientAttributesGetter))
    .addAttributesExtractor(NetworkAttributesExtractor.create(netClientAttributesGetter));

One concrete extractor is GrpcRpcAttributesGetter , which supplies the fixed system name grpc and parses the full method name to obtain the service name.

enum GrpcRpcAttributesGetter implements RpcAttributesGetter
{
  INSTANCE;

  @Override
  public String getSystem(GrpcRequest request) {
    return "grpc";
  }

  @Override
  @Nullable
  public String getService(GrpcRequest request) {
    String fullMethodName = request.getMethod().getFullMethodName();
    int slashIndex = fullMethodName.lastIndexOf('/');
    if (slashIndex == -1) {
      return null;
    }
    return fullMethodName.substring(0, slashIndex);
  }
}

To obtain the request payload size, a helper method checks whether the message implements MessageLite (protobuf) and returns its serialized size.

static
Long getBodySize(T message) {
  if (message instanceof MessageLite) {
    return (long) ((MessageLite) message).getSerializedSize();
  } else {
    // Message is not a protobuf message
    return null;
  }
}

Custom metrics are added via the addOperationMetrics API. The onStart callback records the start time, and onEnd computes the duration and records it in a histogram.

@Override
public Context onStart(Context context, Attributes startAttributes, long startNanos) {
  return context.with(RPC_CLIENT_REQUEST_METRICS_STATE,
      new AutoValue_RpcClientMetrics_State(startAttributes, startNanos));
}

@Override
public void onEnd(Context context, Attributes endAttributes, long endNanos) {
  State state = context.get(RPC_CLIENT_REQUEST_METRICS_STATE);
  Attributes attributes = state.startAttributes().toBuilder().putAll(endAttributes).build();
  clientDurationHistogram.record((endNanos - state.startTimeNanos()) / NANOS_PER_MS,
      attributes, context);
}

In Go, where Byte‑Buddy is unavailable, the instrumentation is performed by passing an OpenTelemetry‑provided stats handler to the gRPC server:

s := grpc.NewServer(
    grpc.StatsHandler(otelgrpc.NewServerHandler()),
)

The overall approach is to first look for extension points exposed by the target framework (e.g., interceptor interfaces). If such points exist, they are used directly; otherwise, bytecode manipulation (Java) or other low‑level techniques (Go) are employed to inject the necessary tracing logic.