Pinpoint Overview and Plugin Development Guide

Author Zhang Lei, a R&D engineer on Dada's infrastructure team, focuses on monitoring and middleware, promotes Pinpoint adoption, and develops custom Pinpoint plugins such as for okhttp3.8.

Pinpoint is a full‑link analysis tool that provides non‑intrusive call‑chain monitoring, method execution details, and application status monitoring. It implements concepts from the Google Dapper paper and offers richer capabilities than Zipkin, including service topology visualization, real‑time active thread charts, request‑response scatter plots, request stack view, and host‑level metrics such as CPU, memory, GC, TPS, and JVM information.

The system architecture consists of three main components—Agent, Collector, and Web UI—backed by an HBase database. The Agent collects monitoring data with minimal code changes, the Collector stores the data, and the Web UI visualizes relationships, detailed call information, and alerts.

Key system characteristics include distributed transaction tracing, automatic topology detection, horizontal scalability for large server groups, code‑level visibility, and bytecode injection that avoids source code modification.

Pinpoint records each request as a series of Spans; each Span represents a single execution node and contains attributes such as transaction ID, parent/child IDs, and annotations. This data model is called TraceData.

Plugin development requires implementing two interfaces: TraceMetadataProvider and ProfilerPlugin. TraceMetadataProvider registers ServiceType and AnnotationKey definitions, while ProfilerPlugin injects bytecode to capture monitoring data.

Steps to create a Pinpoint plugin:

Configure ServiceLoader files under META-INF/services for TraceMetadataProvider and ProfilerPlugin implementations.

Implement TraceMetadataProvider to define custom ServiceType and AnnotationKey codes.

Implement ProfilerPlugin (or extend provided base interceptors) to intercept target methods, record service type, RPC name, endpoints, parent application information, and attributes such as arguments and results.

Example code for a server‑side RPC plugin:

public class Sample_14_RPC_Server implements TransformCallback {
    @Override
    public byte[] doInTransform(Instrumentor instrumentor, ClassLoader classLoader, String className,
                                Class
classBeingRedefined, ProtectionDomain protectionDomain,
                                byte[] classfileBuffer) throws InstrumentException {
        InstrumentClass target = instrumentor.getInstrumentClass(classLoader, className, classfileBuffer);
        target.getDeclaredMethod("process", "com.navercorp.plugin.sample.target.TargetClass14_Request")
              .addInterceptor("com.navercorp.pinpoint.plugin.sample._14_RPC_Server.ProcessInterceptor");
        return target.toBytecode();
    }
}

public class ProcessInterceptor extends SpanSimpleAroundInterceptor {
    public ProcessInterceptor(TraceContext traceContext, MethodDescriptor descriptor) {
        super(traceContext, descriptor, ProcessInterceptor.class);
    }
    @Override
    protected Trace createTrace(Object target, Object[] args) {
        TargetClass14_Request request = (TargetClass14_Request)args[0];
        if (request.getMetadata(SamplePluginConstants.META_DO_NOT_TRACE) != null) {
            return traceContext.disableSampling();
        }
        String transactionId = request.getMetadata(SamplePluginConstants.META_TRANSACTION_ID);
        if (transactionId == null) {
            return traceContext.newTraceObject();
        }
        long parentSpanID = NumberUtils.parseLong(request.getMetadata(SamplePluginConstants.META_PARENT_SPAN_ID), SpanId.NULL);
        long spanID = NumberUtils.parseLong(request.getMetadata(SamplePluginConstants.META_SPAN_ID), SpanId.NULL);
        short flags = NumberUtils.parseShort(request.getMetadata(SamplePluginConstants.META_FLAGS), (short)0);
        TraceId traceId = traceContext.createTraceId(transactionId, parentSpanID, spanID, flags);
        return traceContext.continueTraceObject(traceId);
    }
    @Override
    protected void doInBeforeTrace(SpanRecorder recorder, Object target, Object[] args) {
        TargetClass14_Server server = (TargetClass14_Server)target;
        TargetClass14_Request request = (TargetClass14_Request)args[0];
        recorder.recordServiceType(SamplePluginConstants.MY_RPC_SERVER_SERVICE_TYPE);
        recorder.recordRpcName(request.getProcedure());
        recorder.recordEndPoint(server.getAddress());
        recorder.recordRemoteAddress(request.getClientAddress());
        if (!recorder.isRoot()) {
            String parentApplicationName = request.getMetadata(SamplePluginConstants.META_PARENT_APPLICATION_NAME);
            if (parentApplicationName != null) {
                short parentApplicationType = NumberUtils.parseShort(request.getMetadata(SamplePluginConstants.META_PARENT_APPLICATION_TYPE), ServiceType.UNDEFINED.getCode());
                recorder.recordParentApplication(parentApplicationName, parentApplicationType);
                String serverHostName = request.getServerHostName();
                recorder.recordAcceptorHost(serverHostName != null ? serverHostName : server.getAddress());
            }
        }
    }
    @Override
    protected void doInAfterTrace(SpanRecorder recorder, Object target, Object[] args,
                                 Object result, Throwable throwable) {
        TargetClass14_Request request = (TargetClass14_Request)args[0];
        recorder.recordApi(methodDescriptor);
        recorder.recordAttribute(SamplePluginConstants.MY_RPC_ARGUMENT_ANNOTATION_KEY, request.getArgument());
        if (throwable == null) {
            recorder.recordAttribute(SamplePluginConstants.MY_RPC_RESULT_ANNOTATION_KEY, result);
        } else {
            recorder.recordException(throwable);
        }
    }
}

Additional examples demonstrate how to inject custom interceptors for ordinary methods and how to handle asynchronous tasks by generating AsyncTraceId and propagating it.

After building the plugin JAR, it must be deployed to the Pinpoint Agent, and the corresponding ServiceType definitions must also be added to the Web UI and Collector so that all modules share consistent tracing information.