Deep Dive into JD's PFinder: Architecture, Bytecode Instrumentation, and Monitoring Features

In modern software development, performance optimization and fault diagnosis are essential, and Java developers rely on APM tools such as SkyWalking, Zipkin, and JD's internally built PFinder to achieve full‑stack observability.

PFinder Overview

PFinder (Problem Finder) is a next‑generation APM system created by JD's UMP team. It offers call‑chain tracing, application topology, and multi‑dimensional monitoring without requiring code changes—only two script lines added to the startup configuration. It supports JD's middleware (jimdb, jmq, jsf) and common open‑source components (Tomcat, HTTP client, MySQL, Elasticsearch).

Key Features

Multi‑dimensional monitoring : aggregate metrics by data center, group, JSF alias, caller, etc.

Automatic instrumentation : auto‑inject probes into Spring MVC, JSF, MySQL, JMQ, etc.

Application topology : automatically map upstream/downstream service dependencies.

Call‑chain tracing : trace cross‑service requests to locate bottlenecks.

AI‑driven fault analysis : automatically determine root causes from monitoring data.

Traffic recording & replay : capture live traffic and replay in test or pre‑release environments.

Cross‑unit escape traffic monitoring : monitor JSF traffic across micro‑service units.

Bytecode Modification Frameworks

PFinder relies on bytecode enhancement. The article compares four popular Java bytecode libraries—ASM, Javassist, ByteBuddy, and ByteKit—by implementing the same functionality (printing "start" before a method and "end" after it).

@Override
public void visitCode() {
    super.visitCode();
    mv.visitFieldInsn(GETSTATIC, "java/lang/System", "out", "Ljava/io/PrintStream;");
    mv.visitLdcInsn("start");
    mv.visitMethodInsn(INVOKEVIRTUAL, "java/io/PrintStream", "println", "(Ljava/lang/String;)V", false);
}

@Override
public void visitInsn(int opcode) {
    if ((opcode >= Opcodes.IRETURN && opcode <= Opcodes.RETURN) || opcode == Opcodes.ATHROW) {
        mv.visitFieldInsn(GETSTATIC, "java/lang/System", "out", "Ljava/io/PrintStream;");
        mv.visitLdcInsn("end");
        mv.visitMethodInsn(INVOKEVIRTUAL, "java/io/PrintStream", "println", "(Ljava/lang/String;)V", false);
    }
    mv.visitInsn(opcode);
}

ClassPool cp = ClassPool.getDefault();
CtClass cc = cp.get("com.ggc.javassist.HelloWord");
CtMethod m = cc.getDeclaredMethod("printHelloWord");
m.insertBefore("{ System.out.println(\"start\"); }");
m.insertAfter("{ System.out.println(\"end\"); }");
Class c = cc.toClass();
cc.writeFile("/path/to/classes");
HelloWord h = (HelloWord)c.newInstance();
h.printHelloWord();

// ByteBuddy example
Class
dynamicType = new ByteBuddy()
    .subclass(HelloWord.class)
    .method(ElementMatchers.named("printHelloWord"))
    .intercept(MethodDelegation.to(LoggingInterceptor.class))
    .make()
    .load(HelloWord.class.getClassLoader())
    .getLoaded();

HelloWord dynamicService = (HelloWord) dynamicType.newInstance();
dynamicService.printHelloWord();

public class LoggingInterceptor {
    @RuntimeType
    public static Object intercept(@AllArguments Object[] allArguments, @Origin Method method, @SuperCall Callable
callable) throws Exception {
        System.out.println("start");
        try {
            Object result = callable.call();
            System.out.println("end");
            return result;
        } catch (Exception e) {
            System.out.println("exception end");
            throw e;
        }
    }
}

Bytecode Injection via JVMTI and java.lang.instrument

JVMTI (JVM Tool Interface) is the low‑level native API that underlies Java agents. The article explains the three core JVMTI callbacks— Agent_OnLoad , Agent_OnAttach , and Agent_OnUnload —and how modern Java agents use java.lang.instrument.Instrumentation to perform bytecode transformation in pure Java.

<archive>
   <manifestEntries>
       <Agent-CLass>com.ggc.agent.GhlAgent</Agent-CLass>
       <Premain-Class>com.ggc.agent.GhlAgent</Premain-Class>
       <Can-Redefine-Classes>true</Can-Redefine-Classes>
       <Can-Retransform-Classes>true</Can-Retransform-Classes>
   </manifestEntries>
</archive>

public class GhlAgent {
    public static void premain(String agentArgs, Instrumentation instrumentation) {
        boot(instrumentation);
    }
    public static void agentmain(String agentArgs, Instrumentation instrumentation) {
        boot(instrumentation);
    }
    private static void boot(Instrumentation instrumentation) {
        new AgentBuilder.Default()
            .type(ElementMatchers.nameStartsWith("com.jd.aviation.performance.service.impl"))
            .transform((builder, typeDescription, classLoader, javaModule) ->
                builder.method(ElementMatchers.isMethod().and(ElementMatchers.isPublic()))
                       .intercept(MethodDelegation.to(TimingInterceptor.class)))
            .installOn(instrumentation);
    }
}

public class TimingInterceptor {
    public static Object intercept(@SuperCall Callable
callable) throws Exception {
        long start = System.currentTimeMillis();
        try {
            return callable.call();
        } finally {
            long end = System.currentTimeMillis();
            log.info("Method call took {} ms", (end - start));
        }
    }
}

PFinder Architecture

When the PFinder agent starts, it loads META-INF/pfinder/service.addon and META-INF/pfinder/plugin.addon to register services and plugins. Services are instantiated via SimplePFinderServiceLoader , while plugins are loaded by PluginLoader . The loaded plugins provide bytecode enhancement points that are applied using ByteBuddy's AgentBuilder .

Service and Plugin Loading Process

The service loading flow creates a service factory iterator, registers, and initializes each service. Plugin loading occurs during the PluginRegistrar service initialization, where all plugin matchers are combined into a single typeMatcherChain and passed to AgentBuilder for transformation.

Multi‑Thread TraceId Propagation

PFinder stores the traceId in MDC (a ThreadLocal). To avoid loss across thread boundaries, it wraps user runnables with TracingRunnable , capturing the originating snapshot and restoring the traceId in the child thread's context.

public class TracingRunnable implements PfinderWrappedRunnable {
    private final Runnable origin;
    private final TracingSnapshot
snapshot;
    // ... constructor omitted ...
    public void run() {
        TracingContext tracingContext = ContextManager.tracingContext();
        if (tracingContext.isTracing() && tracingContext.traceId().equals(this.snapshot.getTraceId())) {
            this.origin.run();
            return;
        }
        LowLevelAroundTracingContext context = SpringAsyncTracingContext.create(this.operationName, this.interceptorName, this.snapshot, this.interceptorClassLoader, this.component);
        context.onMethodEnter();
        try {
            this.origin.run();
        } catch (RuntimeException ex) {
            context.onException(ex);
            throw ex;
        } finally {
            context.onMethodExit();
        }
    }
}

Hot Deployment Prototype

Leveraging the Java agent’s ability to redefine classes, PFinder can receive commands from the server via JMTP to search, decompile, and hot‑replace classes. The article shows screenshots of class search, decompilation, and live update, while noting current limitations such as lack of Spring XML/MyBatis XML support and the inability to add fields or methods without using DCEVM.

Overall, the article serves as a detailed technical guide for developers interested in building or extending Java APM solutions through bytecode instrumentation, agent development, and runtime monitoring.