All libraries · ← executors

executors

Instrumentation v2.13.3 · 23 → 24 · 84 tests
from v2.13.3 to v2.14.0
from v2.13.3 to v2.14.0

Step 1: generate report using Claude

The prompt had the following elements as context :

  • instrumentation tests' stderr and stdout diff.
  • execution graphs diff (window=10)
  • library diff

The generated report is :

Verdict : unuseful suggestions the real problem reside in lambda-internal module

Java compiles a Runnable/Callable lambda expression to an invokedynamic call that the JVM resolves lazily at runtime via LambdaMetafactory, which spins up a brand-new hidden class on the fly. Because that class doesn't exist until the moment the lambda is first used, OTel can't rely on its normal "match Callable/Runnable and transform on class-load" mechanism working reliably for lambdas — that's the exact class-loading-order race I originally hypothesized. So instead of matching the generated lambda class itself, OTel instruments java.lang.invoke.InnerClassLambdaMetafactory — the JDK's own factory that builds every lambda class — and uses raw ASM to splice a call to LambdaTransformerHelper.transform(...) into its spinInnerClass()/generateInnerClass() method, right where the JDK turns its in-memory class-builder object into a byte[]. That injected call runs the OTel javaagent's normal instrumentation pipeline over the lambda's bytecode before the JVM ever defines the class — guaranteeing CallableInstrumentation/RunnableInstrumentation's call()/run() advice is woven into the lambda from its very first invocation, no race possible.

Why JDK 24 broke it: the ASM-splice targeted one very specific instruction — INVOKEVIRTUAL ClassWriter.toByteArray() — because that's how every JDK ≤23 built the lambda's bytecode internally (ASM's ClassWriter). JDK 24 replaced that internal machinery with the new java.lang.classfile API (JEP 484), so the method that finally produces the byte[] is now an interface call — INVOKEINTERFACE ...build()...)[B — not ClassWriter.toByteArray(). The old instruction-scan matched nothing on JDK 24, so the transform hook was silently never inserted, and every lambda-based Runnable/Callable — not just ForkJoinPool ones — lost context propagation on JDK 24, since their generated classes never passed through the javaagent at all.

Updated predcate to match new JDK