Creation, dynamic loading and instrumentation with javaagents

Referrers: ReverseEngineeringStackExchange, StackOverflow.

Sometime back I had to delve into the depths of how one could dynamically and programmatically load a javaagent at runtime – in other words how a javaagent could be attached to a running process. Since then I’ve been meaning to blog my findings. Finally, after quite some time, I’ve got around to it but here I take the opportunity to expand the scope of the article to a complete treatment of how to create a javaagent, load it statically at startup or dynamically at runtime and also how to perform instrumentation of a simple class. As always the code will reside in an eclipse maven project that will be made available for download.

Introduction

Java agents are self contained components through which application classes pass at the level of byte code instructions in the form of byte arrays. They were introduced in java5 along with the powerful java.lang.instrument package. Java agents can be loaded statically at startup or dynamically (programmatically) at runtime to attach to a running process in a fail-safe fashion.

Lifecycle of a javaagent

The most important thing to understand is the lifecycle of a javaagent and its behaviour in relation to the application itself. The lifecycle hook points are as follows.

• PreMain – The PreMain method is invoked first prior to running the main method. This is the hookpoint for loading the javaagent statically at startup.
• Main – The main method is always invoked after the PreMain invocation.
• AgentMain – The AgentMain method can be invoked at any time before or after the Main method. This hookpoint is for loading the javaagent dynamically at runtime and attaching to a running process.

Creation of a javaagent

In order to create a java agent you must first define your chosen hook points from the lifecycle above as below.

package name.dhruba.javaagent;import java.lang.instrument.Instrumentation;import org.slf4j.Logger;import org.slf4j.LoggerFactory;public class MyJavaAgent {    static final Logger logger = LoggerFactory.getLogger(MyJavaAgent.class);    private static Instrumentation instrumentation;    /**     * JVM hook to statically load the javaagent at startup.     *      * After the Java Virtual Machine (JVM) has initialized, the premain method     * will be called. Then the real application main method will be called.     *      * @param args     * @param inst     * @throws Exception     */    public static void premain(String args, Instrumentation inst) throws Exception {        logger.info("premain method invoked with args: {} and inst: {}", args, inst);        instrumentation = inst;        instrumentation.addTransformer(new MyClassFileTransformer());    }    /**     * JVM hook to dynamically load javaagent at runtime.     *      * The agent class may have an agentmain method for use when the agent is     * started after VM startup.     *      * @param args     * @param inst     * @throws Exception     */    public static void agentmain(String args, Instrumentation inst) throws Exception {        logger.info("agentmain method invoked with args: {} and inst: {}", args, inst);        instrumentation = inst;        instrumentation.addTransformer(new MyClassFileTransformer());    }    /**     * Programmatic hook to dynamically load javaagent at runtime.     */    public static void initialize() {        if (instrumentation == null) {            MyJavaAgentLoader.loadAgent();        }    }}

Once you’ve defined your hook points you must make the JVM aware by putting them in a manifest file.

Main-Class: name.dhruba.javaagent.MyMainClassAgent-Class: name.dhruba.javaagent.MyJavaAgentCan-Redefine-Classes: trueCan-Retransform-Classes: truePremain-Class: name.dhruba.javaagent.MyJavaAgent

In the example project I’m doing so using maven and the maven assembly plugin which also packages the project as a single all inclusive uber executable jar for ease of testing.

<plugin>  <groupId>org.apache.maven.plugins</groupId>  <artifactId>maven-assembly-plugin</artifactId>  <executions>    <execution>      <goals>        <goal>attached</goal>      </goals>      <phase>package</phase>      <configuration>        <descriptorRefs>          <descriptorRef>jar-with-dependencies</descriptorRef>        </descriptorRefs>        <archive>          <manifest>            <mainClass>name.dhruba.javaagent.MyMainClass</mainClass>          </manifest>          <manifestEntries>            <Premain-Class>name.dhruba.javaagent.MyJavaAgent</Premain-Class>            <Agent-Class>name.dhruba.javaagent.MyJavaAgent</Agent-Class>            <Can-Redefine-Classes>true</Can-Redefine-Classes>            <Can-Retransform-Classes>true</Can-Retransform-Classes>          </manifestEntries>        </archive>      </configuration>    </execution>  </executions></plugin>

Instrumenting classes

Within the javaagent hook points one can define java.lang.instrument.ClassFileTransformer implementations that are invoked for all classes being loaded within an application. These receive classes as byte arrays and have the option of redefining them also in terms of byte arrays. Here I provide an example class file transformation.

The following User pojo returns ‘foo’ as its name.

package name.dhruba.user;public class MyUser {    public String getName() {        return "foo";    }}

The following class file transformer (using ASM) redefines the User pojo to return ‘bar’ instead.

package name.dhruba.javaagent;import java.lang.instrument.ClassFileTransformer;import java.lang.instrument.IllegalClassFormatException;import java.security.ProtectionDomain;import org.objectweb.asm.ClassWriter;import org.objectweb.asm.Label;import org.objectweb.asm.MethodVisitor;import org.objectweb.asm.Opcodes;import org.slf4j.Logger;import org.slf4j.LoggerFactory;public class MyClassFileTransformer implements ClassFileTransformer, Opcodes {    static final Logger logger = LoggerFactory.getLogger(MyClassFileTransformer.class);    @Override    public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined,            ProtectionDomain protectionDomain, byte[] classfileBuffer)            throws IllegalClassFormatException {        logger.info("class file transformer invoked for className: {}", className);        if (className.equals("name/dhruba/user/MyUser")) {            ClassWriter cw = new ClassWriter(0);            MethodVisitor mv;            cw.visit(V1_6, ACC_PUBLIC + ACC_SUPER, "name/dhruba/user/MyUser", null,                    "java/lang/Object", null);            cw.visitSource(null, null);            {                mv = cw.visitMethod(ACC_PUBLIC, "", "()V", null, null);                mv.visitCode();                Label l0 = new Label();                mv.visitLabel(l0);                mv.visitLineNumber(3, l0);                mv.visitVarInsn(ALOAD, 0);                mv.visitMethodInsn(INVOKESPECIAL, "java/lang/Object", "", "()V");                mv.visitInsn(RETURN);                Label l1 = new Label();                mv.visitLabel(l1);                mv.visitLocalVariable("this", "Lname/dhruba/user/MyUser;", null, l0, l1, 0);                mv.visitMaxs(1, 1);                mv.visitEnd();            }            {                mv = cw.visitMethod(ACC_PUBLIC, "getName", "()Ljava/lang/String;", null, null);                mv.visitCode();                Label l0 = new Label();                mv.visitLabel(l0);                mv.visitLineNumber(6, l0);                mv.visitLdcInsn("bar");                mv.visitInsn(ARETURN);                Label l1 = new Label();                mv.visitLabel(l1);                mv.visitLocalVariable("this", "Lname/dhruba/user/MyUser;", null, l0, l1, 0);                mv.visitMaxs(1, 1);                mv.visitEnd();            }            cw.visitEnd();            return cw.toByteArray();        }        return classfileBuffer;    }}

Note that this is not standard AOP or proxy logic. The class file transformer is literally redefining the bytecode instructions incrementally in the form of byte arrays.

Static loading of a javaagent at startup

Static loading of a javaagent is done by using the -javaagent=/path/to/file.jar=options command line option to the java executable as below.

$ java -javaagent:target/javaagent-examples-jar-with-dependencies.jar=foobarbaz name.dhruba.javaagent.MyMainClass foo bar baz21:37:50.783 [main] INFO  name.dhruba.javaagent.MyJavaAgent - premain method invoked with args: foobarbaz and inst: sun.instrument.InstrumentationImpl@1786e6421:37:50.789 [main] INFO  n.d.javaagent.MyClassFileTransformer - class file transformer invoked for className: name/dhruba/javaagent/MyMainClass21:37:50.789 [main] INFO  name.dhruba.javaagent.MyMainClass - main method invoked with args: [foo, bar, baz]21:37:50.789 [main] INFO  n.d.javaagent.MyClassFileTransformer - class file transformer invoked for className: name/dhruba/user/MyUser21:37:50.800 [main] INFO  name.dhruba.javaagent.MyMainClass - userName: bar21:37:50.801 [DestroyJavaVM] INFO  n.d.javaagent.MyClassFileTransformer - class file transformer invoked for className: java/lang/Shutdown21:37:50.801 [DestroyJavaVM] INFO  n.d.javaagent.MyClassFileTransformer - class file transformer invoked for className: java/lang/Shutdown$Lock

Above, the javaagent lifecycle is clearly visible. The premain (and not the agentmain) method is invoked first. The class file transformer is passed classes as they are loaded. The transformer chooses to redefine the User object prior to its use. Subsequently the classes loaded at shutdown also pass through the transformer.

Dynamic loading of a javaagent at runtime

Dynamic loading of a javaagent at runtime can be done quite easily in a programmatic fashion but requires the sun tools jar to be present on the classpath. Certain libraries like jmockit have avoided this by opting to absorb the relevant classes from the sun tools jar into its library under the same package names. In Maven the tools jar can be added to the classpath very easily.

<dependency>  <groupId>com.sun</groupId>  <artifactId>tools</artifactId>  <version>1.6.0</version>  <scope>system</scope>  <systemPath>${java.home}/../lib/tools.jar</systemPath></dependency>

The sun tools jar api can then be used to load the java agent simply by provided the path to the jar file as follows.

package name.dhruba.javaagent;import java.lang.management.ManagementFactory;import org.slf4j.Logger;import org.slf4j.LoggerFactory;import com.sun.tools.attach.VirtualMachine;public class MyJavaAgentLoader {    static final Logger logger = LoggerFactory.getLogger(MyJavaAgentLoader.class);    private static final String jarFilePath = "/home/dhruba/.m2/repository/"            + "javaagent-examples/javaagent-examples/1.0/"            + "javaagent-examples-1.0-jar-with-dependencies.jar";    public static void loadAgent() {        logger.info("dynamically loading javaagent");        String nameOfRunningVM = ManagementFactory.getRuntimeMXBean().getName();        int p = nameOfRunningVM.indexOf(‘@‘);        String pid = nameOfRunningVM.substring(0, p);        try {            VirtualMachine vm = VirtualMachine.attach(pid);            vm.loadAgent(jarFilePath, "");            vm.detach();        } catch (Exception e) {            throw new RuntimeException(e);        }    }}

As you can see – the code above is querying the pid of the running process and attaching the javaagent to the process. If any kind of failure occurs at this point it is ignored silently whereas if an agent is loaded on startup failures result in termination of startup.

We can now initialise the java agent prior to invoking our main method using the MyJavaAgent.initialize() hookpoint we declared earlier.

package name.dhruba.javaagent;import java.util.Arrays;import name.dhruba.user.MyUser;import org.slf4j.Logger;import org.slf4j.LoggerFactory;public class MyMainClass {    static final Logger logger = LoggerFactory.getLogger(MyMainClass.class);    static {        MyJavaAgent.initialize();    }    /**     * Main method.     *      * @param args     */    public static void main(String[] args) {        logger.info("main method invoked with args: {}", Arrays.asList(args));        logger.info("userName: {}", new MyUser().getName());    }}

The output is very similar but with a subtely different path through the code.

20:58:50.923 [main] INFO  n.dhruba.javaagent.MyJavaAgentLoader - dynamically loading javaagent20:58:51.249 [Attach Listener] INFO  name.dhruba.javaagent.MyJavaAgent - agentmain method invoked with args:  and inst: sun.instrument.InstrumentationImpl@c2ff520:58:51.266 [main] INFO  name.dhruba.javaagent.MyMainClass - main method invoked with args: []20:58:51.267 [main] INFO  n.d.javaagent.MyClassFileTransformer - class file transformer invoked for className: name/dhruba/user/MyUser20:58:51.276 [main] INFO  name.dhruba.javaagent.MyMainClass - userName: bar20:58:51.276 [DestroyJavaVM] INFO  n.d.javaagent.MyClassFileTransformer - class file transformer invoked for className: java/lang/Shutdown20:58:51.276 [DestroyJavaVM] INFO  n.d.javaagent.MyClassFileTransformer - class file transformer invoked for className: java/lang/Shutdown$Lock

Note that this time the agentmain method was invoked instead of the premain method. The rest of the output is the same.

Conclusion

Javaagents along with the java.lang.instrument package are a powerful feature set of the Java language that allow you complete control over the classes of any given application. Instrumentation also has far more liberty with its capabilities than proxying or pointcut weaving. This gives complete dynamic capability to an otherwise very static language. It has allowed development of powerful non-intrusive tools like JMockit which immediately gains numerous advantages over other mocking tools largely because it is based on instrumentation. I look forward to further exploring the possibilities going forward.

Resources

ASM, ByteCodeOutline Plugin for Eclipse (Eclipse 3.5 update site), java.lang.instrument, JMockit.

Update [2/7/2011]: It’s nice to see this featured on StackOverflow.

Thanks to Anton Arhipov, Software Engineer at JRebel, for kindly linking to this post.