ASM instrumentation-multi-threaded operation monitoring

ASM instrumentation-multi-threaded operation monitoring

Recently, it is necessary to optimize the startup time of the App. The existing code has the following problems:

  1. Threads are not reused (using new Thread\HandlerThread), creating too many threads
  2. Use HandlerThread, not destroyed after use (Looper has been waiting), occupying memory
  3. Start thread early, but not used
  4. Some business parties initialize business code prematurely (although asynchronous), which affects startup time

Due to the above problems, it is necessary to scan the execution of the sub-thread and the main thread of the App from the cold start to the home page display.

The data to be monitored are as follows:

  1. Threads created, including the number and usage
  2. Execution time of functions such as runnable.run and AsyncTask.doInBackground executed

The number of threads created can be seen in the profiler->cpu->threads of Android Studio. However, as long as the profiler is turned on on my mobile phone, it is stuck and it is useless at all.

Based on the above requirements, ASM is used to instrument the thread code.

  1. Execution time: Calculating the execution time of run, doInBackground, etc. is very simple. You only need to add code before and at the end of these methods, and then you can calculate the time.
  2. Count the number of threads: Because Thread is a system code, it cannot be
    Thread.run
    Instrumentation in the method, but also unable to
    Thread.start
    Insert the code later, because in the case of a thread pool, it is also system code. It can only be instrumented in the business code. So consider inserting code in the following code:
  • Runnable.run
  • AsyncTask.doInBackground
  • Callable.call
  • Handler.handleMessage, Handler.Callback.handleMessage
  • Thread.run
  • TimerTask.run

In these methods are running in the thread, in these methods can be passed

Thread.currentThread()
Get the current thread data. This can cover most situations, because most of the threads are created only when there are tasks, and the time for using threads and creating threads is close. But there is a special case of HandlerThread, which can be created first, and then Looper waits until there is a task to execute. Therefore, if the looper has not been able to wait for the task, it will not be counted by the method just now, so this case needs to be handled specially. Do we have to scan line by line?
new HandlerThread
If the code is scanned, it should also be counted in the thread creation.

At this point, we have basically finished our thread instrumentation ideas. So how to instrument the stakes? It is to use the ASM library. His principle is to use Transform in Gradle to instrument the class file before the class file is packaged into the dex (you can check it yourself for details, and I will not go into details here). We need to use the gradle plug-in to instrument the code. How to develop the plug-in? There are two ways:

  1. In this project, create a buildSrc module (the module name is specifically used for plug-in development) for development.
  2. Independent engineering development

You can view the article for details .

This chapter adopts the first method, which creates buildSrc in the demo project. Then create a plug-in groovy file:

class ThreadInjectPlugin implements Plugin<Project> { @Override void apply(Project project) { def android if (project.plugins.hasPlugin(AppPlugin)) { android = project.extensions.getByType(AppExtension) } else { android = project.extensions.getByType(LibraryExtension) } //Handle runnable and other methods android.registerTransform(new ThreadRunTransform()) //Processing of new HandlerThread android.registerTransform(new HandlerThreadTransform()) } } Copy code

The focus is on these two Transforms. The first Transform is to scan to runnable, before the function (

onMethodEnter
) And the end of the function (
onMethodExit
) Insert code. The second Transform is dedicated to handling new HandlerThread.

First look at ThreadRunTransform:

class ThreadRunTransform extends Transform { @Override String getName() { return "ThreadTransform" } @Override Set<QualifiedContent.ContentType> getInputTypes() { return TransformManager.CONTENT_CLASS } @Override Set<? super QualifiedContent.Scope> getScopes() { return Sets.immutableEnumSet(QualifiedContent.Scope.PROJECT) } @Override boolean isIncremental() { return false } @Override void transform(TransformInvocation transformInvocation) throws TransformException, InterruptedException, IOException { super.transform(transformInvocation) transformInvocation.inputs.each {TransformInput input -> input.directoryInputs.each {DirectoryInput directoryInput -> transformDirectory(directoryInput, transformInvocation.outputProvider) } } } private void transformDirectory(DirectoryInput directoryInput, TransformOutputProvider outputProvider) { if (directoryInput.file.isDirectory()) { directoryInput.file.eachFileRecurse {File file -> def className = file.name def path = file.path if (isAppClass(className, path)) { try { FileInputStream fileInputStream = new FileInputStream(file.getAbsolutePath()) //-------------Key code, get the class code, scan through ClassVisitor, and insert the code ClassReader classReader = new ClassReader(fileInputStream) ClassWriter classWriter = new ClassWriter(classReader, ClassWriter.COMPUTE_MAXS) ClassVisitor visitor = new ThreadRunVisitor(className, classWriter) classReader.accept(visitor, ClassReader.EXPAND_FRAMES) byte[] code = classWriter.toByteArray(); FileOutputStream fos = new FileOutputStream(file.parentFile.absolutePath + File.separator + className) fos.write(code) fos.close() //------------End of key code------------------ } catch (Exception e) { } } } } def dest = outputProvider.getContentLocation(directoryInput.name, directoryInput.contentTypes, directoryInput.scopes, Format.DIRECTORY) FileUtils.copyDirectory(directoryInput.file, dest) } private boolean isAppClass(String className, String path) { //Check whether the class is a package under the app project, excluding third-party packages return className.endsWith(".class") && !className.contains("R\$") && !"R.class".equals(className) && !"BuildConfig.class".equals(className); } } Copy code
public class ThreadRunVisitor extends ClassVisitor { private String className; private boolean needInject; public ThreadRunVisitor(String className, ClassVisitor classVisitor) { super(Opcodes.ASM5, classVisitor); this.className = className; } @Override public void visit(int version, int access, String name, String signature, String superName, String[] interfaces) { //Determine whether you need to inject the method of this class this.needInject = isInjectClass(className, interfaces, superName); super.visit(version, access, name, signature, superName, interfaces); } @Override public MethodVisitor visitMethod(int access, String name, String descriptor, String signature, String[] exceptions) { MethodVisitor methodVisitor = this.cv.visitMethod(access, name, descriptor, signature, exceptions); boolean isInject = this.needInject && isInjectMethod(name, descriptor); if (!isInject) { return methodVisitor; } return (MethodVisitor) new AdviceAdapter(groovyjarjarasm.asm.Opcodes.ASM5, methodVisitor, access, name, descriptor) { @Override protected void onMethodEnter() { super.onMethodEnter(); //Insert the code you want to insert before the method (this code is in your app project) this.mv.visitMethodInsn(INVOKESTATIC, "com/example/project/AopUtil", "runStart", "()V", false); } @Override protected void onMethodExit(int opcode) { //Insert the code you want to insert at the end of the method (this code is in your app project) this.mv.visitMethodInsn(INVOKESTATIC, "com/example/project/AopUtil", "runEnd", "()V", false); } }; } public boolean isInjectClass(String className, String[] interfaces, String superName) { if (className == null) return false; //1, support runnable and android.os.Handler.Callback if (interfaces != null) { for (String inter: interfaces) { if ("java/lang/Runnable".equals(inter) || "android/os/Handler$Callback".equals(inter)) return true; } } //2, support ExtendsAsyncTask if ("android/os/AsyncTask".equals(superName)) { return true; } //3, support Handler.handleMessage if ("android/os/Handler".equals(superName)) { return true; } //4, support Thread.run if ("java/lang/Thread".equals(superName)) { return true; } return false; } public boolean isInjectMethod(String methodName, String methodDesc) { if (methodName == null || methodDesc == null) return false; //1, run method of runnable and thread if (methodName.equals("run") && methodDesc.equals("()V")) return true; //2, extendedAsyncTask.doInBackground method if (methodName.equals("doInBackground")) return true; //3, handleMessage method of handler and callback if (methodName.equals("handleMessage")) { return methodDesc.equals("(Landroid/os/Message;)V") || methodDesc.equals("(Landroid/os/Message;)Z"); } return false; } } Copy code

To explain here, in the following code, new Runnable will be compiled as an internal class. During code scanning, two classes will be created, Test.class and Test$1.class, and they will be scanned separately:

public class Test{ void test(){ new Thread(new Runnable(){ @override public void run(){ //xxxx } }).start(); } } Copy code

Look at the second Transform, because it is not clear which method will exist

new HandlerThread
, Can t match according to method name and desc like the first Transform, only scan line by line. Therefore, you need to use ClassNode to scan, get the list of methods of this class, and then get the instructions of each method (instructions that are executed after being compiled), and analyze whether there is this statement in the instruction.

The code of HandlerThreadTransform is similar to the code of ThreadRunTransform. Look at the key code part:

private void transformDirectory(DirectoryInput directoryInput, TransformOutputProvider outputProvider) { if (directoryInput.file.isDirectory()) { directoryInput.file.eachFileRecurse {File file -> def className = file.name def path = file.path if (isAppClass(className, path)) { try { FileInputStream fileInputStream = new FileInputStream(file.getAbsolutePath()) ClassReader classReader = new ClassReader(fileInputStream) ClassWriter classWriter = new ClassWriter(classReader, ClassWriter.COMPUTE_MAXS) ClassVisitor visitor = new HandlerThreadVisitor(classReader,classWriter) classReader.accept(visitor, ClassReader.EXPAND_FRAMES) byte[] code = classWriter.toByteArray(); FileOutputStream fos = new FileOutputStream(file.parentFile.absolutePath + File.separator + className) fos.write(code) fos.close() } catch (Exception e) { e.printStackTrace() } } } } def dest = outputProvider.getContentLocation(directoryInput.name, directoryInput.contentTypes, directoryInput.scopes, Format.DIRECTORY) FileUtils.copyDirectory(directoryInput.file, dest) } Copy code

First of all, in the visit method of HandlerThreadVisitor, you need to scan the list of methods that need to be instrumented in the class. How to judge whether this method needs to be instrumented? When passed

ClassNode
When parsing a class file, you can get the method list of the class through classNode.methods. Traverse the list and get
MethodNode
, Take out the instructions that can be executed by the JVM after each method is compiled (instructions,
AbstractInsnNode
). In each AbstractInsnNode, there will be an int value: opcode (the specific value is in
org.objectweb.asm.Opcodes
In), this instruction can explain the current execution content. for example:

The opcode value of the loaded variable: Opcodes.ILOAD (load int type variable) = 21 Opcodes.LLOAD (load long type variable) = 22 Opcodes.FLOAD (load variables of type float) = 23 Opcodes.DLOAD (load variable of type double) = 24 Opcodes.ALOAD (load reference type variables) = 25 Assign a value to a certain type of variable Opcodes.ISTORE = 54, LSTORE = 55, FSTORE = 56, DSTORE = 57, ASTORE = 58 Call a method of a class: Opcodes.INVOKESTATIC (call static method) = 184 Opcodes.INVOKEVIRTUAL (method of calling instance, non-static)=182 Opcodes.INVOKESPECIAL (call instance construction method, non-static)=183 Opcodes.INVOKEDYNAMIC (lambda desugar method, will be explained later)=186 Create variable Opcodes.NEW (load the constructor of a certain class) Copy code

Give an example of code:

public void test(){ HandlerThread ht = new HandlerThread("xxx"); ht.start(); } The above code will be compiled into the following instructions //new handlerThread("xxx") is translated into the following instructions TypeInsnNode(opcodes:187, desc:android/os/HandlerThread) LdcInsnNode(opcodes:18, cst:xx) ----load constant MethodInsnNode(opcodes:183, owner:android/os/HandlerThread, name:<init>, desc:(Ljava/lang/String;)V) --call the construction method VarInsnNode(opcodes:58, var:1) ---- assign the object created by the above construction method to the second variable (the first is this of the class, var is in the order of variable creation, if the method has parameters, Will be ranked after this position) //ht.start() is translated into the following instructions VarInsnNode(opcodes:25, var:1) ----load the second variable, the thread variable MethodInsnNode(opcodes:182, owner:android/os/HandlerThread, name:start, desc:()V) ----call the start method of loading variables. Copy code

The effect of the monitoring effect code we want is as follows:

public void test(){ HandlerThread ht = new HandlerThread("xx"); ht.start(); AopUtil.addThread(ht);//Insert our own monitoring code Copy code

Because Thread is being initialized, its thread id and thread name have been determined. Therefore, when we detect that the thread.start method is executed, we can add the following instructions after it:

VarInsnNode(opcodes:25, var:1) ----load thread variable MethodInsnNode(opcodes:184, owner:com/example/project/AopUtil, name:addThread, desc:(Ljava/lang/Thread;)V) Copy code

Of course, we have to remember which variable the compiler loads before calling the start method, that is, remember the value of VarInsnNode.var when VarInsnNode.opcodes is 25 (OpCodes.ALOAD), so that we can load this variable later to call us Method of instrumentation.

So the problem is, sometimes the business side code is written like this:

new HandlerThread("xxx").start(); This code is translated into instructions as follows: TypeInsnNode(opcodes:187, desc:android/os/HandlerThread) LdcInsnNode(opcodes:18, cst:xx) ----load constant MethodInsnNode(opcodes:183, owner:android/os/HandlerThread, name:<init>, desc:(Ljava/lang/String;)V) --call the construction method MethodInsnNode(opcodes:182, owner:android/os/HandlerThread, name:start, desc:()V) ----call the start method of loading variables. The only difference from just now is that this instruction is missing Opcodes.ASTORE and Opcodes.ALOAD Copy code

At this time, there is no Thread variable in this method, so we need to add instructions. Before the start instruction, add instructions for creating variables (newLocal), storing objects (Opcodes.ASTORE), and reading variables (Opcodes.ALOAD). Therefore, we must remember the instruction before the start method. If the instruction is directly the instruction to call the init construction method, we need to add the instruction just mentioned. If the ALOAD instruction is called before the start method, then we only need to remember ALOAD The var parameter in the instruction is fine.

Take a look at the core code (all the code later):

@Override public MethodVisitor visitMethod(int access, String name, String descriptor, String signature, String[] exceptions) { MethodVisitor methodVisitor = this.cv.visitMethod(access, name, descriptor, signature, exceptions); boolean injectLambda = hasLambda(name); boolean isInject = injectMethods.contains(new Method(name, descriptor)); if (!isInject && !injectLambda) { return methodVisitor; } return new AdviceAdapter(groovyjarjarasm.asm.Opcodes.ASM5, methodVisitor, access, name, descriptor) { int lastThreadVarIndex = -1;//Remember the position of the thread variable String lastThreadInstruction;//The last instruction to execute thread @Override public void visitVarInsn(int opcode, int var) { super.visitVarInsn(opcode, var); if(isInject) { if (opcode == ALOAD) { lastThreadInstruction = VISIT_VAR_INSN_LOAD; lastThreadVarIndex = var; } } } @Override public void visitMethodInsn(int opcode, String owner, String name, String descriptor, boolean isInterface) { if (isInject) { if (!THREAD.equals(owner) && !HANDLER_THREAD.equals(owner)) { super.visitMethodInsn(opcode, owner, name, descriptor, isInterface); return; } if (!"<init>".equals(name) && !"start".equals(name)) { super.visitMethodInsn(opcode, owner, name, descriptor, isInterface); return; } //If you go to thread.start or handler thread.start method if ("<init>".equals(name)) { super.visitMethodInsn(opcode, owner, name, descriptor, isInterface); lastThreadInstruction = VISIT_METHOD_THREAD_INIT; } else if ("start".equals(name)) { //First check whether the thread is stored as a local variable if (lastThreadInstruction.equals(VISIT_METHOD_THREAD_INIT)) { //If the last sentence of start is init, it means that thread is not stored as a local variable, then create a local variable Type threadType = Type.getObjectType("java/lang/Thread"); lastThreadVarIndex = newLocal(threadType); this.mv.visitVarInsn(ASTORE, lastThreadVarIndex); this.mv.visitVarInsn(ALOAD, lastThreadVarIndex); } //Continue to call the start method super.visitMethodInsn(opcode, owner, name, descriptor, isInterface); if (lastThreadVarIndex> 0) { //Get the last thread variable this.mv.visitVarInsn(ALOAD, lastThreadVarIndex); //Get the thread id value //this.mv.visitMethodInsn(INVOKEVIRTUAL, owner, "getId", "()J", false); this.mv.visitMethodInsn(INVOKESTATIC, "com/example/project/AopUtil", "addThread", "(Ljava/lang/Thread;)V", false); } } } else { super.visitMethodInsn(opcode, owner, name, descriptor, isInterface); } } }; } Copy code

After talking about this, most of the situations have been achieved, and then the ASM will deal with lambda expressions.

class Java8 { interface Logger { void log(String s); } public static void main(String... args) { sayHi(s -> System.out.println(s)); } private static void sayHi(Logger logger) { logger.log("Hello!"); } } The above code becomes: public class Java8 { interface Logger { void log(String s); } public static void main(String... args) { //Use Logger's implementation class Java8$1 here sayHi(s -> new Java8$1()); } private static void sayHi(Logger logger) { logger.log("Hello!"); } //The content in the method body is moved here static void lambda$main$0(String str){ System.out.println(str); } } public class Java8$1 implements Java8.Logger { public Java8$1(){ } @Override public void log(String s) { //Here to call the static method of the Java8 method Java8.lambda$main$0(s); } } Copy code

In the main function, there will be an Opcodes.INVOKEDYNAMIC instruction (InvokeDynamicInsnNode), check the parameters in the instruction:

First of all, we judge whether the desc in the instruction contains java/lang/Runnable and the name is run. If the match is successful, then get the real execution function (bsmArgs[1]) of the method after being desugared, and add it to the function We instrumented the code. View specific code:

public class HandlerThreadVisitor extends ClassVisitor { public static final String HANDLER_THREAD = "android/os/HandlerThread"; public static final String THREAD = "java/lang/Thread"; private final String VISIT_VAR_INSN_LOAD = "visitVarInsn-Load"; private final String VISIT_METHOD_THREAD_INIT = "visitMethod-ThreadInit"; private ClassNode classnode; ArrayList<Method> injectMethods = new ArrayList<>(); ArrayList<String> lambdaMethods = new ArrayList<>(); public HandlerThreadVisitor(ClassReader classReader, ClassVisitor classVisitor) { super(Opcodes.ASM5, classVisitor); classnode = new ClassNode(); classReader.accept(classnode, 0); } @Override public void visit(int version, int access, String name, String signature, String superName, String[] interfaces) { getInjectMethods(classnode); super.visit(version, access, name, signature, superName, interfaces); } @Override public MethodVisitor visitMethod(int access, String name, String descriptor, String signature, String[] exceptions) { MethodVisitor methodVisitor = this.cv.visitMethod(access, name, descriptor, signature, exceptions); boolean injectLambda = hasLambda(name); boolean isInject = injectMethods.contains(new Method(name, descriptor)); if (!isInject && !injectLambda) { return methodVisitor; } return new AdviceAdapter(groovyjarjarasm.asm.Opcodes.ASM5, methodVisitor, access, name, descriptor) { int lastThreadVarIndex = -1; String lastThreadInstruction; @Override protected void onMethodEnter() { super.onMethodEnter(); if (injectLambda) { this.mv.visitMethodInsn(INVOKESTATIC, "com/example/project/AopUtil", "runStart", "()V", false); } } @Override protected void onMethodExit(int opcode) { super.onMethodExit(opcode); if (injectLambda) { this.mv.visitMethodInsn(INVOKESTATIC, "com/example/project/AopUtil", "runEnd", "()V", false); } } @Override public void visitVarInsn(int opcode, int var) { super.visitVarInsn(opcode, var); if(isInject) { if (opcode == ALOAD) { lastThreadInstruction = VISIT_VAR_INSN_LOAD; lastThreadVarIndex = var; } } } @Override public void visitMethodInsn(int opcode, String owner, String name, String descriptor, boolean isInterface) { if (isInject) { if (!THREAD.equals(owner) && !HANDLER_THREAD.equals(owner)) { super.visitMethodInsn(opcode, owner, name, descriptor, isInterface); return; } if (!"<init>".equals(name) && !"start".equals(name)) { super.visitMethodInsn(opcode, owner, name, descriptor, isInterface); return; } //If you go to thread.start or handler thread.start method if ("<init>".equals(name)) { super.visitMethodInsn(opcode, owner, name, descriptor, isInterface); lastThreadInstruction = VISIT_METHOD_THREAD_INIT; } else if ("start".equals(name)) { //First check whether the thread is stored as a local variable if (lastThreadInstruction.equals(VISIT_METHOD_THREAD_INIT)) { //If the last sentence of start is init, it means that thread is not stored as a local variable, then create a local variable Type threadType = Type.getObjectType("java/lang/Thread"); lastThreadVarIndex = newLocal(threadType); this.mv.visitVarInsn(ASTORE, lastThreadVarIndex); this.mv.visitVarInsn(ALOAD, lastThreadVarIndex); } //Continue to call the start method super.visitMethodInsn(opcode, owner, name, descriptor, isInterface); if (lastThreadVarIndex> 0) { //Get the last thread variable this.mv.visitVarInsn(ALOAD, lastThreadVarIndex); this.mv.visitMethodInsn(INVOKESTATIC, "com/example/project/AopUtil", "addThread", "(Ljava/lang/Thread;)V", false); } } } else { super.visitMethodInsn(opcode, owner, name, descriptor, isInterface); } } }; } public void getInjectMethods(ClassNode classnode) { for (MethodNode method: classnode.methods) { for (int i = 0; i <method.instructions.size(); i++) { AbstractInsnNode insnNode = method.instructions.get(i); if (insnNode.getOpcode() == Opcodes.NEW) { TypeInsnNode methodInsnNode = (TypeInsnNode) insnNode; if (HANDLER_THREAD.equals(methodInsnNode.desc) || THREAD.equals(methodInsnNode.desc)) { injectMethods.add(new Method(method.name, method.desc)); } } else if (insnNode instanceof InvokeDynamicInsnNode) { //Determine whether it is a runnable lambda expression if (((InvokeDynamicInsnNode) insnNode).desc.contains("Ljava/lang/Runnable;") && ((InvokeDynamicInsnNode) insnNode).name.equals("run")) { lambdaMethods.add(((Handle) ((InvokeDynamicInsnNode) insnNode).bsmArgs[1]).getName()); } } } } } private boolean hasLambda(String name){ for (int i = 0; i <lambdaMethods.size(); i++) { if (lambdaMethods.get(i).contains(name)) { return true; } } return false; } static class Method { String name; String desc; public Method(String name, String desc) { this.name = name; this.desc = desc; } @Override public boolean equals(Object o) { Method temp = (Method) o; return name.equals(temp.name) && desc.equals(temp.desc); } } } Copy code

So far all the code has been explained. Take a look at the code in our AopUtils:

public class AopUtil { static HashSet<Long> allThread = new HashSet<>(); static HashSet<Long> usedThread = new HashSet<>(); static ConcurrentHashMap<String, Long> threadRunStartTime = new ConcurrentHashMap<>(); public static void runStart() { logThreadUsage(Thread.currentThread(), true); threadRunStartTime.put(getKey(), System.currentTimeMillis()); } private static String getKey(){ String stackTrace = Log.getStackTraceString(new Throwable()); stackTrace = stackTrace.split("\n\t")[3];//Get the first few lines to execute the run function and store them as the key return stackTrace.substring(0,stackTrace.indexOf("(")); } public static void runEnd() { String key = getKey(); Long start = threadRunStartTime.get(key); if (start != null) { Log.d("ThreadAop-runCost", key + "cost time:" + (System.currentTimeMillis()-start)); threadRunStartTime.remove(key); } } public static void addThread(Thread thread) { logThreadUsage(thread, false); } private static void logThreadUsage(Thread thread, boolean isFromRun) { if (thread.getName().equals("main")) { return; } synchronized (AopUtil.class) { if (usedThread == null) { usedThread = new HashSet<>(); } if (isFromRun) { Log.d("ThreadAop-used1", "thread is used: "+ thread.getId() + ", name is" + thread.getName()); usedThread.add(thread.getId()); } if (allThread == null) { allThread = new HashSet<>(); } if (allThread.contains(thread.getId())) { Log.d("ThreadAop", Log.getStackTraceString(new Throwable())); return; } allThread.add(thread.getId()); Log.d("ThreadAop-1", "current size:" + allThread.size() + "add new thread:" + thread.getName() + ", usedThread:" + usedThread.size()); Log.d("ThreadAop", Log.getStackTraceString(new Throwable())); } } } Copy code

Now that the plug-in has been developed, we will introduce in our project:

1. add the following files in the buildSrc module:

Then add the following code to build.gradle in the app module:

plugins { id'thread-inject' } Or apply plugin:'thread-inject' Copy code

After packaging and running the app, you can see the log output of our instrumented code. So far the code is finished.