Understanding Android Gradle Build Process and Dex Generation: Practical Insights and Source Code Analysis

This article is based on the author’s practical experience with Android Gradle at work, providing a concise summary and sharing of Gradle-related knowledge.

It first explains the general Gradle workflow and implementation principles, supported by partial source code analysis. Topics include when project configuration data is accessed, the timing of task creation, and how to customize the compilation process.

Next, it focuses on the specific process of converting class files to dex files during compilation, discussing encountered problems and their solutions.

Main Workflow

Gradle build consists of three phases:

Initialization phase

Gradle reads the include information from settings.gradle in the root project, determines how many sub‑projects are included, creates a Project instance for each, and maps each build.gradle to a Project instance.

Configuration phase

Gradle reads each sub‑project’s build.gradle , configures the Gradle object, and builds the task dependency directed graph.

Execution phase

Based on the configuration information and the task dependency graph, Gradle executes the corresponding tasks.

Example code:

task hello {
    print 'hello'
}

Compared with:

task hello {
    doLast{
        print 'hello'
    }
}

A project contains several tasks, and each task contains several actions. The first example runs during the configuration phase, while the second runs only when the hello task is executed.

Hooks can be added at each phase. Gradle provides two listener interfaces:

BuildListener and TaskExecutionListener

BuildListener

void buildStarted(Gradle gradle);
void settingsEvaluated(Settings settings);
void projectsLoaded(Gradle gradle);
void projectsEvaluated(Gradle gradle);
void buildFinished(BuildResult result);

TaskExecutionListener

void beforeExecute(Task task);
void afterExecute(Task task, TaskState state);

Implement these interfaces and add them via gradle.addListener YourListener . This enables custom tasks, task replacement, dependency removal, or insertion (e.g., inserting task B between A and C: ...->A->C->... can be changed to B.dependsOn A.taskDependencies.getDependencies(A) A.dependsOn B ).

More tips can be found in the Gradle Userguide and Gradle Goodness Notebook.

Source Code Analysis

Key Gradle classes involved in Android builds include VariantManager , TaskManager , AndroidBuilder , and ConfigAction :

VariantManager collects variable data; basic configuration variables can be seen in AndroidSourceSet .

TaskManager manages task creation and execution.

AndroidBuilder executes specific Android build commands such as AIDL compilation, AAPT, and class‑to‑dex conversion.

ConfigAction defines the concrete behavior of a task; each task is assigned a ConfigAction that specifies its name, inputs, outputs, etc.

When the Gradle process starts, VariantManager gathers variantData , creates default AndroidTask instances, and invokes the appropriate TaskManager subclass (e.g., ApplicationTaskManager ) to call createTasksForVariantData and build the task dependency graph.

ThreadRecorder.get().record(ExecutionType.APP_TASK_MANAGER_CREATE_COMPILE_TASK,
    new Recorder.Block
() {
        @Override
        public Void call () {
            AndroidTask
javacTask = createJavacTask(tasks, variantScope);
            if (variantData.getVariantConfiguration().getUseJack()) {
                createJackTask(tasks, variantScope);
            } else {
                setJavaCompilerTask(javacTask, tasks, variantScope);
                createJarTask(tasks, variantScope);
                createPostCompilationTasks(tasks, variantScope);
            }
            return null;
        }
    });

The plugin used is com.android.application , which ultimately calls ApplicationTaskManager.createTasksForVariantData . Each created task receives a ConfigAction that defines its inputs, outputs, and associated AndroidBuilder tools.

Dex Process Analysis

In multi‑dex projects, the class‑to‑dex conversion (release build) involves several steps, illustrated by the following diagram:

Key tasks:

collectReleaseMultidexCompents (also called mainfestKeepListTask ) invokes CreateManifestKeepList to analyze AndroidManifest.xml and generate mainfest_keep.txt containing default entries such as activity, service, receiver, provider, instrumenter, etc.

shrinkReleaseMultidexCompents (aka proguardComponentsTask ) takes class.jar , shrinkAndroid.jar , mainfest_keep.txt , and produces componentClass.jar .

createReleaseMainDexClassList uses MainDexListBuilder to analyze entry‑point classes from componentClass.jar , filters the full class set, and generates maindexlist.txt .

retraceReleaseMainDexClassList de‑obfuscates maindexlist.txt to obtain an unobfuscated mapping file.

dexRelease calls the dx tool’s com.android.dx.command.Main.processAllFiles to produce the primary dex and secondary dex files based on maindexlist.txt and the complete class set.

Problems encountered:

When the number of methods/fields in the primary dex exceeds 65,536, the build fails. The dx class com.android.dx.command.dexer throws an exception when the limit is breached (see code snippet).

If maindexlist.txt is too large, the dx merger creates additional dex files, and the final step may still exceed the main dex capacity, causing a DexException .

if (args.multiDex && (outputDex.getClassDefs().items().size() > 0) &&
    ((maxMethodIdsInDex > args.maxNumberOfIdxPerDex) ||
     (maxFieldIdsInDex > args.maxNumberOfIdxPerDex))) {
    DexFile completeDex = outputDex;
    createDexFile();
    assert (completeDex.getMethodIds().items().size() <= numMethodIds +
            MAX_METHOD_ADDED_DURING_DEX_CREATION) &&
           (completeDex.getFieldIds().items().size() <= numFieldIds +
            MAX_FIELD_ADDED_DURING_DEX_CREATION);
}
...
if (dexOutputArrays.size() > 0) {
    throw new DexException("Too many classes in " + Arguments.MAIN_DEX_LIST_OPTION +
            ", main dex capacity exceeded");
}

Two mitigation strategies:

Reduce the number of entry classes by shrinking mainfest_keep.txt .

Directly control the size of maindexlist.txt .

Method 1 (early approach) modified the immutable KEEP_SPECS field of CreateManifestKeepList via reflection, but this stopped working after Gradle 2.0.0. The solution adopted was to write a custom task that parses AndroidManifest.xml , automatically adds required entry classes, generates a new maindest_keep.txt , and replaces the default keep file.

Method 2 involves writing a custom dependency‑analysis tool that builds maindexlist.txt itself. The default dx tool analyzes dependencies by traversing the constant pool of each class (including type, field, and method references) and recursively adding super‑classes and interfaces. Core code:

private void addDependencies(ConstantPool pool) {
    for (Constant constant : pool.getEntries())
        if (constant instanceof CstType) {
            checkDescriptor(((CstType) constant).getClassType());
        } else if (constant instanceof CstFieldRef) {
            checkDescriptor(((CstFieldRef) constant).getType());
        } else if (constant instanceof CstMethodRef) {
            Prototype proto = ((CstMethodRef) constant).getPrototype();
            checkDescriptor(proto.getReturnType());
            StdTypeList args = proto.getParameterTypes();
            for (int i = 0; i < args.size(); i++)
                checkDescriptor(args.get(i));
        }
}

private void checkDescriptor(Type type) {
    String descriptor = type.getDescriptor();
    if (descriptor.endsWith(";")) {
        int lastBrace = descriptor.lastIndexOf('[');
        if (lastBrace < 0) {
            addClassWithHierachy(descriptor.substring(1, descriptor.length() - 1));
        } else {
            assert ((descriptor.length() > lastBrace + 3) && (descriptor.charAt(lastBrace + 1) == 'L'));
            addClassWithHierachy(descriptor.substring(lastBrace + 2, descriptor.length() - 1));
        }
    }
}

private void addClassWithHierachy(String classBinaryName) {
    if (this.classNames.contains(classBinaryName)) {
        return;
    }
    try {
        DirectClassFile classFile = this.path.getClass(classBinaryName + ".class");
        this.classNames.add(classBinaryName);
        CstType superClass = classFile.getSuperclass();
        if (superClass != null) {
            addClassWithHierachy(superClass.getClassType().getClassName());
        }
        TypeList interfaceList = classFile.getInterfaces();
        int interfaceNumber = interfaceList.size();
        for (int i = 0; i < interfaceNumber; i++)
            addClassWithHierachy(interfaceList.getType(i).getClassName());
    } catch (FileNotFoundException e) {
    }
}

In the author’s project, asynchronous loading requires the primary dex’s dependency set to be complete; otherwise, NoClassDefError occurs when a class referenced from the main dex is not yet loaded.

Manual keep files can be added in build.gradle :

multiDexKeepProguard file('multiDexKeep.pro')

Because this manual approach is insufficient, a custom class‑dependency analysis tool was created, using ASM to add necessary inner and anonymous classes to the keep list, and a custom Gradle task replaces the default analysis.

public static void anylsisInnerClassDependence(Set
toKeep, String fullClassName, InputStream inputStream) throws IOException {
    if (!toKeep.contains(fullClassName)) {
        ClassReader classReader = new ClassReader(inputStream);
        ClassNode classNode = new ClassNode();
        classReader.accept(classNode, 0);
        for (int i = 0; i < classNode.innerClasses.size(); i++) {
            InnerClassNode innerClassNode = (InnerClassNode) classNode.innerClasses.get(i);
            String innerClassName = innerClassNode.name;
            if (PisaClassReferenceBuilder.isAnysisClass(innerClassName)) {
                PisaClassReferenceBuilder.getDefault().addClassWithHierachy(innerClassName);
                System.out.println("innerClassNode name=" + innerClassNode.name);
            }
        }
    }
}

Conclusion

This article shares practical insights and experiences with Gradle in Android development, offering a brief source‑code analysis of task creation and a detailed discussion of the dex generation process, aiming to demystify the Android Gradle build chain.