Android

When it comes to building and managing projects, Gradle has become a popular choice among developers due to its flexibility, extensibility, and efficiency. One of the key aspects of Gradle’s functionality lies in how it organizes and utilizes directories and files within a project. In this blog post, we will take an in-depth look at the directories and files Gradle uses, understanding their purposes and significance in the build process.

Project Structure

Before diving into the specifics of directories and files, let’s briefly discuss the typical structure of a Gradle project. Gradle projects are structured in a way that allows for clear separation of source code, resources, configuration files, and build artifacts. The most common structure includes directories such as:

Project Root
├── build.gradle.kts (build.gradle)
├── settings.gradle.kts (settings.gradle)
├── gradle.properties
├── gradlew (Unix-like systems)
├── gradlew.bat (Windows)
├── gradle
│   └── wrapper
│       └── gradle-wrapper.properties
├── src
│   ├── main
│   │   ├── java
│   │   ├── resources
│   │   └── ...
│   └── test
│       ├── java
│       ├── resources
│       └── ...
└── build
    ├── ...
    ├── outputs
    └── ...

• src: This directory contains the source code and resources for your project. It’s usually divided into subdirectories like main and test, each containing corresponding code and resources. The main directory holds the main application code, while the test directory contains unit tests.
• build: Gradle generates build artifacts in this directory. This includes compiled code, JARs, test reports, and other artifacts resulting from the build process. The build directory is typically temporary and gets regenerated each time you build the project.
• gradle: This directory contains Gradle-specific files and configurations. It includes the wrapper subdirectory, which holds the Gradle Wrapper files. The Gradle Wrapper is a script that allows you to use a specific version of Gradle without installing it globally on your system.

Directories

Gradle relies on two main directories: the Gradle User Home directory and the Project root directory. Let’s explore what’s inside each of them and how temporary files and directories are cleaned up.

Gradle User Home directory

The Gradle User Home (usually found at <home directory of the current user>/.gradle) is like a special storage area for Gradle. It keeps important settings, like configuration, initialization scripts as well as caches and logs, safe and organized.

├── caches   // 1
│   ├── 4.8  // 2
│   ├── 4.9  // 2
│   ├── ⋮
│   ├── jars-3 // 3
│   └── modules-2 // 3
├── daemon   // 4
│   ├── ⋮
│   ├── 4.8
│   └── 4.9
├── init.d   // 5
│   └── my-setup.gradle
├── jdks     // 6
│   ├── ⋮
│   └── jdk-14.0.2+12
├── wrapper
│   └── dists   // 7
│       ├── ⋮
│       ├── gradle-4.8-bin
│       ├── gradle-4.9-all
│       └── gradle-4.9-bin
└── gradle.properties   // 8 

1. Global cache directory (for everything that’s not project-specific): This directory stores the results of tasks that are not specific to any particular project. This includes things like the results of downloading dependencies and the results of compiling code. The default location of this directory is $USER_HOME/.gradle/caches.

2. Version-specific caches (e.g. to support incremental builds): This directory stores the results of tasks that are specific to a particular version of Gradle. This includes things like the results of parsing the project’s build script and the results of configuring the project’s dependencies. The default location of this directory is $USER_HOME/.gradle/<gradle-version>/caches.

3. Shared caches (e.g. for artifacts of dependencies): This directory stores the results of tasks that are shared by multiple projects. This includes things like the results of downloading dependencies and the results of compiling code. The default location of this directory is $USER_HOME/.gradle/shared/caches.

4. Registry and logs of the Gradle Daemon (the daemon is a long-running process that can be used to speed up builds): This directory stores the registry of the Gradle Daemon and the logs of the Gradle Daemon. The default location of this directory is $USER_HOME/.gradle/daemon.

5. Global initialization scripts (scripts that are executed before any build starts): This directory stores the global initialization scripts. The default location of this directory is $USER_HOME/.gradle/init.d.

6. JDKs downloaded by the toolchain support: This directory stores the JDKs that are downloaded by the toolchain support. The toolchain support is used to compile code for different platforms. The default location of this directory is $USER_HOME/.gradle/toolchains.

7. Distributions downloaded by the Gradle Wrapper: This directory stores the distributions that are downloaded by the Gradle Wrapper. The Gradle Wrapper is a script that can be used to simplify the installation and execution of Gradle. The default location of this directory is $USER_HOME/.gradle/wrapper.

8. Global Gradle configuration properties (properties that are used by all Gradle builds): This directory stores the global Gradle configuration properties. The default location of this directory is $USER_HOME/.gradle/gradle.properties.

Cleaning Up Caches and Distributions

When you use Gradle for building projects, it creates temporary files and data in your computer’s user home directory. Gradle automatically cleans up these files to free up space. Here’s how it works:

Background Cleanup

Gradle cleans up in the background when you stop the Gradle tool (daemon). If you don’t use the background cleanup, it happens after each build with a progress bar.

For example, imagine you’re working on a software project using Gradle for building. After you finish your work and close the Gradle tool, it automatically cleans up any temporary files it created. This ensures that your computer doesn’t get cluttered with unnecessary files over time. It’s like cleaning up your workspace after you’re done with a task.

Cleaning Strategies

In a software project, you often use different versions of Gradle. Gradle keeps some files specific to each version. If a version hasn’t been used for a while, these files are removed to save space. This is similar to getting rid of old documents or files you no longer need. For instance, if you’re not using a particular version of a library anymore, Gradle will clean up the related files.

Gradle has different ways to clean up:

• Version-specific Caches: These are files for specific versions of Gradle. If they’re not used, Gradle deletes release version files after 30 days of inactivity and snapshot version files after 7 days of inactivity.
• Shared Caches: These are files used by multiple versions of Gradle. If no Gradle version needs them, they’re deleted.
• Files for Current Gradle Version: Files for the version of Gradle you’re using are checked. Depending on if they can be made again or need to be downloaded, they’re deleted after 7 or 30 days of not being used.
• Unused Distributions: If a distribution of Gradle isn’t used, it’s removed.

Configuring Cleanup

Think about a project where you frequently switch between different Gradle versions. You can decide how long Gradle keeps files before cleaning them up. For example, if you want to keep the files of the released versions for 45 days and the files of the snapshots (unstable versions) for 10 days, you can adjust these settings. It’s like deciding how long you want to keep your emails before they are automatically deleted.

You can set how long Gradle keeps these files:

• Released Versions: 30 days for released versions.
• Snapshot Versions: 7 days for snapshot versions.
• Downloaded Resources: 30 days for resources from the internet.
• Created Resources: 7 days for resources Gradle makes.

How to Configure

You can change these settings in a file called “cache-settings.gradle.kts” in your Gradle User Home directory. Here’s an example of how you can do it:

beforeSettings {
    caches {
        releasedWrappers.setRemoveUnusedEntriesAfterDays(45)
        snapshotWrappers.setRemoveUnusedEntriesAfterDays(10)
        downloadedResources.setRemoveUnusedEntriesAfterDays(45)
        createdResources.setRemoveUnusedEntriesAfterDays(10)
    }
}

Here,

1. beforeSettings: This is a Gradle lifecycle event that allows you to execute certain actions before the settings of your build script are applied.
2. caches: This part refers to the caches configuration within the beforeSettings block.
3. releasedWrappers.setRemoveUnusedEntriesAfterDays(45): This line sets the retention period for released versions and their related caches to 45 days. It means that if a released version of Gradle or its cache files haven”t been used for 45 days, they will be removed during cleanup.
4. snapshotWrappers.setRemoveUnusedEntriesAfterDays(10): This line sets the retention period for snapshot versions (unstable, in-development versions) and their related caches to 10 days. If they haven’t been used for 10 days, they will be removed during cleanup.
5. downloadedResources.setRemoveUnusedEntriesAfterDays(45): This line sets the retention period for resources downloaded from remote repositories (e.g., cached dependencies) to 45 days. If these resources haven’t been used for 45 days, they will be removed.
6. createdResources.setRemoveUnusedEntriesAfterDays(10): This line sets the retention period for resources created by Gradle during the build process (e.g., artifact transformations) to 10 days. If these resources haven’t been used for 10 days, they will be removed.

In essence, this code configures how long different types of files should be retained before Gradle’s automatic cleanup process removes them. The numbers you see (45, 10) represent the number of days of inactivity after which the files will be considered for cleanup. You can adjust these numbers based on your project’s needs and your preferred cleanup frequency.

Cleaning Frequency

You can choose how often cleanup happens:

• DEFAULT: Happens every 24 hours.
• DISABLED: Never cleans up (useful for specific cases).
• ALWAYS: Cleans up after each build (useful but can be slow).

Sometimes you might want to control when the cleanup happens. If you choose the “DEFAULT” option, It will automatically clean up every 24 hours in the background. However, if you have limited storage and need to manage space carefully, you might choose the “ALWAYS” option. This way, cleanup occurs after each build, ensuring that space is cleared right away. This can be compared to deciding whether to clean your room every day (DEFAULT) or cleaning it immediately after a project (ALWAYS).

Disabling Cleanup

Here’s how you can disable cleanup:

beforeSettings {
    caches {
        cleanup.set(Cleanup.DISABLED)
    }
}

Above I mentioned “useful for specific cases,” I meant that the option to disable cleanup (CLEANUP.DISABLED) might be helpful in certain situations where you have a specific reason to avoid cleaning up the temporary files and data created by it.

For example, imagine you’re working on a project where you need to keep these temporary files for a longer time because you frequently switch between different builds or versions. In this scenario, you might want to delay the cleanup process until a later time when it’s more convenient for you, rather than having Gradle automatically clean up these files.

So, “useful for specific cases” means there are situations where you might want to keep the temporary files around for a longer duration due to your project’s requirements or your workflow.

Remember, you can only change these settings using specific files in your Gradle User Home directory. This helps prevent different projects from conflicting with each other’s settings.

Sharing a Gradle User Home Directory between Multiple Gradle Versions

Sharing a single Gradle User Home among various Gradle versions is a common practice. In this shared home, there are caches that belong to specific versions of Gradle. Each Gradle version usually manages its own caches.

However, there are some caches that are used by multiple Gradle versions, like the cache for dependency artifacts or the artifact transform cache. Starting from version 8.0, you can adjust settings to control how long these caches are kept. But in older versions, the retention periods are fixed (either 7 or 30 days depending on the cache).

This situation can lead to a scenario where different versions might have different settings for how long cache artifacts are retained. As a result, shared caches could be accessed by various versions with different retention settings.

This means that:

• If you don’t customize the retention period, all versions of Gradle that do cleanup will follow the same retention periods. This means that sharing a Gradle User Home among multiple versions won’t cause any issues in this case. The cleanup behavior will be consistent across all versions.
• If you set a custom retention period for Gradle versions equal to or greater than 8.0, making it shorter than the older fixed periods, it won’t cause any issues. The newer versions will clean up their artifacts sooner than the old fixed periods. However, the older versions won’t be aware of these custom settings, so they won’t participate in the cleanup of shared caches. This means the cleanup behavior might not be consistent across all versions.
• If you set a custom retention period for Gradle versions equal to or greater than 8.0, now making it longer than the older fixed periods, there could be an issue. The older versions might clean the shared caches sooner than your custom settings. If you want the newer versions to keep the shared cache entries for a longer period, they can’t share the same Gradle User Home with the older versions. Instead, they should use a separate directory to ensure the desired retention periods are maintained.

When sharing the Gradle User Home with Gradle versions before 8.0, there’s another thing to keep in mind. In older versions, the DSL elements used to set cache retention settings aren’t available. So, if you’re using a shared init script among different versions, you need to consider this.

//gradleUserHome/init.d/cache-settings.gradle.kts

if (GradleVersion.current() >= GradleVersion.version("8.0")) {
    apply(from = "gradle8/cache-settings.gradle.kts")
}

//gradleUserHome/init.d/gradle8/cache-settings.gradle.kts

beforeSettings {
    caches {
        releasedWrappers { setRemoveUnusedEntriesAfterDays(45) }
        snapshotWrappers { setRemoveUnusedEntriesAfterDays(10) }
        downloadedResources { setRemoveUnusedEntriesAfterDays(45) }
        createdResources { setRemoveUnusedEntriesAfterDays(10) }
    }
}

To handle this, you can apply a script that matches the version requirements. Make sure this version-specific script is stored outside the init.d directory, perhaps in a sub-directory. This way, it won’t be automatically applied, and you can ensure that the right settings are used for each Gradle version.

Cache marking

Starting from Gradle version 8.1, a new feature is available. Gradle now lets you mark caches using a file called CACHEDIR.TAG, following the format defined in the Cache Directory Tagging Specification. This file serves a specific purpose: it helps tools recognize directories that don’t require searching or backing up.

By default, in the Gradle User Home, several directories are already marked with this file: caches, wrapper/dists, daemon, and jdks. This means these directories are identified as ones that don’t need to be extensively searched or included in backups.

Here is a sample CACHEDIR.TAG file:

# This file is a cache tag file, created by Gradle version 8.1.
# It identifies the directory `caches` as a Gradle cache directory.

name = caches
version = 8.1
signature = sha256:<signature>

The name field specifies the name of the directory that is being tagged. In this case, the directory is caches.

The version field specifies the version of Gradle that created the tag. In this case, the version is 8.1.

The signature field is a signature that can be used to verify the authenticity of the tag. This signature is created using a cryptographic hash function.

The CACHEDIR.TAG file is a simple text file, so you can create it using any text editor. However, it is important to make sure that the file is created with the correct permissions. The file should have the following permissions:

-rw-r--r--          

This means that the file is readable by everyone, but only writable by the owner.

Configuring cache marking

The cache marking feature can be configured via an init script in the Gradle User Home:

//gradleUserHome/init.d/cache-settings.gradle.kts

beforeSettings {
    caches {
        // Disable cache marking for all caches
        markingStrategy.set(MarkingStrategy.NONE)
    }
}

Note that cache marking settings can only be configured via init scripts and should be placed under the init.d directory in the Gradle User Home. This is because the init.d directory is loaded before any other scripts, so the cache marking settings will be applied to all projects that use the Gradle User Home.

This also limits the possibility of different conflicting settings from different projects being applied to the same directory. If the cache marking settings were not coupled to the Gradle User Home, then it would be possible for different projects to apply different settings to the same directory. This could lead to confusion and errors.

Project Root Directory

The project root directory holds all the source files for your project. It also includes files and folders created by Gradle, like .gradle and build. While source files are typically added to version control, the ones created by Gradle are temporary and used to enable features like incremental builds. A typical project root directory structure looks something like this:

├── .gradle    // 1     (Folder for caches)
│   ├── 4.8    // 2 
│   ├── 4.9    // 2
│   └── ⋮
├── build      // 3     (Generated build files)
├── gradle              // (Folder for Gradle tools)
│   └── wrapper   // 4     (Wrapper configuration)
├── gradle.properties   // 5  (Project properties)
├── gradlew   // 6          (Script to run Gradle on Unix-like systems)
├── gradlew.bat   // 6      (Script to run Gradle on Windows)
├── settings.gradle or settings.gradle.kts  // 7 (Project settings)
├── subproject-one   // 8                     (Subproject folder)
|   └── build.gradle or build.gradle.kts   // 9 (Build script for subproject)
├── subproject-two   // 8                       (Another subproject folder)
|   └── build.gradle or build.gradle.kts   // 9 (Build script for another subproject)
└── ⋮                                        // (And more subprojects)

1. Project-specific cache directory generated by Gradle: This is a folder where Gradle stores temporary files and data that it uses to speed up building projects. It’s specific to your project and helps Gradle avoid redoing certain tasks each time you build, which can save time.
2. Version-specific caches (e.g. to support incremental builds): These caches are used to remember previous build information, allowing Gradle to only rebuild parts of your project that have changed. This is especially helpful for “incremental builds” where you make small changes and don’t want to redo everything.
3. The build directory of this project into which Gradle generates all build artifacts: When you build your project using Gradle, it generates various files and outputs. This “build directory” is where Gradle puts all of those created files like compiled code, libraries, and other artifacts.
4. Contains the JAR file and configuration of the Gradle Wrapper: The JAR file is a packaged software component. Here, it refers to the Gradle Wrapper’s JAR file, which allows you to use Gradle without installing it separately. The configuration helps the Wrapper know how to work with Gradle.
5. Project-specific Gradle configuration properties: These are settings that are specific to your project and control how Gradle behaves when building. For example, they might determine which plugins to use or how to package your project.
6. Scripts for executing builds using the Gradle Wrapper: The gradlew and gradlew.bat scripts are used to execute builds using the Gradle Wrapper. These scripts are special commands that let you run Gradle tasks without needing to have Gradle installed globally on your system.
7. The project’s settings file where the list of subprojects is defined: This file defines how your project is structured, including the list of smaller “subprojects” that make up the whole. It helps Gradle understand the layout of your project.
8. Usually a project is organized into one or multiple subprojects: A project can be split into smaller pieces called subprojects. This is useful for organizing complex projects into manageable parts, each with its own set of tasks.
9. Each subproject has its own Gradle build script: Each subproject within your project has its own build script. This script provides instructions to Gradle on how to build that specific part of your project. It can include tasks like compiling code, running tests, and generating outputs.

Project cache cleanup

From version 4.10 onwards, Gradle automatically cleans the project-specific cache directory. After building the project, version-specific cache directories in .gradle/<gradle-version>/ are checked periodically (at most every 24 hours) for whether they are still in use. They are deleted if they haven’t been used for 7 days.

This helps to keep the cache directories clean and free up disk space. It also helps to ensure that the build process is as efficient as possible.

Conclusion

In conclusion, delving into the directories and files that Gradle utilizes provides a valuable understanding of how this powerful build tool operates. Navigating through the cache directory, version-specific caches, build artifacts, Gradle Wrapper components, project configuration properties, and subproject structures sheds light on the intricate mechanisms that streamline the development process. With Gradle’s continuous enhancements, such as automated cache cleaning from version 4.10 onwards, developers can harness an optimized environment for building projects efficiently. By comprehending the roles of these directories and files, developers are empowered to leverage Gradle to its fullest potential, ensuring smooth and effective project management.