High RAM usage for app:koverXmlReport command in multi-module Android project on Azure CI

[CazimirRoman]

We are using Kover in a multi-module Android project written in Kotlin and running our CI on Azure. We've observed a significant increase in RAM usage when running the app:koverXmlReport command. The CI is currently showing a peak RAM usage of 24GB, which is approximately 10GB higher compared to when we don't run the Kover report generation.

We have already attempted to exclude specific classes that are not relevant for code coverage analysis, but the RAM consumption remains the same.

Has anyone experienced similar high RAM usage with app:koverXmlReport in multi-module Android projects?

• What are some potential causes for this high memory consumption?

• What strategies can we use to reduce the RAM usage of the app:koverXmlReport command?

• Are there any specific configurations or optimizations within Kover that can be applied to address this issue?

Any guidance or suggestions would be greatly appreciated. Thank you!

[shanshin]

Hi, could you please specify which command is running koverXmlReport? What does the Gradle command look like?
like gradlew koverXmlReport or gradlew :app:koverXmlReport

[CazimirRoman]

We are running it like this tasks: ./gradlew clean assembleDebug app:koverXmlReport

[shanshin]

Do you have any observations at what stage exactly is the peak of memory consumption?

The fact is that the app:koverXmlReport command runs the building of all build variants in all modules, and then calls all unit tests for them (both for release and debug).

Also, please double-check that the koverXmlReport (without a colon) command is not accidentally called anywhere. This causes the generation of a local report for each module, which, due to specific of Gradle, may increase memory consumption.

[deep4nsh]

@CazimirRoman i found this article hope it helps.

Experiencing high RAM usage when running the app:koverXmlReport command in a multi-module Android project can be challenging. Here are some insights into potential causes and strategies to mitigate the issue.

Potential Causes of High RAM Usage

1. Instrumentation Overhead: Kover instrumentation can add significant overhead, especially in multi-module projects where multiple classes are instrumented simultaneously. This can lead to increased memory consumption during report generation.

2. Concurrent Tasks: Running multiple tasks or plugins concurrently in the CI pipeline might exacerbate memory usage, as each task can consume additional resources.

3. Garbage Collection: Inefficient garbage collection during the report generation process may lead to temporary spikes in memory usage.

Strategies to Reduce RAM Usage

1. Limit Instrumentation Scope: Ensure that you are excluding unnecessary classes from instrumentation. You mentioned that you have already attempted this; however, double-checking the configurations might reveal additional optimizations.

2. Increase CI Resources Temporarily: If feasible, temporarily increasing the allocated resources (e.g., RAM) for your CI environment during report generation can help manage peak loads without failing the build.

3. Run Reports Sequentially: If you are generating multiple reports (e.g., XML and HTML), consider running them sequentially rather than concurrently to reduce simultaneous memory demand.

4. Optimize Gradle Configuration: Review your Gradle build configuration for any optimizations. For instance, using the --no-daemon flag can sometimes help with memory issues.

5. Profile Memory Usage: Utilize tools like VisualVM or YourKit to profile memory usage during the report generation process. This can help identify specific areas consuming excessive memory.

Kover Configuration and Optimizations

1. Adjust Kover Settings: Review Kover's configuration options, such as adjusting thresholds or modifying how coverage is reported. For example, you can specify which classes to include or exclude more granularly.

2. Use Latest Version: Ensure that you are using the latest version of Kover, as performance improvements and bug fixes are regularly released.

3. Merge Reports: If applicable, consider merging coverage reports from different modules instead of generating separate reports for each module, which may help reduce overall memory consumption.

4. Custom Report Generation Logic: If feasible, implement custom logic for generating reports that only includes essential information rather than all coverage data.

By implementing these strategies and configurations, you should be able to manage and potentially reduce the high RAM usage associated with generating Kover reports in your multi-module Android project.

[shanshin]

@deep4nsh , thanks!

Let me make a few comments

Instrumentation Overhead: Kover instrumentation can add significant overhead, especially in multi-module projects where multiple classes are instrumented simultaneously. This can lead to increased memory consumption during report generation.

By itself, instrumentation does not bring much memory consumption to Gradle, because instrumentation occurs in the Java process in which the tests are run, this one ends after the tests are completed.
So the additional memory required for on-the-fly instrumentation of classes is released by the time koverXmlReport is called.

Unfortunately, without profiling memory usage, it is difficult to say what exactly caused the memory overruns.
This is very likely due to the use of Worker API, because when using classpath isolation, there is a problem with memory leakage, which can lead to OOM on large projects.