JaCoCo class mismatch when running together with Firebase Perf plugin

Hi @Rolf-Smit, after trying to upgrade the root coverage plugin 1.5.1 it works perfectly. But after I check again the report, one of unit test inside the app module isn’t covered but the other modules are covered correctly. When I run the command I have an error like this [ant:jacocoReport] Classes in bundle 'My App' do not match with execution data. For report generation the same class files must be used as at runtime.

[ant:jacocoReport] Execution data for class com/example/myapp/presentation/viewmodel/SplashScreenViewModel does not match.

Here’s the report

For your information,

1. I was already set testCoverageEnabled as true and running the command at the correct build type
2. I was already searching this issue at stackoverflow, and the cause is about a different JAVA version or using a different Gradle. The question is, It’s okay if compileOptions in my project is different from the actual java version I used? Because, I’ve used this setup (compileOptions and java version) before, and running well.
3. I was already upgraded and downgraded the jacoco too, from existing 0.8.6 > 0.8.7 > 0.8.8 but still not covered.

My Version

• AGP: 7.2.0
• Root Coverage: 1.5.1
• Kotlin Gradle: 1.6.21
• Jacoco: 0.8.6
• Java Version: 11.0.9
• Compile Options: sourceCompatibility JavaVersion.VERSION_1_8 targetCompatibility JavaVersion.VERSION_1_8

[Rolf-Smit]

@MirzaUkas thanks, this is helpful.

So what’s happening is that both JaCoCO and Firebase Performance manipulate and rewrite your class files. JaCoCo does this to add code so that it can measure which lines have been “visited” during a test. Firebase Performance does this so it can add code to measure performance.

The issue is that for JaCoCo to work the class files used at runtime must be exactly the same class files that are given to JaCoCo (otherwise coverage data does not match up). This plugin uses the official Android Gradle Plugin APIs to provide those class files to JaCoCo. It seems to be the case that JaCoCo (and this Plugin) is provided with class files that do not contain the Firebase Performance changes, while at runtime those changes are in the code, this creates a mismatch between runtime and the class files available for calculating the coverage.

Where does this need fixing? To be honest I’m also not too sure, since there could be multiple things at play here. Maybe the ordering in which the Firebase Performance plugin is applied and JaCoCo (this plugin) could make a difference. Maybe you could move them around and make sure JaCoCo is the last one applied? Or maybe first one?

You could do this by using (or maybe just changing the order in the plugins block already make a difference):

plugins {
  id "someArtifact" version "1.0.0" apply false
}

apply "someArtifact"

But a quick fix that is guaranteed to work would be to not enable Firebase Performance while running tests in the first place. Firebase Performance is a tool to monitor performance issues on production and to my knowledge it is often disabled on debug (test) builds because it is not needed there, but more importantly when disabled it also has a positive impact on compile times.

Here is some documentation on how you can disable it for specific build-types: https://firebase.google.com/docs/perf-mon/disable-sdk?platform=android#disable-gradle-plugin

[MirzaUkas]

@Rolf-Smit, after trying to check my plugin my first suspicious plugin that cause this problem is firebase crashlytic because this plugin is crash reporting plugin like you asked before. But after trying to comment this plugin, the test is still uncovered. Interestingly, after trying to comment and uncomment another ‘suspicious’ dependencies. I found that when I comment firebase performance dependencies the test is covered and the issue is gone. I don’t know which plugin must solve this issue, the firebase perf, jacoco, or root coverage plugin.

This is my dependencies on my build.gradle project

com.google.gms:google-services:4.3.3 org.jacoco:org.jacoco.core:0.8.6 com.github.triplet.gradle:play-publisher:3.7.0 com.github.ben-manes:gradle-versions-plugin:0.39.0 nl.neotech.plugin:android-root-coverage-plugin:1.5.1 com.google.firebase:firebase-crashlytics-gradle:17.0.1 com.google.firebase:firebase-appdistribution-gradle:2.0.0 com.google.firebase:perf-plugin:1.4.1