question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

No coverage report generated for large project

See original GitHub issue

Hi there,

This library is now recommended for code coverage when using v11 of the angular-cli. We are currently using karma-coverage-istanbul-reporter because we have yet to get successful coverage reports with karma-coverage. Seeing, however, that there are deprecation warnings when using the istanbul reporter library with the v11 angular-cli, we would very much like to move off it.

Perhaps someone here can help discern what is wrong with our configuration or point us in the right direction.

I will include the karma.config, and test output debug logs for two different projects.

The first project (PROJECT-1) is a bare project that was initialized with ng new command. It was updated to Angular 11 using the ng update command from angular-cli. The point of including this project is to serve as a ‘control’ for the second project.

The second project (PROJECT-2) comes from a relatively large production codebase. In this codebase is roughly 5000 unit tests.

Important points

  1. The angular.json test configuration block, tsconfig.spec.ts, and karma.config for both PROJECT-1 and PROJECT-2 are exactly the same. I will post the shared karma.config below.
  2. v11 angular-cli still has memory management issues at build time, similar to those mentioned here. For build/run/test on PROJECT-2 we have to use the --max_old_space_size flag. Note that we are running with latest version of the angular-cli and node v12.x.

PROJECT-1 tests are run with: ng test. PROJECT-2 tests are run with: node --max_old_space_size=8192 node_modules/@angular/cli/bin/ng test.

  1. When running tests on PROJECT-1, I get consistent coverage reports. I never see coverage reports for PROJECT-2.
  2. The only discernable difference between PROJECT-1 and PROJECT-2 is the --max_old_space_size configuration property mentioned about.

Below is the karma config file shared by the two projects and the debug logs that are output when running tests.

Karma config

// Karma configuration file, see link for more information
// https://karma-runner.github.io/1.0/config/configuration-file.html

module.exports = function (config) {
  config.set({
    basePath: '',
    frameworks: ['jasmine', '@angular-devkit/build-angular'],
    plugins: [
      require('karma-spec-reporter'),
      require('karma-jasmine'),
      require('karma-chrome-launcher'),
      require('karma-coverage'),
      require('@angular-devkit/build-angular/plugins/karma')
    ],
    client: {
      clearContext: false // leave Jasmine Spec Runner output visible in browser
    },
    coverage: {
      dir: require('path').join(__dirname, '../coverage/karma-coverage'),
      reports: ['html', 'lcovonly', 'text-summary']
    },
    reporters: ['dots', 'spec', 'coverage'],
    specReporter: {
	maxLogLines: 5, // limit number of lines logged per test
	suppressErrorSummary: true, // do not print error summary
	suppressFailed: false, // do not print information about failed tests
	suppressPassed: true, // do not print information about passed tests
	suppressSkipped: true, // do not print information about skipped tests
	showSpecTiming: false, // print the time elapsed for each spec
	failFast: false // test would finish with error when a first fail occurs.
     },
    customLaunchers: {
	'CustomChromeHeadless': {
		base: 'ChromeHeadless',
		flags: [
			'--disable-gpu',
			'--disable-web-security',
			'--no-sandbox',
			'--remote-debugging-port=9222'
		],
		debug: true
	}
    },
	browsers: ['CustomChromeHeadless'],
	browserDisconnectTolerance: 2,
	browserNoActivityTimeout: 300000, // five minutes
	browserDisconnectTimeout: 60000,
	captureTimeout: 300000,
	hostname: '127.0.0.1',
	port: 9876,
	colors: true,
	logLevel: config.LOG_DEBUG,
	autoWatch: false,
	singleRun: true,
	restartOnFileChange: false
  });
};

PROJECT-1-debug-output.txt PROJECT-2-debug-output.txt

In the PROJECT-2 output you will see only ~20 tests ran. This is because I used an fdescribe to run a single test file so as to reduce the noise a little. The same result–no coverage–happens with or without this change.

The point of failure seems to be here: image

Thanks in advance for your support.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:2
  • Comments:58 (17 by maintainers)

github_iconTop GitHub Comments

3reactions
jginsburgncommented, Sep 9, 2021

It is absolutely not dead! We work with limited resources, so we cannot address every issue immediately and sadly that one got closed a long time ago; I will be adding it to my working queue. Also, your collaboration is welcome.

2reactions
ozkoidicommented, Mar 11, 2022

Thank you @devoto13, I tried your branches yesterday and the result was the same (no coverage generated half the time). With Karma’s logLevel set to debug the logs are huge so I put here just the beginning and end of the two types of run.

I include the reduced and cleaned logs below but as far as I can see the differences are:

Coverage report generated No coverage report generated
Logs show [coverage]: onBrowserComplete completed No logs about onBrowserComplete being called
In “[coverage]: onRunComplete called BrowserCollection” the Browser object’s pendingDisconnect property is null In “[coverage]: onRunComplete called BrowserCollection” the Browser object’s pendingDisconnect contains a Timeout object
The launcher sends the SIGKILL no SIGKILL is sent
[coverage]: Writing coverage is present before onExit completed no “Writing coverage” debug log before onExit completed
[reporter.junit]: JUnit results written no “JUnit results written” log

With test coverage generated 👍

> ng test --no-watch --code-coverage

- Generating browser application bundles (phase: setup)...
  [config]: Loading config /home/vsts/work/1/s/Client/karma.conf.js
  [config]: autoWatch set to false, because of singleRun
.
.
.
INFO [coverage]: onBrowserComplete completed
DEBUG [launcher]: CAPTURED -> BEING_KILLED
DEBUG [launcher]: BEING_KILLED -> BEING_FORCE_KILLED
DEBUG [Chrome Headless 99.0.4844.51 (Linux x86_64)]: CONNECTED -> DISCONNECTED
TOTAL: 3231 SUCCESS

Chrome Headless 99.0.4844.51 (Linux x86_64): Executed 3231 of 3234 (skipped 3) SUCCESS (1 min 38.135 secs / 59.497 secs)
TOTAL: 3231 SUCCESS

10 03 2022 16:10:38.524:INFO [coverage]: onRunComplete called BrowserCollection {
browsers: [
Browser {
id: '25015465',
fullName: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/99.0.4844.51 Safari/537.36',
name: 'Chrome Headless 99.0.4844.51 (Linux x86_64)',
lastResult: [BrowserResult],
disconnectsCount: 0,
activeSockets: [Array],
noActivityTimeout: 210000,
singleRun: true,
clientConfig: [Object],
collection: [BrowserCollection],
emitter: [Server],
socket: [Socket],
timer: [Object],
disconnectDelay: 210000,
log: [Logger],
noActivityTimeoutId: Timeout {
\_idleTimeout: 210000,
\_idlePrev: [TimersList],
\_idleNext: [Timeout],
\_idleStart: 278390,
\_onTimeout: [Function (anonymous)],
\_timerArgs: undefined,
\_repeat: null,
\_destroyed: false,
[Symbol(refed)]: true,
[Symbol(kHasPrimitive)]: false,
[Symbol(asyncId)]: 680445,
[Symbol(triggerId)]: 680443
},
pendingDisconnect: null,
state: 'DISCONNECTED'
}
],
emitter: EventEmitter {
\_events: [Object: null prototype] {},
\_eventsCount: 0,
\_maxListeners: undefined,
[Symbol(kCapture)]: false
}
}
INFO [coverage]: onRunComplete completed
DEBUG [karma-server]: Run complete, exiting.
DEBUG [launcher]: Disconnecting all browsers
DEBUG [launcher]: BEING_FORCE_KILLED -> BEING_FORCE_KILLED
DEBUG [proxy]: Destroying proxy agents
INFO [coverage]: onExit called

=============================== Coverage summary ===============================
Statements : 91.97% ( 23967/26059 )
Branches : 79.05% ( 9673/12237 )
Functions : 88.99% ( 6694/7522 )
Lines : 92.2% ( 22669/24586 )
================================================================================
WARN [launcher]: ChromeHeadless was not killed in 2000 ms, sending SIGKILL.
DEBUG [coverage]: Writing coverage to /home/vsts/work/1/s/Client/coverage
DEBUG [launcher]: Process ChromeHeadless exited with code null and signal SIGTERM
DEBUG [temp-dir]: Cleaning temp dir /tmp/karma-25015465
DEBUG [coverage]: Writing coverage to /home/vsts/work/1/s/Client/coverage
DEBUG [coverage]: Writing coverage to /home/vsts/work/1/s/Client/coverage
INFO [coverage]: onExit completed
DEBUG [reporter.junit]: JUnit results written to "/home/vsts/work/1/s/Client/TESTS-Chrome*Headless_99.0.4844.51*(Linux_x86_64).xml".
DEBUG [launcher]: Finished all browsers
DEBUG [launcher]: BEING_FORCE_KILLED -> FINISHED
DEBUG [launcher]: FINISHED -> FINISHED
DEBUG [karma-server]: Received stop event, exiting.
DEBUG [launcher]: Disconnecting all browsers
DEBUG [launcher]: FINISHED -> BEING_FORCE_KILLED
DEBUG [proxy]: Destroying proxy agents
INFO [coverage]: onExit called
INFO [coverage]: onExit completed ##[section]Finishing: Test

Without test coverage 👎

> ng test --no-watch --code-coverage

- Generating browser application bundles (phase: setup)...
  DEBUG [config]: Loading config /home/vsts/work/1/s/Client/karma.conf.js
  DEBUG [config]: autoWatch set to false, because of singleRun
.
.
.
DEBUG [launcher]: CAPTURED -> BEING_KILLED
DEBUG [launcher]: BEING_KILLED -> BEING_FORCE_KILLED
DEBUG [Chrome Headless 99.0.4844.51 (Linux x86_64)]: CONNECTED -> DISCONNECTED
TOTAL: 3231 SUCCESS

Chrome Headless 99.0.4844.51 (Linux x86_64): Executed 3231 of 0 (skipped 3) SUCCESS (2 mins 15.127 secs / 59.651 secs)
TOTAL: 3231 SUCCESS

INFO [coverage]: onRunComplete called BrowserCollection {
browsers: [
Browser {
id: '13380493',
fullName: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/99.0.4844.51 Safari/537.36',
name: 'Chrome Headless 99.0.4844.51 (Linux x86_64)',
lastResult: [BrowserResult],
disconnectsCount: 0,
activeSockets: [Array],
noActivityTimeout: 210000,
singleRun: true,
clientConfig: [Object],
collection: [BrowserCollection],
emitter: [Server],
socket: [Socket],
timer: [Object],
disconnectDelay: 210000,
log: [Logger],
noActivityTimeoutId: Timeout {
\_idleTimeout: 210000,
\_idlePrev: [TimersList],
\_idleNext: [Timeout],
\_idleStart: 285367,
\_onTimeout: [Function (anonymous)],
\_timerArgs: undefined,
\_repeat: null,
\_destroyed: false,
[Symbol(refed)]: true,
[Symbol(kHasPrimitive)]: false,
[Symbol(asyncId)]: 681197,
[Symbol(triggerId)]: 681196
},
pendingDisconnect: Timeout {
\_idleTimeout: -1,
\_idlePrev: null,
\_idleNext: null,
\_idleStart: 187011,
\_onTimeout: null,
\_timerArgs: undefined,
\_repeat: null,
\_destroyed: true,
[Symbol(refed)]: true,
[Symbol(kHasPrimitive)]: false,
[Symbol(asyncId)]: 646232,
[Symbol(triggerId)]: 646172
},
state: 'DISCONNECTED'
}
],
emitter: EventEmitter {
\_events: [Object: null prototype] {},
\_eventsCount: 0,
\_maxListeners: undefined,
[Symbol(kCapture)]: false
}
}
INFO [coverage]: onRunComplete completed
DEBUG [karma-server]: Run complete, exiting.
DEBUG [launcher]: Disconnecting all browsers
DEBUG [launcher]: BEING_FORCE_KILLED -> BEING_FORCE_KILLED
DEBUG [proxy]: Destroying proxy agents
INFO [coverage]: onExit called
INFO [coverage]: onExit completed
DEBUG [launcher]: Process ChromeHeadless exited with code null and signal SIGTERM
DEBUG [temp-dir]: Cleaning temp dir /tmp/karma-13380493
DEBUG [launcher]: Finished all browsers
DEBUG [launcher]: BEING_FORCE_KILLED -> FINISHED
DEBUG [launcher]: FINISHED -> FINISHED
DEBUG [karma-server]: Received stop event, exiting.
DEBUG [launcher]: Disconnecting all browsers
DEBUG [launcher]: FINISHED -> BEING_FORCE_KILLED
DEBUG [proxy]: Destroying proxy agents
INFO [coverage]: onExit called
INFO [coverage]: onExit completed ##[section]Finishing: Test
Read more comments on GitHub >

github_iconTop Results From Across the Web

Code coverage generation fails on … | Apple Developer Forums
Code coverage generation fails on M1 Mac when excluding Arm64 architecture. Hi Everyone, our project has a few vendor libraries that don't support...
Read more >
Angular karma code coverage report folder not generated
When I run ng test --code-coverage , The coverage report is sometimes not generating, sometimes it ...
Read more >
Coverage reports is not generating after sonar scan analysis
We do generate some reported coverage with our sonar scan report. The reason we are concerned is because our line coverage reported by...
Read more >
Adding test coverage to your CI pipeline | Coveralls and CircleCI
You push changes to your code at your SCM (ie. · Your CI service builds your project, runs your tests, and generates your...
Read more >
HTML report generation takes a very long time - OpenClover
For some projects these optimisations may slow down report generation. ... of the code coverage a test methods provides that is provided by...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found