Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Dependency Track High CPU Behaviour

See original GitHub issue

Issue Type:

defect report
enhancement request

Current Behavior:

Dependency-Track v3.3.1 was running with 25 projects (all maven) succesfully analysed (manual upload of CycloneDX BOM) and with a total of 1000 components.

CPU usage was around 3%.

On 14 December, I then succesfully analysed my first npm project (also using cycloneDX but this time working in a Jenkins pipeline) with 1003 components.

Three days later (after a weekend and with me now being on vacation) the owner of the npm project disabled dependencyTrackPublisher in the pipeline as it was endlessly stuck on…

[DependencyTrack] Polling Dependency-Track for BoM processing status

The DT server logs show that the the last occurence of DependencyCheckScanAgent event was on 19 December.

I returned to work 3 weeks later on 7th January to find DT UI displaying…

There were no errors in the logs from 14th December to 7th January apart from a couple of PostgreSQL connect errors (8003 and 8006). Mostly, things looked OK (lots of NistMirrorTask etc).

After upgrading Dependeny-Track to v3.4.0 on 7th January, the UI looked OK… but things were still not working properly.

A manual upload of a CycloneDX BOM reported success in the UI - but resulted in no analysis being performed. A dependencyTrackPublisher synch of CycloneDX BOM from Jenkins resulted in endless “Polling” entries in the Jenkins console output. A restart of DT folllowed by an immediate manual CycloneDX BOM upload did produce and analysis… but this was temporary.

I think I have solved the problem. I found that DT was still running on an Azure B Series burstable VM

This gave the folllowing CPU usage…

Thus, I upgraded the server to a higher spec and things look OK so far. I have read #255 and the response (worker threads, etc), but upgrading the VM took only a few minutes to do!

In retrospect there are still a couple of things that were not as expected…

Expected Behavior:

Should an npm project have such a huge performance impact? (see CPU graph). Although it doubled the number of components known to the system, the biggest visible difference seems to be that almost every license is “resolved” (has a link) whereas the maven projects all list licences as text alone.
DT server should spot and log CPU problems. In my environment, I have access to the logs - but I had to request the CPU graph from someone else. Covered by #260?
When polling for dependency-track-plugin should not wait so long when polling DT server before failing.

Environment:

Dependency-Track Version: 3.3.1 and then 3.4.0
Distribution: [Executable WAR]
BOM Format & Version: CycloneDX
Database Server: [PostgreSQL]

Other Details:

Occurred a couple of time between 14th and 18th December but not sure if there is any connection to this issue. The database is “Azure PostgreSQL” (ie, separate to DT VM) and uptime is supposed to be really good.

2018-12-14 18:08:03,169 [] WARN [com.zaxxer.hikari.pool.ProxyConnection] HikariPool-2 - Connection org.postgresql.jdbc.PgConnection@62928ff4 marked as broken because of SQLSTATE(08006), ErrorCode(0)
org.postgresql.util.PSQLException: An I/O error occurred while sending to the backend.
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:335)
	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)