Too many requests to Bitbucket, scans are really slow (JENKINS-55071)

Your checklist for this issue

• Jenkins version

• Plugin version

• Bitbucket cloud

• Bitbucket server and version

Description

Hello, first thanks for this plugin 😃

We have a Jenkins that scans periodically (every minute) some repositories (5 repositories) on Bitbucket cloud using a technical account that have read access on these repositories. All the repositories are part of a single team. The jobs are multibranch pipelines.

We saw that after some minutes, the scans start to become very slow as described at https://issues.jenkins-ci.org/browse/JENKINS-55071. In our case, 2mn to scan a repository of 2 branches and a size of 249.7 KB.

After checking the logs (see attached files), looks like Bitbucket is throttling the requests which makes me think that Jenkins or the plugin is making too many requests because the limits of Bitbucket API is very high: 60000 requests per hour (https://confluence.atlassian.com/bitbucket/rate-limits-668173227.html). Is it possible to log the requests made to Bitbucket ? I only get logs about the API rate limit ?

And also, the cache works for the team repositories list when creating a new multibranch Pipeline.

Jenkins: 2.172 Plugin version: 2.4.4

[lifeofguenter]

I am going to close this for now. I don’t think this is something this plugin can fix. Scanning is a very expensive process - irrespectively when using Bitbucket Cloud or even Bitbucket Server. It just does not scale by design - no matter how we would try to mitigate the issue.

I would recommend building either a proxy or allowing specific webhook endpoints from Bitbucket server IPs (https://support.atlassian.com/organization-administration/docs/ip-addresses-and-domains-for-atlassian-cloud-products/#Outgoing-Connections).

[MateusVMachado]

Hi, I’m having this problem too, but it started a few days ago and there was no change in Jenkins, nor the addition of new branchs. I contacted Atlassian and they sent me a file with all the called endpoints in the last 3 days. The result is strange, it shows that a commit endpoint has been called more than 2k times.

After restarting Jenkins, I realized that the problem stopped, and Atlassian support confirmed this. But after a few hours the problem returned.

Is there a log that can be activated to track these requests? We are unable to work due to this problem.