Disable rate_limit check on GitHub Enterprise completely

GitHub Enterprise prior to 2.10 does not have rate limiting, requests to https://hostname/api/v3/rate_limit will return a 404 Not Found status code. in 2.10+ rate limiting can be enabled, but is disabled by default.

Based on the change in #78 there is a check for rate_limit, and then the rate limit is set to an arbitrary high number. On very busy systems this results in tens of thousands of requests per hour to the GitHub Enterprise appliance. Although returning a 404 is a relatively simple operation, there are other things such as logging that are impacted by the volume of requests.

~~A better method would be to look for /api/v3/ within the configured API URL, and then simply skip the rate limit check altogether. /api/v3/ is unique to GitHub Enterprise, so this will not have any impact on the GitHub.com rate_limit endpoint of https://api.github.com/rate_limit~~

[djdefi]

The check I am referring to is here:

https://github.com/kohsuke/github-api/blob/a9fb4546e19e453766a46256c43232ed2ea35e6a/src/main/java/org/kohsuke/github/GitHub.java#L258-L274

Each return retrieve().to("/rate_limit", JsonRateLimit.class).rate; is a request to the GitHub Enterprise instance, which returns 404. This works sure, but on busy Jenkins systems this accounts for a huge number of extra requests, which still needs to be processed (consuming CPU), and fills up logs with noise, etc.

Let me know if I can provide any further information.

[v1v]

I’m afraid I’ve seen this behaviour too in one of my jenkins instances:

• Jenkins version: 1.651.1
• GitHub API plugin: 1.71 (although I’ve just updated to 1.77)
• Github Pull Request plugin: 1.29

Sep 12, 2016 11:37:24 AM org.jenkinsci.plugins.ghprb.GhprbRepository initGhRepository
SEVERE: Error while accessing rate limit API
org.kohsuke.github.HttpException: Server returned HTTP response code: 401, message: 'Unauthorized' for URL: https://github./api/v3/rate_limit
    at org.kohsuke.github.Requester.parse(Requester.java:540)
    at org.kohsuke.github.Requester._to(Requester.java:251)
    at org.kohsuke.github.Requester.to(Requester.java:213)
    at org.kohsuke.github.GitHub.getRateLimit(GitHub.java:263)
    at org.jenkinsci.plugins.ghprb.GhprbRepository.initGhRepository(GhprbRepository.java:71)
    at org.jenkinsci.plugins.ghprb.GhprbRepository.init(GhprbRepository.java:47)
    at org.jenkinsci.plugins.ghprb.GhprbRepository.getGitHubRepo(GhprbRepository.java:334)

± |jira_es_14127 ✓| → curl https://github/api/v3/rate_limit -v
*   Trying X.Y.Z.D...
* Connected to github (X.Y.Z.D) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.sto
* Server certificate: DigiCert SHA2 Secure Server CA
* Server certificate: DigiCert Global Root CA
> GET /api/v3/rate_limit HTTP/1.1
> Host: github
> User-Agent: curl/7.43.0
> Accept: */*
> 
< HTTP/1.1 404 Not Found
< Server: GitHub
< Date: Mon, 12 Sep 2016 09:15:29 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 122
< Status: 404 Not Found
< Cache-Control: no-cache
< X-GitHub-Media-Type: github.v3
< Access-Control-Expose-Headers: ETag, Link, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval
< Access-Control-Allow-Origin: *
< X-GitHub-Request-Id: e5214e1e-5de9-45ab-ab5d-da2fa930d306
< Content-Security-Policy: default-src 'none'
< Strict-Transport-Security: max-age=31536000; includeSubdomains
< X-Content-Type-Options: nosniff
< X-Frame-Options: deny
< X-XSS-Protection: 1; mode=block
< 
{
  "message": "Rate limiting is not enabled.",
  "documentation_url": "https://developer.github.com/enterprise/2.6/v3"
}
* Connection #0 to host github left intact

I’ve just fixed by upgrading:

•  github                    => '1.21.1',
  

•  github_api              => '1.77',
  

•  ghprb                     => '1.33.1',
  

The above http 401/404 errors are not prompting when restarting Jenkins often but every 1/2 minutes

Sep 12, 2016 12:13:00 PM org.jenkinsci.plugins.ghprb.GhprbRepository initGhRepository
SEVERE: Error while accessing rate limit API
org.kohsuke.github.HttpException: Server returned HTTP response code: 401, message: 'Unauthorized' for URL: https://github/api/v3/rate_limit
    at org.kohsuke.github.Requester.parse(Requester.java:540)
    at org.kohsuke.github.Requester._to(Requester.java:251)
    at org.kohsuke.github.Requester.to(Requester.java:213)
    at org.kohsuke.github.GitHub.getRateLimit(GitHub.java:263)
    at org.jenkinsci.plugins.ghprb.GhprbRepository.initGhRepository(GhprbRepository.java:98)
    at org.jenkinsci.plugins.ghprb.GhprbRepository.check(GhprbRepository.java:130)
    at org.jenkinsci.plugins.ghprb.GhprbTrigger.run(GhprbTrigger.java:294)
    at hudson.triggers.Trigger.checkTriggers(Trigger.java:272)
    at hudson.triggers.Trigger$Cron.doRun(Trigger.java:221)
    at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:50)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Server returned HTTP response code: 401 for URL: https://github/api/v3/rate_limit
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1675)
    at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1673)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1671)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1244)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
    at org.kohsuke.github.Requester.parse(Requester.java:524)
    ... 16 more
Caused by: java.io.IOException: Server returned HTTP response code: 401 for URL: https://github/api/v3/rate_limit
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1626)
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338)
    at org.kohsuke.github.Requester.parse(Requester.java:514)
    ... 16 more