GoogleApis: Use of ApachetHttpTransport behind a proxy produces NonRepeatableRequestException

Environment details

• Java 8
• google-http-client versions:
  • google-http-client-apache-v2-1.30.1
    • google-http-client-1.30.1
  • google-api-client-1.30.2
    • google-http-client-jackson2-1.30.1
    • google-oauth-client-1.30.1

Steps to reproduce

1. Trying to make a POST request behind a proxy produces a NonRepeatableRequestException.
2. It seems like ApacheHttpRequest wraps ByteArrayContent that is repeatable (JavaDoc) inside a ContentEntity that is non-repeatable.
3. Debuging execution, first proxy is returning “407 Proxy Authentication Required”, then it tries to repeat the request (guess that including the credentials) and that exception arises because ContentEntity used is non-repeatable.

IDE screenshot

Code example

// 1.- Setting ssl and proxy
HttpClientBuilder builder = HttpClientBuilder.create();
            
SSLContext sslContext = SslUtils.getTlsSslContext();
SslUtils.initSslContext(sslContext, GoogleUtils.getCertificateTrustStore(), SslUtils.getPkixTrustManagerFactory());
builder.setSSLSocketFactory(new SSLConnectionSocketFactory(sslContext));
            
builder.setProxy(new HttpHost(host, port));
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(new AuthScope(host, port), new UsernamePasswordCredentials(user, pass));
builder.setDefaultCredentialsProvider(credentialsProvider);

// 2.- Build request
HttpTransport httpTransport = new ApacheHttpTransport(builder.build());
HttpRequestFactory factory = httpTransport.createRequestFactory(credential);

HttpContent httpContent = new ByteArrayContent("application/json", "{}")
HttpRequest request = factory.buildRequest("POST", new GenericUrl(url), httpContent);

// 3.- Execute request
HttpResponse httpResponse = request.execute();

Stack trace

Stack trace

org.apache.http.client.ClientProtocolException
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.13.jar!/:4.5.13]
    at com.google.api.client.http.apache.v2.ApacheHttpRequest.execute(ApacheHttpRequest.java:73) ~[google-http-client-apache-v2-1.39.2.jar!/:?]
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012) ~[google-http-client-1.39.2.jar!/:1.39.2]
    at 
    ...
Caused by: org.apache.http.client.NonRepeatableRequestException: Cannot retry request with a non-repeatable request entity.
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:225) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.13.jar!/:4.5.13]
    at com.google.api.client.http.apache.v2.ApacheHttpRequest.execute(ApacheHttpRequest.java:73) ~[google-http-client-apache-v2-1.39.2.jar!/:?]
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012) ~[google-http-client-1.39.2.jar!/:1.39.2]

</etails>

[chanseokoh]

who repeat the request is Apache HttpClient

Yes, of course. And it is you who manually programmed your app to allow Apache HttpClient to re-send a request for POST (which may be a viable workaround for you in your circumstances where you think it’s safe to do so, but this shouldn’t be generalized for other cases).

Now I see what you are missing and where you have a wrong idea. When interacting with a proxy (or a non-proxy) that requires auth, generally you (whether it is you or the Apache library) will have to make at least TWO requests. First, you try without sending any sensitive information (why would you disclose your information upfront to someone who cannot be trusted? Even if you trust them, you don’t really know if they are going to need your info at all. Moreover, even so, you don’t know how correctly you should present your sensitive info). That first request may (or may not) fail with an error like “407 Proxy Authentication Required” (people call this that the server is “challenging” you), and based on what kinds of challenges the server gives you, you will need to take the right action to prepare an auth header for the second request. And the Apache library does that for you.

(despite include the credentials)

What did you expect that calling .setDefaultCredentialsProvider would do? It doesn’t do what you are currently thinking. The Apache library does nothing about your password in the first request. As I said earlier, in the end, you need to provide the right form of credentials that the server wants after checking the value of Proxy-Authenticate, which tells you how you should correctly auth with the server. That is why generally you have to repeat a request. If all these sound alien to you, please take a moment to read this introductory doc to understand how this challenge-based HTTP auth framework works. (The doc makes a note that it will explain only with the “Basic” scheme for educational purposes, but note that there are other non-basic schemes.)

[chanseokoh]

Sorry, forgot this one.

This is working as intended. POST requests are fundamentally considered non-retryable, as they are most likely to have a server store data. For example, a server is recommended to return 201 (Created) as a response when the server successfully created one or more resources. Retrying a POST request may end up inserting, uploading, or posting data multiple times. This is why sometimes web browsers show the following prompt to avoid “a duplicate credit card transaction”:

A potential retry logic for POST should be implemented at the user application level, not at the library level.

In your case, the cause of the error is that you are not authorized to use the proxy. Therefore, you need to authenticate with the proxy first before attempting to use it, and then send (or re-send) a POST request.