Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

GCS: `setRootUrl` override does not apply to PUT requests in resumable uploads

See original GitHub issue

Environment details

OS type and version: Debian 10
Java version: 8u292
google-api-client version(s): 1.39.0

Steps to reproduce

Override root URL using Storage.setRootUrl("...") method.
Execute resumable upload.

Stack trace

Oct 26, 2021 5:42:51 AM com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.http.HttpResponse <init>
CONFIG: -------------- RESPONSE --------------
HTTP/1.1 200 OK
Server: UploadServer
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
X-GUploader-UploadID: ADPycdtrIid39eNFjGnt570wYOCu3aCCy872EmJISL1_euGWONVNnyFR_FiFkZJebIh0WVNwx0_PlBZPzvV_Ll4S2e6JDHIlOA
Vary: X-Origin
Vary: Origin
Pragma: no-cache
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Content-Length: 0
Date: Tue, 26 Oct 2021 05:42:51 GMT
Location: https://storage.googleapis.com/upload/storage/v1/b/<BUCKET>/o?ifGenerationMatch=0&name=test-empty-04._COPYING_&uploadType=resumable&upload_id=ADPycdtrIid39eNFjGnt570wYOCu3aCCy872EmJISL1_euGWONVNnyFR_FiFkZJebIh0WVNwx0_PlBZPzvV_Ll4S2e6JDHIlOA
Content-Type: text/plain; charset=utf-8

Oct 26, 2021 5:42:51 AM com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.http.HttpRequest execute
CONFIG: -------------- REQUEST  --------------
PUT https://storage.googleapis.com/upload/storage/v1/b/<BUCKET>/o?ifGenerationMatch=0&name=test-empty-04._COPYING_&uploadType=resumable&upload_id=ADPycdtrIid39eNFjGnt570wYOCu3aCCy872EmJISL1_euGWONVNnyFR_FiFkZJebIh0WVNwx0_PlBZPzvV_Ll4S2e6JDHIlOA
Accept-Encoding: gzip
Authorization: <Not Logged>
Content-Range: bytes */0
User-Agent: GHFS/hadoop3-2.2.2-dataproc zone/us-central1-c Google-HTTP-Java-Client/1.39.0 (gzip)
Content-Type: application/octet-stream
Content-Length: 0

Oct 26, 2021 5:42:51 AM com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.http.HttpRequest execute
CONFIG: curl -v --compressed -X PUT -H 'Accept-Encoding: gzip' -H 'Authorization: <Not Logged>' -H 'Content-Range: bytes */0' -H 'User-Agent: GHFS/hadoop3-2.2.2-dataproc zone/us-central1-c Google-HTTP-Java-Client/1.39.0 (gzip)' -H 'Content-Type: application/octet-stream' -d '@-' -- 'https://storage.googleapis.com/upload/storage/v1/b/<BUCKET>/o?ifGenerationMatch=0&name=test-empty-04._COPYING_&uploadType=resumable&upload_id=ADPycdtrIid39eNFjGnt570wYOCu3aCCy872EmJISL1_euGWONVNnyFR_FiFkZJebIh0WVNwx0_PlBZPzvV_Ll4S2e6JDHIlOA' << $$$

External references such as API reference guides

https://cloud.google.com/storage/docs/resumable-uploads

Any additional information below

Seems like this bug caused by this line of code: https://github.com/googleapis/google-api-java-client/blob/f5ea30eb751519392c8c48038baef12331fd8ce4/google-api-client/src/main/java/com/google/api/client/googleapis/media/MediaHttpUploader.java#L363

Issue Analytics

State:
Created 2 years ago
Comments:8 (4 by maintainers)

Top GitHub Comments

1reaction

medbcommented, Nov 1, 2021

@frankyn agree, it’s proper to fix this on backend, but it could be goo to keep this issue open until its fixed on backed for public tracking purposes.

1reaction

BenWhiteheadcommented, Nov 1, 2021

One thing to be very clear about, making this change client side will mean that the client does not properly implement handling of the Location header according to HTTP expectations. When a fully qualified Location is provided a client is expected to respect that Location without any modification or relative resolution