Unable to deploy to s3 in cn-north-1

Hi, thanks for all your hard work on this plugin. It’s been really useful for us.

I’m having trouble deploying artifacts using s3-wagon-private to any S3 bucket in the cn-north-1 region. I suspect this is related to request signing as cn-north-1 only supports the signature v4 signing process.

When I try lein deploy, I see the following error:

Could not transfer artifact myartifact:myartifact:jar:1.0.16 from/to releases 
(s3p://mybucket/releases/): The AWS Access Key Id you provided does not exist in our
records. (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId;
Request ID: 14856B606B12E5CD)

I’m using [s3-wagon-private "1.3.0-alpha2"] and my credentials are set as environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_DEFAULT_REGION. My lein configuration looks like:

:deploy-repositories [["releases" {:url "s3p://mybucket/releases/"
                                   :no-auth true}]]

I can execute other operations using the AWS CLI, such as aws s3api list-buckets so I’m certain that the credentials are valid. I can also deploy successfully to eu-west-1 with this lein configuration (using different credentials of course).

I haven’t yet worked out the exact cause of the problem here although as mentioned above it seems likely that this is related to the signing process. I tried updating the aws-sdk to 1.11.86 but this didn’t fix the problem.

[sheelc]

@danielcompton @joelittlejohn Is it necessary to set the endpoint for China? I’m wondering if setting the region is sufficient. The China region is in the region enum of the SDK so perhaps when using this region the SDK takes care of the endpoint stuff for us?

Unfortunately I don’t have any credentials to try this. @joelittlejohn do you know if it’s possible to build an s3 client with credentials and the region without the endpoint? Or would you be able to try with your credentials?

If we can just set the region, here’s another idea for the mix:

• The same way we started using the DefaultCredentialsProviderChain, we can use the AWS SDK DefaultAwsRegionProviderChain which will look for AWS_REGION env var, region from the profile file, or from instance metadata. I haven’t checked yet but we may already be using the default region provider chain. The only issue is for folks relying on this auto-region detecting feature that the wagon attempts to provide, which leads to the second piece.

• It looks like the SDK has a function for getBucketLocation which can be translated to a region which can be set on the client. Since @joelittlejohn mentioned that we can’t make any API calls to any endpoints outside of China with China IAM creds, we would have to make this auto-detect behavior fallback only so that the China region would succeed. (Alternatively, we could just leave the default aws-maven could for auto-detection in but it seems like that hard-coded list of regions will get out of date as compared to the SDK.)

[sheelc]

Excellent news @joelittlejohn! The only trickiness I can think of then is the DefaultAwsRegionProviderChain errors when it can’t find a region so we’ll just have to be careful of that to make sure we get to the fallback behavior.

I should be able to take a stab at implementing this over the weekend for review.