Setup issue: proxy not forwarding headers properly when trying to use artifacts with local wandb

ERROR Thread-34 :1030531 [internal_api.py:upload_file():1212] upload_file exception 400 Client Error: Bad Request for url:

upon debugging for actual response body, I got :

<?xml version="1.0" encoding="UTF-8"?>
<Error>
    <Code>BadDigest</Code>
    <Message>The Content-Md5 you specified did not match what we received.</Message>
    <Key>rajdeep/dsviz-demo-colab/wandb_artifacts/3/12/a477b63ffd858368c31e10f5def86715</Key>
    <BucketName>local-files</BucketName>
    <Resource>/local-files/rajdeep/dsviz-demo-colab/wandb_artifacts/3/12/a477b63ffd858368c31e10f5def86715</Resource>
    <Region>wandb-local</Region>
    <RequestId>164A6E783DFD7FE9</RequestId>
    <HostId>e3c2fe3d-5ef3-4c3b-bce0-ae64555a0eff</HostId>
</Error>

• response status code: 400

by the way this is a kubernetes install where the requests are proxied via a standalone (non-k8s) nginx load balancer, (nginx server sits infront of k8s cluster, forwarding requests to nodeports as the cloud provider doesnot support ingress load balancer provisioning)

Thanks 😃

[cvphelps]

Hey there, we don’t have bandwidth to support custom configuration questions for free installs of W&B local. Closing this for now.

[vanpelt]

We’re actually running nginx inside of the docker container so we know it works when properly configured. I would try copying the example here:

https://docs.min.io/docs/setup-nginx-proxy-with-minio.html

You can also check your nginx access or error logs to see if it’s truncating the post bodies.