question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Unable to send/receive custom metadata with S3 Upload

See original GitHub issue

I am using ng-file-upload to upload a JPG file to my S3 Bucket.

file.upload = Upload.upload({
    url: "https://<my-bucket-name>.s3.amazonaws.com/", 
    method: "POST",
    data: {
        key: "custom-filename.jpg", 
        AWSAccessKeyId: "<AWSAccessKeyId>",
        acl: "public-read", 
        policy: <policy>, 
        signature: <signature>, 
        "Content-Type": "image/jpeg", 
        filename: file.name, 
        file: file, 
        Metadata: {
            "x-amz-meta-hello": "Custom Metadata Value"
        }
    }
});

I have also tried the following (in the above code)

Metadata: {
    hello: "Custom Metadata Value"
}

& simply

"x-amz-meta-hello": "Custom Metadata Value"

I have included the Custom Metadata in my Policy file as

["starts-with", "x-amz-meta-hello", ""]

Also, the CORS Configuration under Bucket Permissions on S3 is

<CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <AllowedMethod>POST</AllowedMethod>
    <AllowedMethod>HEAD</AllowedMethod>
    <MaxAgeSeconds>3000</MaxAgeSeconds>
    <ExposeHeader>x-amz-meta-hello</ExposeHeader>
    <AllowedHeader>*</AllowedHeader>
</CORSRule>

The above code and settings are working, the JPG file is successfully uploaded, but somehow the Custom Metadata value is not being set.

On successful upload, I am calling a Lambda function to resize the JPG and storing it in a separate bucket. Even this part is working but I am not able to read the Custom Metadata (x-amz-meta-hello) value in my Lambda Function. I need that value to assign a separate folder to the uploaded file.

To read the Custom Metadata in my Lambda Function

var s3 = new AWS.S3();
s3.headObject({
    Bucket: <BucketName>,
    Key: <S3ObjectKey>
}, function(err, data) {
    if (err) {
        console.log(err);
    }
    else
    {   
        console.log(data);
    }     
});

Not sure what am I missing here … Please advise.

Thanks. (AngularJS version 1.5.0, ng-file-upload version 12.2.9, Google Chrome version 53.0.2785.113 on OSX 10.10.5)

Issue Analytics

  • State:open
  • Created 7 years ago
  • Comments:6 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
rahatarmanahmedcommented, Jan 30, 2017

I ran into this issue also and the stacko answer wasn’t that helpful because it still had some issues. Here’s what I did to fix it:

  1. Metadata must be provided in the form of a x-amz-meta-{key}: {value} header. But POST requests are special in that headers are instead sent as form data. So what we need to send looks like:
data: {
    key: "custom-filename.jpg", 
    AWSAccessKeyId: "<AWSAccessKeyId>",
    acl: "public-read", 
    policy: <policy>, 
    signature: <signature>, 
    "Content-Type": "image/jpeg", 
    filename: file.name, 
    file: file, 
    "x-amz-meta-hello": "Custom Metadata Value"
}
  1. The upload policy needs to have a condition for this metadata field, or else it will complain with an error about receiving extra data. @sh59 almost has the right condition, it just needs to reference the metadata with a $ in order to work.
["starts-with", "$x-amz-meta-hello", ""]

Fixing these two things made it work for me. This issue can be closed.

0reactions
vaibhavshah2108commented, Dec 9, 2022

I am following this documentation to upload S3 object using API gateway, with no Lambda in between. This works using Postman! The problem happens when I am trying to add user-defined metadata to headers, but that seems to be not working. I also tried to add “x-amz-meta-{key}”: “{value}” in the form-data field of the body, and that did not work as well. Any recommended solution? Thanks in advance!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Unable to send/receive custom metadata on Amazon S3 ...
I need that value to assign a separate folder to the uploaded file. To read the Custom Metadata in my Lambda Function var...
Read more >
Working with object metadata - Amazon Simple Storage Service
Name Description Can user modify the value? Date Current date and time. No Content‑Disposition Object presentational information. Yes Content‑Length Object size in bytes. No
Read more >
Using IAM roles to allow the Pods in AWS EKS to read the ...
Following policy, will only allow the Kubernetes service account named 'my-sa' (in the namespace 'dev') to assume the role 'custom-read-s3- ...
Read more >
SQS to S3: Move Data Using AWS Lambda and AWS Firehose
SQS to S3: Limitations of the Custom-Code Approach; Conclusion ... It helps to send, receive and store messages between software systems.
Read more >
How to create a Helm chart repository using Amazon S3
In this post, I'll show how you can use an AWS S3 bucket to host a Helm chart repository, how to push custom...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found