Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Feature Request: Option to return object metadata with Storage.list()

See original GitHub issue

It’s often very useful to store extra information about the files stored on S3 in metadata. e.g. the row count in a csv file, a user readable name etc, but this info isn’t available using Storage.list().

Storage.list() uses S3.listObjects() which doesn’t return object metadata, so an extra S3.headObject() call would need to be made for each file returned by S3.listObjects() in order to get the metadata for each object, which is a performance hit, but it would be nice to have the option, e.g. to pass in a metadata: true flag in the options for Storage.list().

If this feature would be acceptable, then I would be happy to create a pull request for it.

Issue Analytics

State:
Created 5 years ago
Reactions:13
Comments:6 (2 by maintainers)

Top GitHub Comments

2reactions

mrichmancommented, May 27, 2020

I’m trying to do this as well.

0reactions

ashika01commented, Nov 17, 2021

@tiomun this seems like a limitation even from s3 sdk side. There was FR -> https://github.com/aws/aws-sdk-js/issues/1181

I would suggest reaching out S3 directly if this is a feature for s3 to provide. A workaround in mentioned in the linked issue above. You can try to use that.

We will work internally to see if we can introduce a API using HeadObject.

Top Results From Across the Web

Working with object metadata - Amazon Simple Storage Service

Name Description Can user modify the value? Date Current date and time. No Content‑Disposition Object presentational information. Yes Content‑Length Object size in bytes. No

Object metadata | Cloud Storage - Google Cloud

Objects stored in Cloud Storage have metadata associated with them. Metadata identifies properties of the object, as well as specifies how the object...

S3 — Boto3 Docs 1.26.37 documentation - Amazon AWS

A low-level client representing Amazon Simple Storage Service (S3) ... In the Complete Multipart Upload request, you must provide the parts list.

Managing Buckets - Oracle Help Center

In the Oracle Cloud Infrastructure Object Storage service, a bucket ... object metadata, download bucket objects, and optionally list bucket ...

Object Storage: An Introduction | IBM

Storing/managing unstructured data · Scalability · Reduced complexity · Disaster recovery/availability · Customizable metadata · Affordability · Cloud compatibility.