list_objects_v2's MaxKeys not working
See original GitHub issueI’ve 16000 records in s3 bucket’s prefixed location. Used below code to listdown all keys in the prefixed location:
s3Contents = client.list_objects_v2(
Bucket=config['aws']['src']['bucket'],
MaxKeys=50000,
Prefix=config['aws']['src']['dir'])
However, it’s only listing default behaviour of listing 1000 records although specified 50000 as max keys.
Hence, using NextContinuationToken, I’ve to get the full list. Is this expected behaviour or a bug?
Issue Analytics
- State:
- Created 4 years ago
- Comments:7 (2 by maintainers)
Top Results From Across the Web
Does Amazon S3 have a limit to MaxKeys when calling ...
I tried setting the MaxKeys setting to 1,000 but it still pulls 1,080 results. My code: $iterator = $s3->getIterator('ListObjects', ...
Read more >ListObjectsV2 - Amazon Simple Storage Service
These rolled-up keys are not returned elsewhere in the response. Each rolled-up result counts as only one return against the MaxKeys value. Type:...
Read more >Amazon S3 ListObjects API
Specifies the key to start with when listing objects in a bucket. max-keys. Sets the maximum number of keys returned in the response....
Read more >Object Storage Service:List objects - Alibaba Cloud
List objects,Object Storage Service:You can call GetBucket (ListObjects) in ... problem while trying to communicate with OSS, " + "such as not being...
Read more >list-objects-v2 — AWS CLI 2.9.8 Command Reference
If the number of results exceeds that specified by MaxKeys, all of the results might not be returned. Contents -> (list). Metadata about...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@kyleknap Thanks for your response. Could you please mention how to list more than 1000 object using list_objects ?
Use the pagination:
From: https://docs.aws.amazon.com/cli/latest/reference/s3api/list-objects-v2.html
Recursively make the call with pagination until there is no Truncation.