Aiobotocore KeyError: 'retries'
See original GitHub issueI am using s3fs with pyarrow, and I am running into a pretty cryptic bug about a key error value. The file begins to write to s3, but then fails with the following exception:
WARNING:aiobotocore.credentials:Refreshing temporary credentials failed during mandatory refresh period.
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/aiobotocore/credentials.py", line 292, in _protected_refresh
metadata = await self._refresh_using()
File "/usr/local/lib/python3.9/site-packages/aiobotocore/credentials.py", line 346, in fetch_credentials
return await self._get_cached_credentials()
File "/usr/local/lib/python3.9/site-packages/aiobotocore/credentials.py", line 356, in _get_cached_credentials
response = await self._get_credentials()
File "/usr/local/lib/python3.9/site-packages/aiobotocore/credentials.py", line 416, in _get_credentials
return await client.assume_role_with_web_identity(**kwargs)
File "/usr/local/lib/python3.9/site-packages/aiobotocore/client.py", line 173, in _make_api_call
http, parsed_response = await self._make_request(
File "/usr/local/lib/python3.9/site-packages/aiobotocore/client.py", line 193, in _make_request
return await self._endpoint.make_request(operation_model, request_dict)
File "/usr/local/lib/python3.9/site-packages/aiobotocore/endpoint.py", line 81, in _send_request
while await self._needs_retry(attempts, operation_model,
File "/usr/local/lib/python3.9/site-packages/aiobotocore/endpoint.py", line 213, in _needs_retry
responses = await self._event_emitter.emit(
File "/usr/local/lib/python3.9/site-packages/aiobotocore/hooks.py", line 29, in _emit
response = handler(**kwargs)
File "/usr/local/lib/python3.9/site-packages/botocore/retryhandler.py", line 189, in __call__
retries_context = kwargs['request_dict']['context']['retries']
KeyError: 'retries'
My pip packages are:
aiobotocore==2.1.0
aiohttp==3.8.1
aioitertools==0.8.0
aiosignal==1.2.0
asn1crypto==1.4.0
async-timeout==4.0.2
attrs==21.4.0
awscli==1.22.30
azure-common==1.1.27
azure-core==1.21.1
azure-storage-blob==12.9.0
biopython==1.79
boto3==1.20.30
botocore==1.23.30
certifi==2020.12.5
cffi==1.15.0
chardet==4.0.0
charset-normalizer==2.0.10
colorama==0.4.3
cryptography==3.4.8
docutils==0.15.2
frozenlist==1.3.0
fsspec==2022.1.0
idna==2.10
iniconfig==1.1.1
isodate==0.6.1
jmespath==0.10.0
msrest==0.6.21
multidict==6.0.2
numpy==1.22.1
oauthlib==3.1.1
oscrypto==1.2.1
packaging==21.3
pandas==1.2.5
pluggy==0.13.1
py==1.11.0
pyarrow==3.0.0
pyasn1==0.4.8
pycparser==2.21
pycryptodomex==3.12.0
PyJWT==2.3.0
pyOpenSSL==20.0.1
pyparsing==3.0.6
pysam==0.18.0
pytest==6.2.4
pytest-mock==3.6.1
python-dateutil==2.8.2
pytz==2021.3
PyYAML==5.4.1
requests==2.27.1
requests-oauthlib==1.3.0
rsa==4.7.2
s3fs==2022.1.0
s3transfer==0.5.0
semver==2.13.0
six==1.16.0
smart-open==5.2.1
snowflake-connector-python==2.4.6
snowflake-rdp-broker==2.5.5
toml==0.10.2
urllib3==1.26.8
wrapt==1.13.3
yarl==1.7.2
Any help is much appreciated! Not sure if this is a bug
Issue Analytics
- State:
- Created 2 years ago
- Comments:7 (1 by maintainers)
Top Results From Across the Web
pandas keyError "Retries" error while saving DataFrame to s3 ...
Context: I am running python code on AWS Sagemaker reading in data from S3 using pandas. Had the same issue as yourself.
Read more >How to use the aiobotocore.config.AioConfig function in ... - Snyk
To help you get started, we've selected a few aiobotocore.config. ... connect_timeout=1, retries={'max_attempts': 0}) async with AIOServer() as server, ...
Read more >ERROR: unexpected error - 'retries' - Questions - DVC
Hi, i run dvc from inside a docker container ,i call dvc commands from python script , inside the container ,everything is configured...
Read more >s3fs.core — S3Fs 2022.11.0 documentation
root_marker = "" connect_timeout = 5 retries = 5 read_timeout = 15 ... **kwargs, ): if key and username: raise KeyError("Supply either key...
Read more >S3 — botocore 1.29.32 documentation - Amazon AWS
... applications should be prepared to retry the failed requests. ... Amazon S3 will return an HTTP status code 404 ("no such key")...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Thanks @martindurant your comment led me to a temp fix at least.
I am guessing for anyone else having this problem these would fix it for you if you pin your dependencies:
I fixed it for now by pinning basically everything to correct versions, note that it wasn’t enough to pin version of
aiobotocore
, it also seemed like the issue was somehow related toawscli
being a compatible version.My pinned dependencies that worked were:
Note that the snowflake-rdp-broker (an internal package) had its own dependencies for awscli/boto3 and that was likely causing the issue. Also, I have to use the
--use-deprecated=legacy-resolver
flag duringpip install
so that the dependencies don’t get downgraded or resolved. sigh python dependency fun.The working version I have tested has these packages for pip freeze:
+1