Connection fails to parse sqs:// URLs with `/` character in secret key
See original GitHub issueWith kombu
version 4.2.1, creating a kombu.Connection
with a sqs://
broker URL that has an AWS secret key with a /
(forward slash) character raises a ValueError
during parse_url()
.
Steps to reproduce
Assume we have an AWS access key of access_key
and an AWS secret key of secret/key
.
sqs_test.py
import kombu
broker_url = 'sqs://access_key:secret/key@'
kombu.Connection(broker_url)
Executing this yields:
$ python sqs_test.py
Traceback (most recent call last):
File "sqs_test.py", line 4, in <module>
kombu.Connection(broker_url)
File ".venv/lib64/python3.7/site-packages/kombu/connection.py", line 181, in __init__
url_params = parse_url(hostname)
File ".venv/lib64/python3.7/site-packages/kombu/utils/url.py", line 34, in parse_url
scheme, host, port, user, password, path, query = _parse_url(url)
File ".venv/lib64/python3.7/site-packages/kombu/utils/url.py", line 52, in url_to_parts
parts.port,
File "/usr/lib64/python3.7/urllib/parse.py", line 169, in port
port = int(port, 10)
ValueError: invalid literal for int() with base 10: 'secret'
Obvious workaround
Regenerating a new key that does not contain a /
character works around this issue, but that is a sub-optimal solution.
Issue Analytics
- State:
- Created 5 years ago
- Reactions:2
- Comments:5 (3 by maintainers)
Top Results From Across the Web
Symfony5 SQS consumer invalid access key - Stack Overflow
I think it comes from the special chars in the secret key because when i try with another key containing only letters and...
Read more >Troubleshoot QueueDoesNotExist errors when making API ...
When I make API calls to my Amazon Simple Queue Service (Amazon SQS) queue, I receive a QueueDoesNotExist error similar to the following: ......
Read more >class SQS. Client - Boto3 Docs 1.26.35 documentation - AWS
Returns the URL of an existing Amazon SQS queue. To access a queue that belongs to another AWS account, use the QueueOwnerAWSAccountId parameter...
Read more >Can Celery be used with Amazon SQS | Edureka Community
I took a look at the SQS URL parsing in kombo.utils.url. ... So, to workaround the issue of secret keys with forward slashes, ......
Read more >Resolve "AWS Access Key Id" errors when sending requests ...
The error message "The AWS Access Key Id you provided does not exist in our records" indicates that there's an issue with the...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
But shouldn’t the bug in the library be fixed in itself, rather than expecting the user to handle it.
It was not obvious at the time, but it is mostly just a simple URL parse call during
kombu.Connection()
.So, yes, URL encoding the username and password are the correct and safe steps. Fortunately, this caveat is now mentioned up front in some of the updated documentation as of https://github.com/celery/celery/commit/57dbd63113f7722a1ee548cd24fe0f1bd84d7073.