question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Connection fails to parse sqs:// URLs with `/` character in secret key

See original GitHub issue

With kombu version 4.2.1, creating a kombu.Connection with a sqs:// broker URL that has an AWS secret key with a / (forward slash) character raises a ValueError during parse_url().

Steps to reproduce

Assume we have an AWS access key of access_key and an AWS secret key of secret/key.

sqs_test.py

import kombu

broker_url = 'sqs://access_key:secret/key@'
kombu.Connection(broker_url)

Executing this yields:

$ python sqs_test.py
Traceback (most recent call last):
  File "sqs_test.py", line 4, in <module>
    kombu.Connection(broker_url)
  File ".venv/lib64/python3.7/site-packages/kombu/connection.py", line 181, in __init__
    url_params = parse_url(hostname)
  File ".venv/lib64/python3.7/site-packages/kombu/utils/url.py", line 34, in parse_url
    scheme, host, port, user, password, path, query = _parse_url(url)
  File ".venv/lib64/python3.7/site-packages/kombu/utils/url.py", line 52, in url_to_parts
    parts.port,
  File "/usr/lib64/python3.7/urllib/parse.py", line 169, in port
    port = int(port, 10)
ValueError: invalid literal for int() with base 10: 'secret'

Obvious workaround

Regenerating a new key that does not contain a / character works around this issue, but that is a sub-optimal solution.

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Reactions:2
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

2reactions
shekhar-kamblecommented, Dec 3, 2018

But shouldn’t the bug in the library be fixed in itself, rather than expecting the user to handle it.

0reactions
jwilgescommented, Mar 22, 2019

Literal / characters are interpreted as path delimiters and thus URL parsing cannot work correctly. As far as I can see there is no bug to be fixed.

It was not obvious at the time, but it is mostly just a simple URL parse call during kombu.Connection().

So, yes, URL encoding the username and password are the correct and safe steps. Fortunately, this caveat is now mentioned up front in some of the updated documentation as of https://github.com/celery/celery/commit/57dbd63113f7722a1ee548cd24fe0f1bd84d7073.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Symfony5 SQS consumer invalid access key - Stack Overflow
I think it comes from the special chars in the secret key because when i try with another key containing only letters and...
Read more >
Troubleshoot QueueDoesNotExist errors when making API ...
When I make API calls to my Amazon Simple Queue Service (Amazon SQS) queue, I receive a QueueDoesNotExist error similar to the following: ......
Read more >
class SQS. Client - Boto3 Docs 1.26.35 documentation - AWS
Returns the URL of an existing Amazon SQS queue. To access a queue that belongs to another AWS account, use the QueueOwnerAWSAccountId parameter...
Read more >
Can Celery be used with Amazon SQS | Edureka Community
I took a look at the SQS URL parsing in kombo.utils.url. ... So, to workaround the issue of secret keys with forward slashes, ......
Read more >
Resolve "AWS Access Key Id" errors when sending requests ...
The error message "The AWS Access Key Id you provided does not exist in our records" indicates that there's an issue with the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found