question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

redis: add configuration parameter to prefix all keys used with a custom string

See original GitHub issue

I use kombu (via celery) with a shared redis instance. The administrators of that instance require me to prefix all the keys I use in the redis server with a custom string. While Celery provides ways to prefix queue names, it seems to me that there is currently no way in Kombu (and hence Celery) to prefix all the redis keys used.

My understanding is that the keys used for channels are defined by keyprefix_queue = '_kombu.binding.%s', which not easily modifiable from Celery (via BROKER_TRANSPORT_OPTIONS). Would it make sense to add a configuration parameter for that? If that sounds like the correct approach, I can try to provide a PR for that.

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Reactions:13
  • Comments:11 (9 by maintainers)

github_iconTop GitHub Comments

3reactions
wetnebcommented, Apr 30, 2018

@georgepsarakis yes exactly - what I would like to do is add another custom prefix in front of that. For instance, if both Mary and John use the same redis database, they could want to make sure that all the keys they use are prefixed with different strings. So they would want to use mary._kombu.binding.<QUEUE_NAME> and john._kombu.binding.<QUEUE_NAME> respectively.

In other words, it would be useful if the _kombu.binding.%s pattern could be altered via the broker_transport_options configuration setting.

This quite common practice: for instance, when using Redis with Django for caching, you can specify such a prefix with KEY_PREFIX:

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient"
        },
        "KEY_PREFIX": "example"
    }
}
1reaction
wetnebcommented, Dec 14, 2020

This is not solved, my PR did not make it. Feel free to have a go at it!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Custom prefix for redis keys with Celery - Stack Overflow
How can I configure Celery to prefix all the keys it uses with a custom string? The docs mention ways to add prefixes...
Read more >
Redis data types tutorial
Learning the basic Redis data types and how to use them.
Read more >
Redis-specific parameters - Amazon ElastiCache for Redis
If you do not specify a parameter group for your Redis cluster, then a default parameter group appropriate to your engine version will...
Read more >
Configuration | StackExchange.Redis
Because there are lots of different ways to configure redis, StackExchange. ... tiebreaker={string}, TieBreaker, __Booksleeve_TieBreak, Key to use for ...
Read more >
RedisCacheConfiguration (Spring Data Redis 3.0.0 API)
Add a Converter for extracting the String representation of a cache key if no suitable Object.toString() method is present. ... Use the given...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found