weird pipeline() combined with watch() behavior

I understand it’s intended and documented, still it’s seems so inconvenient, illogical and prohibits efficient use of pipelining (not MULTI).

Why pipe suddenly breaks when I call .watch()? It is still kind of pipe, but it doesn’t work as one. Maybe, old 2.2 approach is not the best, but it didn’t cause this confusion. I used .pipeline(transaction=false) to make pipeline and with a flag on to make atomic pipeline. Cluttering everything in on method is questionable but concept is clear. Now, we still use .pipeline() to get atomic pipeline until we don’t use .watch() when we suddenly loose pipelining and using .multi() directly to get atomic, which makes things with MULTI very inconsistent.

My use case, implemented for redis-py 2.2:

# set up pipeline not for multi, but to avoid unneeded network roundtrips
pipe = redis_conn.pipeline(transaction=False)  
pipe.watch(version_key, *conjs_keys)
# get something from redis
pipe.get(...)
pipe.sunion(...)
...
# ignore watch() result, gather all other
_, result1, result2, ... = pipe.execute() 

if <some condition involving result*>:
    redis_conn.unwatch()
else:
    try:
        txn = redis_conn.pipeline()
        # Change something in redis
        txn.delete(...)
        txn.execute()
    except WatchError:
        <Optimistic locking failed: just redo everything.>

Actual working example is here https://github.com/Suor/django-cacheops/blob/master/cacheops/invalidation.py#L111

[roncohen]

Sorry to revisit this old thread, but is there a way to buffer up those three commands that @Suor mentioned? I’m trying to get a lot of data from watched keys, and would like to avoid those roundtrips. I also tried to start the overall pipeline with pipeline(transaction=false) but it does not seem to be the way to go:

with redis.pipeline(transaction=False) as pipe:
    pipe.watch(*keys)
    for key in keys:
        pipe.lrange(key, 0, -1)

    data = pipe.execute()

    pipe.multi()
    # set stuff back, then:
    pipe.execute()

I get ResponseError: DISCARD without MULTI

[github-actions[bot]]

This issue is marked stale. It will be closed in 30 days if it is not updated.