Thread-safety problem in RoundRobinJedisPool

First of all, issue raised up from io.codis.jodis:0.4.1. We have frequently ran into exception Pool not open which threw by RedisPool

RoundRobinJedisPool.resetPools is not thread-safety. Although RoundRobinJedisPool#pools is carefully declared volatile and ImmutableList, the inner pool (thus, JedisPool) can still be modified. At the end of resetPools, it starts closing pool(s) that not exists under zookeeper path:

        for (PooledObject pool: addr2Pool.values()) {
            LOG.info("Remove proxy: " + pool.addr);
            pool.pool.close();
        }

this operation may close pool(s) already inited by other thread (and successfully assigned to RoundRobinJedisPool#pools) by mistake. For example:

• pools holds JedisPool: a, b, c
• Thread A reads a, b, c from zookeeper
• Thread B reads a, b from zookeeper
• Thread B build new list instance, then assign to pools (anyway it loses the competition)
• Thread A build new list instance (note that reference of a, b, c were copied), then assign to pools
• Thread B start closing resources c

This makes pools contains a closed JedisPool c.

[Apache9]

Is it possible that two threads want to call resetPools concurrently? IIRC, the curator will always uses a single threaded thread pool to execute the callbacks. If not, then we need to add a synchronized modifier to the resetPools method.

[Apache9]

Please try 0.5.1.