Cannot connect to redis server when running ray.init()

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Mac OS X 10.13.6
• Ray installed from (source or binary): binary (pip install ray)
• Ray version: 0.7.0
• Python version: 3.6.7 (python.org distribution installed with pyenv)
• Exact command to reproduce: import ray; ray.init()

Describe the problem

When I try to start ray by running import ray and then ray.init(), ray initially tries to connect to the redis server at 127.0.0.1:X (where X is a random number) with the message:

2019-06-04 16:02:50,770	INFO services.py:409 -- Waiting for redis server at 127.0.0.1:X to respond...

However, this operation never succeeds; after about 30 seconds it fails with the error message:

"ConnectionError: Error 60 connecting to 10.30.1.10:X. Operation timed out."

Note that the error message indicates a different (network) IP address than the initial “connecting” message.

I tried specifying node_ip_address="127.0.0.1" in the ray.init() call (per the ray.init() documentation), but this did not change any of the outputs or error messages.

I cannot specify the redis_address keyword argument in the ray.init() call because the port number is selected randomly each time.

If I try running ps to view the redis process, I can see that it is listening on the correct port and is bound to all addresses:

$ ps aux | grep redi[s]
charles          58564   0.0  0.1  4311924   6484   ??  S     3:59PM   0:00.05 /Users/charles/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/site-packages/ray/core/src/ray/thirdparty/redis/src/redis-server *:X

(where again X is the random port number).

Source code / logs

$ ipython
Python 3.6.7 | packaged by conda-forge | (default, Feb 28 2019, 02:16:08)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import ray

In [2]: ray.init()
2019-06-04 16:02:50,663	INFO node.py:497 -- Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-06-04_16-02-50_661319_58800/logs.
2019-06-04 16:02:50,770	INFO services.py:409 -- Waiting for redis server at 127.0.0.1:53157 to respond...
---------------------------------------------------------------------------
TimeoutError                              Traceback (most recent call last)
~/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/site-packages/redis/connection.py in connect(self)
    491         try:
--> 492             sock = self._connect()
    493         except socket.timeout:

~/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/site-packages/redis/connection.py in _connect(self)
    549         if err is not None:
--> 550             raise err
    551         raise socket.error("socket.getaddrinfo returned an empty list")

~/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/site-packages/redis/connection.py in _connect(self)
    537                 # connect
--> 538                 sock.connect(socket_address)
    539

TimeoutError: [Errno 60] Operation timed out

During handling of the above exception, another exception occurred:

ConnectionError                           Traceback (most recent call last)
<ipython-input-2-3f68a533b944> in <module>
----> 1 ray.init()

~/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/site-packages/ray/worker.py in init(redis_address, num_cpus, num_gpus, resources, object_store_memory, redis_max_memory, log_to_driver, node_ip_address, object_id_seed, local_mode, redirect_worker_output, redirect_output, ignore_reinit_error, num_redis_shards, redis_max_clients, redis_password, plasma_directory, huge_pages, include_webui, driver_id, configure_logging, logging_level, logging_format, plasma_store_socket_name, raylet_socket_name, temp_dir, load_code_from_local, _internal_config)
   1395         # handler.
   1396         _global_node = ray.node.Node(
-> 1397             head=True, shutdown_at_exit=False, ray_params=ray_params)
   1398     else:
   1399         # In this case, we are connecting to an existing cluster.

~/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/site-packages/ray/node.py in __init__(self, ray_params, head, shutdown_at_exit, connect_only)
    141         # Start processes.
    142         if head:
--> 143             self.start_head_processes()
    144             redis_client = self.create_redis_client()
    145             redis_client.set("session_name", self.session_name)

~/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/site-packages/ray/node.py in start_head_processes(self)
    498         assert self._redis_address is None
    499         # If this is the head node, start the relevant head node processes.
--> 500         self.start_redis()
    501         self.start_monitor()
    502         self.start_raylet_monitor()

~/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/site-packages/ray/node.py in start_redis(self)
    352              password=self._ray_params.redis_password,
    353              include_java=self._ray_params.include_java,
--> 354              redis_max_memory=self._ray_params.redis_max_memory)
    355         assert (
    356             ray_constants.PROCESS_TYPE_REDIS_SERVER not in self.all_processes)

~/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/site-packages/ray/services.py in start_redis(node_ip_address, redirect_files, port, redis_shard_ports, num_redis_shards, redis_max_clients, redirect_worker_output, password, use_credis, redis_max_memory, include_java)
    605     primary_redis_client = redis.StrictRedis(
    606         host=node_ip_address, port=port, password=password)
--> 607     primary_redis_client.set("NumRedisShards", str(num_redis_shards))
    608
    609     # Put the redirect_worker_output bool in the Redis shard so that workers

~/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/site-packages/redis/client.py in set(self, name, value, ex, px, nx, xx)
   1449         if xx:
   1450             pieces.append('XX')
-> 1451         return self.execute_command('SET', *pieces)
   1452
   1453     def __setitem__(self, name, value):

~/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/site-packages/redis/client.py in execute_command(self, *args, **options)
    770         pool = self.connection_pool
    771         command_name = args[0]
--> 772         connection = pool.get_connection(command_name, **options)
    773         try:
    774             connection.send_command(*args)

~/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/site-packages/redis/connection.py in get_connection(self, command_name, *keys, **options)
    992         try:
    993             # ensure this connection is connected to Redis
--> 994             connection.connect()
    995             # connections that the pool provides should be ready to send
    996             # a command. if not, the connection was either returned to the

~/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/site-packages/redis/connection.py in connect(self)
    495         except socket.error:
    496             e = sys.exc_info()[1]
--> 497             raise ConnectionError(self._error_message(e))
    498
    499         self._sock = sock

ConnectionError: Error 60 connecting to 10.30.1.10:53157. Operation timed out.

[MarcoAlejandro]

It happened to me running ray on my laptop when I had a VPN activated on my laptop.

[charlesreid1]

The solution was to turn off my wifi hardware, which prevented my computer from connecting to the public wifi network with the firewall. That was why I couldn’t connect to the ray server.