[Core] [Bug] ray.util.multiprocessing.Pool slower than multiprocessing.Pool

Search before asking

• I searched the issues and found no similar issues.

Ray Component

Others

What happened + What you expected to happen

I checked the efficiency of ray.util.multiprocessing.Pool and found that locally it is slower than multiprocessing.Pool I tested with Ray 1.6.0 Python 3.6.9 linux.

Code: `

import os import time import numpy as np from ray.util.multiprocessing import Pool as RPool from multiprocessing import Pool as MPool import ray

def f(index: int): return np.log2(index + 1) + np.log10(index + 1)

if name == ‘main’:

  ray.init()
  s = time.time()
  pool = RPool()
  results = list(pool.map(f, range(1000)))
  print(len(results))
  print('RPool', time.time() - s)
  ray.shutdown()

  s = time.time()
  pool = MPool()
  results = list(pool.map(f, range(1000)))
  print(len(results))
  print('MPool', time.time() - s)

`

Output was:
1000 RPool 0.8084681034088135 1000 MPool 0.03540182113647461

Versions / Dependencies

Ray 1.6.0 Python 3.6.9 linux.

Reproduction script

import os import time

import numpy as np from ray.util.multiprocessing import Pool as RPool from multiprocessing import Pool as MPool

import ray

def f(index: int): return np.log2(index + 1) + np.log10(index + 1)

if name == ‘main’: ray.init() s = time.time() pool = RPool() results = list(pool.map(f, range(1000))) print(len(results)) print(‘RPool’, time.time() - s) ray.shutdown()

s = time.time()
pool = MPool()
results = list(pool.map(f, range(1000)))
print(len(results))
print('MPool', time.time() - s)

Anything else

Every run.

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

[yogeveran]

Ray version 1.9.1 and 2.0.0.dev0 still suffers with a ~4 slowdown (ran on google colab). Python 3.7.12

[stale[bot]]

Hi again! The issue will be closed because there has been no more activity in the 14 days since the last message.

Please feel free to reopen or open a new issue if you’d still like it to be addressed.

Again, you can always ask for help on our discussion forum or Ray’s public slack channel.

Thanks again for opening the issue!