PID check failed. RNG must be re-initialized after fork(). Hint: Try Random.atfork()

PID check failed. RNG must be re-initialized after fork(). Hint: Try Random.atfork()

gsutil exits with this message when I’m trying to copy a lot of large files to google storage:

 gsutil -m cp -r /mnt/some/src gs://some/dest

It’s around 5000 files which vary in size. Their total size is around 100GB. Transfer sometimes succeeds and sometimes fails with the message above.

[houglum]

So, I think most folks don’t see this because it happens under a subset of runtime circumstances. We use oauth2client, which has mechanisms in place to fall back to “pure Python crypto” in the absence of pyOpenSSL – additionally, I believe this only happens when using service account credentials. Fun.

From the PyCrypto docs, I think the fix here is to call Crypto’s Random.atfork() method (if Crypto is installed and can be imported) in the parent and child every time we spawn a new process, whether that be with os.fork(), multiprocessing.Process(), etc.:

Caveat: For the random number generator to work correctly, you must call Random.atfork() in both the parent and child processes after using os.fork()

[jmcabandara]

I got the error as follows.

Failed RNG.