gsutil cp hangs on many small files when running in parallel

I have a GCS bucket with millions of small files in different folders. When I run:

$ gsutil -m cp -r gs://my-bucket .

The process will eventually hang before completion, sometimes after 5 minutes and sometimes after several hours. This seems to be is 100% reproducible. I’m using version 4.27 but this has happened in older versions as well. As a workaround have I to use:

$ gsutil cp -r gs://my-bucket .

which works but it takes several days to download everything so it’s not optimal.

[obrie]

For what it’s worth, I found that using only threads for parallelization (and not child processes) appears to avoid the underlying deadlock here. e.g. -o GSUtil:parallel_process_count=1 -o GSUtil:parallel_thread_count=24

[vkaul11]

I am still facing the issue, it reaches 99% of copied files and then terminates.