Multi GPUs training raise an error

Single GPU training is normal. But multi gpus training will raise an error below. How can I solve it ?

Traceback (most recent call last):
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 1, in <module>
  File "/home/dl/anaconda3/lib/python3.6/multiprocessing/spawn.py", line 105, in spawn_main
  File "/home/dl/anaconda3/lib/python3.6/multiprocessing/spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
    exitcode = _main(fd)
  File "/home/dl/anaconda3/lib/python3.6/multiprocessing/spawn.py", line 115, in _main
  File "/home/dl/anaconda3/lib/python3.6/multiprocessing/spawn.py", line 115, in _main
    self = reduction.pickle.load(from_parent)
    self = reduction.pickle.load(from_parent)
MemoryError
MemoryError
Traceback (most recent call last):
  File "/home/dl/anaconda3/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/dl/anaconda3/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/dl/anaconda3/lib/python3.6/site-packages/torch/distributed/launch.py", line 235, in <module>
    main()
  File "/home/dl/anaconda3/lib/python3.6/site-packages/torch/distributed/launch.py", line 231, in main
    cmd=process.args)
subprocess.CalledProcessError: Command '['/home/dl/anaconda3/bin/python', '-u', './tools/train.py', '--local_rank=0', 'configs/cascade_rcnn_x101_64x4d_fpn_1x.py', '--launcher', 'pytorch']' died with <Signals.SIGKILL: 9>

[ZenFSheng]

80G memory is consumed with 8*2 workers, so 64G may be enough for 4 GPUs. However, if you plan to train on larger datasets, maybe you need even larger memory. BTW, Fast R-CNN takes more memory than end-to-end Faster R-CNN, sometimes it can eat 200G memory or more.

😮 Oh my~ Thank you for your reply，that really helps a lot.🙂

[hellock]

How much memory does your server have?