Ray generating core dump on Program abort

When program/script abort while running ray processes. It’s generating core dump. Please follow the below procedure to re-create issue @rkooo567

Underlying OS: NAME=“Red Hat Enterprise Linux Server” VERSION=“7.5 (Maipo)”

sample program to re-create issue “ray_core_dump_issue.py”

import ray
import time, datetime
import sys, os

@ray.remote
def sample_check(batch):
    print(f"Batch:{batch}")
    a=10
    b = a + a
    time.sleep(10)
    print(b)
    return


def split_proc(proc_cnt, batch_size):
    start_time = time.time()
    ray.init(dashboard_host="0.0.0.0",num_cpus=proc_cnt)
    total_batches = batch_size
    ret_ids = []
    while total_batches!=0:
        total_batches = total_batches -1
        print(f"Batch Number:{total_batches}")
        red_id= sample_check.remote( total_batches)
        ret_ids.append(red_id)
    ret = ray.get(ret_ids)
    ray.shutdown()

if __name__ == "__main__":
    split_proc(8, 12)


1) Pull the docker image
docker pull subbareddy425/python36-ray:1.0.0

2) Start the Docker container
 docker run -d -it --name ray-core-issue subbareddy425/python36-ray:1.0.0 bash

3) Exec console to docker container
  docker exec -it ray-core-issue bash
  
4) Run the python program in the directory /apps/python_scripts/

cd /apps/python_scripts/
python3 ray_core_dump_issue.py

5) it will create core dump in the current directory program ran

bash-4.2# ls -ltrh
total 52M
-rw-r----- 1 root root  649 Nov 30 19:31 ray_core_dump_issue.py
-rw------- 1 root root 153M Nov 30 19:47 core.41

[subbareddydagumati]

Adding —ulimit -c 0 to docker run worked. No more core dumps.

[rkooo567]

https://discuss.ray.io/t/how-to-abort-a-driver-program-elegant/152/2 Can you try this?