Segfault of some algorithms on cluster

Hi,

I am trying to run all the algorithms on the TwoArmTransport environment, and I ran into Segmentation issue when trying td3_bc, bcq and cql on our school’s cluster (with GeForce GTX 1080 with 8120 MB memory). Here is an example of the segmentation fault when running the td3_bc algorithm on the low_dim dataset. I tried to investigate a little bit, but it’s not clear to me what is causing this segfault issue (I’ve attached the error message below from the terminal). There is no such issue if I run these algorithms on my own laptop. It would be great if there are solutions to the segfault so that I can run my experiments on the cluster. Thanks a lot in advance.

SequenceDataset (
	path=robomimic_data/low_dim.hdf5
	obs_keys=('object', 'robot0_eef_pos', 'robot0_eef_quat', 'robot0_gripper_qpos')
	seq_length=1
	filter_key=none
	frame_stack=1
	pad_seq_length=True
	pad_frame_stack=True
	goal_mode=none
	cache_mode=all
	num_demos=200
	num_sequences=93752
)

 10%|#         | 519/5000 [00:28<04:03, 18.43it/s]Segmentation fault (core dumped)

[amandlek]

You can go ahead and monitor it in this function here.

[amandlek]

Closing this issue for now - please re-open if this issue persists