Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Interference between Neptune and Ray causes processes to hang

See original GitHub issue

Hello everyone! We’re starting to use neptune to log a bunch of metrics related to optimisation runs using evolutionary computing techniques. We use the Ray framework to parallelise large parts of our code.

We noticed that our scripts would hang fairly often when using Ray. When running an experiment, all the data is uploaded to neptune, and the experiment is marked as completed, but the process ahngs and does not exit. This happened consistently on different local machines (Mac OS 11, Ubuntu) and on different k8s pods.

After a lot of digging around, we noticed that the issue was resolved by setting the upload_stderr argument to False when calling the create_experiment function.

This is not a huge deal for us, but I am just letting you know in case there is an easy fix!

Issue Analytics

State:
Created 3 years ago
Comments:9 (5 by maintainers)

Top GitHub Comments

1reaction

thatGreekGuy96commented, Feb 1, 2021

I’ll try to get that for you asap!

0reactions

pkasprzykcommented, Aug 25, 2021

Thanks for replying, and I’m very happy your experience is positive now 😃

Closing the issue – though don’t hesitate to reach out in the future, of course!

Top Results From Across the Web

Astronomy Test #2 Flashcards - Quizlet

Discuss several disadvantages of refractor versus reflector telescopes. Answer: Refractors must use lenses, and thus all suffer from some chromatic aberration; ...

Science & Technology - Lick Observatory

Because the atmosphere is not uniform, pockets of varying density air bend (refract) the light from observed objects. This causes the ...

Why NASA's James Webb Space Telescope Matters So Much

The James Webb Space Telescope has the potential to rewrite the history of the cosmos and reshape humanity's position within it.

Troubleshooting Hanging or Looping Processes

A hang can occur for many reasons but often stems from a deadlock in application code, API code, or library code. A hang...

8th Grade Physical Science - Stockton Unified School District

One newton is the amount of force that causes a mass of 1 kilogram to ... When an object hangs from the scale,...