[Bug] `publish_error_to_driver` is broken in the master

Search before asking

• I searched the issues and found no similar issues.

Ray Component

Ray Core

What happened + What you expected to happen

I think this is a bug introduced after GCS pubsub is introduced. But it is also possible the regression happened before.

publish_error_to_driver seems to be broken in the master. Please look at https://github.com/ray-project/ray/pull/22172 for more details.

• I added an artificial error to log_monitor.py to be crashed,
• log_monitor.py should publish the error to the driver using publish_error_to_driver.

But the driver didn’t get the error message although this code block was called. I could reproduce this by ray.init() inside ipython.

cc @mwtian

Versions / Dependencies

Master

Reproduction script

Follow the description.

Anything else

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

[mwtian]

The behavior from https://github.com/ray-project/ray/pull/22172 is not a real issue. The exception in log_monitor.py is raised before driver subscribes to the errors. If a 3s sleep is added before the exception, the exception will be printed at the driver.

[zhe-thoughts]

@rkooo567 Could you help determine if this issue is still a valid issue and if so assign a priority? Thanks