calling handler.close() from different thread causes deadlock when using AzureLogHandler

Describe your environment. Python 3.8.13 on Linux, opencensus-ext-azure == 1.1.4, azureml-sdk[automl] ~= 1.42.0, azureml-mlflow==1.42.0, mlflow-skinny==1.26.0

Steps to reproduce. I can’t give an exact test code. The problem is related to #1112. We add an AzureLogHandler on the main thread. Then later we run automl Experiment in multiple threads.

The code in the thread at some point runs import mlflow, and the program stucks.

The import causes to run _configure_mlflow_loggers(root_module_name=__name__), which then runs the logging.config.dictConfig

This runs _clearExistingHandlers in logging.config, and inside of this, the shutdown function runs. The shutdown runs the following calls for all defined logging handlers:

   h.acquire()
   h.flush()
   h.close()

With previous versions, for example opencensus-ext-azure == 1.1.3 the program stuck at the h.flush() call. With the 1.1.4 version the h.flush() finishes, but then the run still stops when the self._worker.stop(timeout) is called inside the h.close().

What is the expected behavior? The import finishes successfully and the program continues

What is the actual behavior? The program stops in deadlock

[jorgenader]

@jorgenader Are you referring to this?

Yes, but more like adding default timeout instead.

[lzchen]

@csadam Ideally, we would use the sentinel pattern as described here to avoid breaking changes, and then remove the custom atexit hook in place of logging shutdown.