Avoid problem with length limit of 5000 in MLflow

Motivation

When using the MLflowCallback it is possible that the user added user_attrs (like lists of float) that are longer then 5000 characters when converted to a str. This causes problems with MLflow which limits the length to 5000. See example below:

[...]
  File "/home/smay/miniconda3/envs/py38/lib/python3.8/site-packages/mlflow/utils/validation.py", line 136, in _validate_length_limit
    raise MlflowException(
mlflow.exceptions.MlflowException: Tag value '[0.8562690322984875, 0.8544098885636596, 0.8544098885636596, 0.8544098885636596, 0.8544098885636596, 0.859181214773054, 0.86273086038245, 0.86273086038245, 0.86273086038245, 0.86273086038245, 0.86273086038245, 0.8562690322984875, 0.8544098885636596, ' had length 5276, which exceeded length limit of 5000

Description

I suggest to check the strings if they are len() > 5000 and cut them if needed. If this happens a warning should be printed.

I can provide a PR if wanted. Just give me feedback to my proposal please.

[PhilipMay]

I asked the MLflow guys if they want to fix this issue on their side: https://github.com/mlflow/mlflow/issues/2931

[PhilipMay]

It’d be good to know if this number 5000 can be retrieved via some public API but I guess it’s not a must.

I will have a look into the MLflow code to retrieve the 5000 limit.

would you mind briefly explaining why you need more than 5000 characters since I suppose is beyond what’s expected by mlflow.

Sometimes I am logging a list of feature importances (SHAP values) to the user attr of optuna. Since I have 50000 features (bioinformatics) it is a list of 50000 floats. That is longer then 5000 characters when converted to a string. But there are also other less extreme cases where I want to log the individual results of a nested cross validation.