Multiindex Lightgbm problem

I’m not sure whether the problem arises on evalml side or Lightgbm but I have a problem with multiindex X passing to AutoMLSearch.search()

Batch 1: (4/9) LightGBM Regressor w/ Imputer            Elapsed:00:01
	Starting cross validation
			Fold 0: Encountered an error.
			Fold 0: All scores will be replaced with nan.
			Fold 0: Please check ...\evalml_debug.log for the current hyperparameters and stack trace.
			Fold 0: Exception during automl search: logical_types contains columns that are not present in dataframe: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]
			Fold 1: Encountered an error...

Sorry, I can’t give you a proper example. I still think this is better than nothing.

On my side, I fixed the problem with

X_train.pipe(lambda df: df.set_axis(['_'.join(col).strip() for col in df.columns.values], axis=1))

In addition, being still a novice in using the library, but I can’t find a proper way to see logs of errors. I see the same information in evalml_debug.log without a clear traceback of the error. This is why I don’t know the exact reason for the problem.

[angela97lin]

@grayskripko We recently released a new version of EvalML (0.18.2) which introduces a fix for MultiIndex issues. Please try it out and let us know if this fixes your original issue! 😁

[angela97lin]

Hi @grayskripko, thank you for filing! RE a proper way to surface the traceback of the error, try:

from evalml.automl.callbacks import raise_error_callback
automl = AutoMLSearch(..., error_callback=raise_error_callback)
automl.search()

Having that traceback could be helpful for us to understand what is happening and better assist you 😄