Cannot find the saved native string model though SUCCSESS files are created
See original GitHub issueDescribe the bug I trained a mmlspark.lightgbm regressor which works greats and generates outputs on the test set. When saving it in a local directory on an azure/k8s pod using the following command:
reg_fitted.saveNativeModel(‘./savemodel/model02/’,overwrite=True)
I do not get the txt file containing the model parameters. I can see the content of that directory which has the _SUCCESS file, but cannot find the txt file that contains the model itself:
ls savemodel/model02/ -la
total 12 drwxr-xr-x 2 root root 4096 Jan 6 22:37 . drwxr-xr-x 4 root root 4096 Jan 6 22:37 … -rw-r–r-- 1 root root 8 Jan 6 22:37 ._SUCCESS.crc -rw-r–r-- 1 root root 0 Jan 6 22:37 _SUCCESS
To Reproduce Not sure how you can exactly reproduce this issue since no error is generated but nothing is saved either.
Expected behavior I expected to see a txt file in the same directory that describes the model. I have done it before with success on azure/databricks cluster, but it seems weird on an azure/k8s deployment.
Info (please complete the following information):
- MMLSpark Version: [e.g. v0.18.1]
- Spark Version [e.g. 2.4.4]
- Spark Platform [e.g. Azure/k8s]
** Stacktrace**
Please post the stacktrace here if applicable
If the bug pertains to a specific feature please tag the appropriate CODEOWNER for better visibility
Additional context Add any other context about the problem here.
AB#1984591
Issue Analytics
- State:
- Created 3 years ago
- Comments:6
Top GitHub Comments
FYI this is what I ended up doing:
I think the output of get_dump() in the drive is exactly what I need - only not sure if there’s a better way
Maybe let me put it this way. I want to train a lightGBM with mmlspark/pyspark on a Spark cluster but save/extract the native model and save the binary (lgb.Booster.save_model) locally for future usage.
How should I approach this?