Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Performance of pytorch-tabnet on forest cover type dataset

See original GitHub issue

Running out of the box the forest_example, the results differ significantly from the ones in the original paper. Specifically, I get the following:

preds = clf.predict_proba(X_test)
y_true = y_test
test_acc = accuracy_score(y_pred=np.argmax(preds, axis=1), y_true=y_true)
print(f"BEST VALID SCORE FOR {dataset_name} : {clf.best_cost}")
BEST VALID SCORE FOR EPIGN : -0.8830427851320214

print(f"FINAL TEST SCORE FOR {dataset_name} : {test_acc}")
FINAL TEST SCORE FOR EPIGN : 0.0499728922661205

Do you get similar results? Many thanks.

Issue Analytics

State:
Created 4 years ago
Comments:25 (1 by maintainers)

Top GitHub Comments

3reactions

carefree0910commented, Feb 8, 2020

It works like a charm now with v1.0.3, the results I’ve got are:

BEST VALID SCORE FOR forest-cover-type : -0.9637142838702182 FINAL TEST SCORE FOR forest-cover-type : 0.963847749197525

Thanks for the quick replies and quick fixes!

1reaction

Optimoxcommented, Feb 7, 2020

@meechos @carefree0910 new version has been updated to Pypi so you can pip install the latest version and try again, hopefully everything is fixed now!

Top Results From Across the Web

Forest Cover Type | Kaggle

The dataset is taken from 30m by 30m areas of forest that are classified as one of seven cover types: Spruce/Fir; Lodgepole Pine;...

Covertype Data Set - UCI Machine Learning Repository

Predicting forest cover type from cartographic variables only (no remotely sensed data). The actual forest cover type for a given observation (30 x...

Implementing TabNet in PyTorch - Towards Data Science

It was able to achieve state of the art results on several datasets in both regression and classification problems. It combines the features...

Talks # 4: Sebastien Fischman - Pytorch-TabNet ... - YouTube

Talks # 4:Speaker: Sebastien Fischman (https://www.linkedin.com/in/sebastienfischman/)Title : Pytorch-tabnet : Beating XGBoost on tabular ...

Tabnet presentation - SlideShare

PYTORCH-TABNET Beating XGBoost on tabular data with deep learning? ... got best results on diﬀerent datasets" Forest Cover Type: Outperforms XGBoost, ...