Missing `fit` feature

In any ML task, the assumption is that the test data are not available during training and just available in the prediction phase.

Assume someone wants to categorize reviews using tfidf + Naive Bayes. The required step would be the following ones:

1. Split train and test
2. Fit tfidf on the train part and generate (the transform part in scikit-learn) the tfidf values on the train part
3. Train the model
4. Generate the tf-idf values on the test part, this time using the already fitted model

The problem is that with the current implementation we don’t have any state (and that brings also many advantages such as simplicity). The tfidf functions do not return any already fitted model, rather the already transformed values.

We need to take a clear position wrt to this point. Having the exact same approach as scikit-learn would not probably make sense, still, we need to consider this fact. Opinions?

[mk2510]

so far I didn’t start thinking about the missing fit feature, as fit is in general called on models, which can be fitted to your dataset (at least from my experience so far). In those cases, where we work with models - which can be fitted, like in ‘pca’ or ‘kmeans’ - I think, we can mention it on the getting started page, that we call everytime fit_transform and don’t provide the option to store the fitted pca model. But the way the API is designed, the user probably wouldn’t assume it anyways, I guess 😬 🙈 :octocat:

[harshraj-wadhwani]

@mk2510 Any plans to add fit method ? Or is there any way by which I can fit on train data, save to pickle file, and transform on test data ?