question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Unable to transform test data after MCA fitting the training data

See original GitHub issue
data = pd.read_csv("data/training set.csv")
X = data.loc[:, 'OS.1':'DSA.1']

df = pd.DataFrame(X)

mca = prince.MCA(
               n_components=2,
               n_iter=3,
               copy=True,
               check_input=True,
               engine='auto',
               random_state=42
                )

mca = mca.fit(df)

df_new = df.loc[0:5, :]
I = mca.transform(df_new)
print(I)

Output: File “C:/…/clustering/k means.py”, line 62, in <module> I = mca.transform(df_new) File “C:..\clustering\interpreter2\lib\site-packages\prince\mca.py”, line 47, in transform return self.row_coordinates(X) File “C:..\clustering\interpreter2\lib\site-packages\prince\mca.py”, line 37, in row_coordinates return super().row_coordinates(self.one_hot_.transform(X)) File “C:..\clustering\interpreter2\lib\site-packages\prince\ca.py”, line 111, in row_coordinates X = X / X.sum(axis=1) File “C:\python36\lib\site-packages\scipy\sparse\base.py”, line 1015, in sum np.ones((n, 1), dtype=res_dtype)) File “C:\python36\lib\site-packages\scipy\sparse\base.py”, line 499, in mul result = self._mul_vector(np.ravel(other)) File “C:\python36\lib\site-packages\scipy\sparse\coo.py”, line 571, in _mul_vector other.dtype.char)) File “C:\python36\lib\site-packages\scipy\sparse\sputils.py”, line 60, in upcast_char t = upcast(*map(np.dtype, args)) File “C:\python36\lib\site-packages\scipy\sparse\sputils.py”, line 52, in upcast raise TypeError(‘no supported conversion for types: %r’ % (args,)) TypeError: no supported conversion for types: (dtype(‘O’), dtype(‘O’))

This is how data looks like

print(df_new)

output:
    0  1   2   3
0  9  8   9   9
1  8  7   8   6
2  8  7   9   9
3  8  7   9   9
4  8  7   8   7
5  9  8  10  10

python 3.6.4 scikit 0.20.2 numpy 1.16.1 pandas 0.24.1

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:11 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
MaxHalfordcommented, Mar 14, 2019

@adeebabdulsalam can you see if the latest version fixes your issue (if you have time of course)?

0reactions
MaxHalfordcommented, Mar 15, 2019

Great! Sorry this took so much time. Have a very nice day!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Unable to transform test data after MCA fitting training data #107
I'm unable to transform a test dataset after MCA is fit on a training dataset. It seems to be a shape issue. Here...
Read more >
Is it acceptable not to transform() test data after train data is ...
No, it does not make sense to do this. You model has learned how to map one input space to another, that is...
Read more >
Use "fit_transform" on training data, but "transform ... - YouTube
Use "fit_transform" on training data, but " transform " (only) on testing /new data.Applies the same transformations to both sets of data, ...
Read more >
Train and Test datasets in Machine Learning - Javatpoint
The training data is the biggest (in -size) subset of the original dataset, which is used to train or fit the machine learning...
Read more >
When scale the data, why the train dataset use 'fit' and ...
Since fit_transform() is already computing and transforming the training data only transformation for testing data is left,since parameter ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found