a strange bug in sklearn.preprocessing.OneHotEncoder() when transform unknown string with handle_unknown='ignore'

Describe the bug

Using sklearn.preprocessing.OneHotEncoder() with parameter handle_unknown=‘ignore’ will get a wrong output, when I manually set categories_ && all objects are of string type && an unknown category is encountered during transform.

Steps/Code to Reproduce

import numpy as np
from sklearn.preprocessing import OneHotEncoder
import pandas as pd
onehot = OneHotEncoder(handle_unknown='ignore')
onehot.categories_=[np.array(['bread', 'milk', 'diaper', 'beer'])for i in range(3)]
output = onehot.transform([['bread', 'milk', 'none'],['bread', 'diaper', 'beer']]).toarray()
print(output)

Expected Results

[[1. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0.] [1. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 1.]]

Actual Results

[[1. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0.] [1. 0. 0. 0. 0. 0. 1. 0. 1. 0. 0. 0.]]

Versions

System: python: 3.7.6 (default, Jan 8 2020, 20:23:39) [MSC v.1916 64 bit (AMD64)] executable: D:\ProgramData\Anaconda3\python.exe machine: Windows-10-10.0.19041-SP0

Python dependencies: pip: 21.0.1 setuptools: 45.2.0.post20200210 sklearn: 0.22.1 numpy: 1.16.0 scipy: 1.4.1 Cython: 0.29.15 pandas: 0.25.0 matplotlib: 3.1.0 joblib: 0.14.1

Built with OpenMP: True

[thomasjpfan]

sklearn’s OneHotEncoder API currently requires fit to be called. In your case:

import numpy as np
from sklearn.preprocessing import OneHotEncoder
import pandas as pd

categories = [np.array(['bread', 'milk', 'diaper', 'beer'])for i in range(3)]
onehot = OneHotEncoder(handle_unknown='ignore', categories=categories,
                       sparse=False)
output = onehot.fit_transform(
    [['bread', 'milk', 'none'],
     ['bread', 'diaper', 'beer']])
print(output)
# [[1. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0.]
#  [1. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 1.]]

In principle, we could design OneHotEncoder to be more stateness and not require fit.

[glemaitre]

So there is no bug then. I am closing the issue. We could convert it to a discussion I think?