Materialization Errors for Float and Double Lists

Expected Behavior

1-hot encoded features should be materializeable to the online store without error.

Current Behavior

Version 0.10 apparently introduced a regression where 1-hot encoded feature materialization throws errors.

Steps to reproduce

from datetime import datetime, timedelta
from feast import Entity, Feature, FeatureStore, FeatureView, FileSource, ValueType
import numpy as np
import pandas as pd
from pathlib import Path
df = pd.DataFrame(columns=['device_type', 'encoding', 'datetime'])
df['device_type'] = ['A', 'B']
df['encoding'] = [
    np.array([0.0, 1.0], dtype=np.float32),
    np.array([1.0, 0.0], dtype=np.float32)
]
df['datetime'] = pd.to_datetime([datetime.now(), datetime.now()])
device_encodings_path_str = 'data/device_encodings.parquet'
device_encodings_path = Path(device_encodings_path_str)
device_encodings_abs_path_str = str(device_encodings_path.absolute())
df.to_parquet(device_encodings_abs_path_str)
device_encodings = FileSource(path=device_encodings_abs_path_str,
                              event_timestamp_column='datetime')
device_type = Entity('device_type',
                     value_type=ValueType.STRING,
                     description='')
device_encoding_view = FeatureView(
    name="device_encoding",
    entities=["device_type"],
    features=[
        Feature(name="encoding", dtype=ValueType.FLOAT_LIST),
    ],
    ttl=timedelta(minutes=60),
    input=device_encodings,
    online=True,
)
fm = FeatureStore(repo_path=".")
fm.apply([device_type, device_encoding_view])
fm.materialize(start_date=datetime(2021, 1, 1), end_date=datetime.now())

The above code yields the below error output:

Materializing 2 feature views from 2021-01-01 00:00:00-08:00 to 2021-06-10 15:24:51-07:00 into the sqlite online store.
driver_hourly_stats:
100%|███████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 2920.42it/s]
device_encoding:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/anaconda3/envs/feast/lib/python3.8/site-packages/feast/telemetry.py", line 155, in exception_logging_wrapper
    result = func(*args, **kwargs)
  File "/usr/local/anaconda3/envs/feast/lib/python3.8/site-packages/feast/feature_store.py", line 445, in materialize
    provider.materialize_single_feature_view(
  File "/usr/local/anaconda3/envs/feast/lib/python3.8/site-packages/feast/infra/local.py", line 200, in materialize_single_feature_view
    rows_to_write = _convert_arrow_to_proto(table, feature_view, join_keys)
  File "/usr/local/anaconda3/envs/feast/lib/python3.8/site-packages/feast/infra/provider.py", line 301, in _convert_arrow_to_proto
    value = python_value_to_proto_value(row[idx], feature.dtype)
  File "/usr/local/anaconda3/envs/feast/lib/python3.8/site-packages/feast/type_map.py", line 339, in python_value_to_proto_value
    return _python_value_to_proto_value(value_type, value)
  File "/usr/local/anaconda3/envs/feast/lib/python3.8/site-packages/feast/type_map.py", line 225, in _python_value_to_proto_value
    val=[
  File "/usr/local/anaconda3/envs/feast/lib/python3.8/site-packages/feast/type_map.py", line 228, in <listcomp>
    else _type_err(item, np.float64)
  File "/usr/local/anaconda3/envs/feast/lib/python3.8/site-packages/feast/type_map.py", line 191, in _type_err
    raise ValueError(f'Value "{item}" is of type {type(item)} not of type {dtype}')
ValueError: Value "0.0" is of type <class 'float'> not of type <class 'numpy.float64'>

Specifications

• Version: 0.10.7
• Platform: local mode
• Subsystem: Python SDK

[karlhigley]

@woop 👍🏻

It’s worth mentioning that empty lists also seem to be problematic with the type map.

[woop]

Thanks @karlhigley. We’ll need to add a more comprehensive test to get a better idea of our type coverage.