Custom feature types in `load_dataset` from CSV

I am trying to load a local file with the load_dataset function and I want to predefine the feature types with the features argument. However, the types are always the same independent of the value of features.

I am working with the local files from the emotion dataset. To get the data you can use the following code:

from pathlib import Path
import wget

EMOTION_PATH = Path("./data/emotion")
DOWNLOAD_URLS = [
    "https://www.dropbox.com/s/1pzkadrvffbqw6o/train.txt?dl=1",
    "https://www.dropbox.com/s/2mzialpsgf9k5l3/val.txt?dl=1",
    "https://www.dropbox.com/s/ikkqxfdbdec3fuj/test.txt?dl=1",
]

if not Path.is_dir(EMOTION_PATH):
     Path.mkdir(EMOTION_PATH)
for url in DOWNLOAD_URLS:
     wget.download(url, str(EMOTION_PATH))

The first five lines of the train set are:

i didnt feel humiliated;sadness
i can go from feeling so hopeless to so damned hopeful just from being around someone who cares and is awake;sadness
im grabbing a minute to post i feel greedy wrong;anger
i am ever feeling nostalgic about the fireplace i will know that it is still on the property;love
i am feeling grouchy;anger

Here the code to reproduce the issue:

from datasets import Features, Value, ClassLabel, load_dataset

class_names = ["sadness", "joy", "love", "anger", "fear", "surprise"]
emotion_features = Features({'text': Value('string'), 'label': ClassLabel(names=class_names)})
file_dict = {'train': EMOTION_PATH/'train.txt'}

dataset = load_dataset('csv', data_files=file_dict, delimiter=';', column_names=['text', 'label'], features=emotion_features)

Observed behaviour:

dataset['train'].features

{'text': Value(dtype='string', id=None),
 'label': Value(dtype='string', id=None)}

Expected behaviour:

dataset['train'].features

{'text': Value(dtype='string', id=None),
 'label': ClassLabel(num_classes=6, names=['sadness', 'joy', 'love', 'anger', 'fear', 'surprise'], names_file=None, id=None)}

Things I’ve tried:

• deleting the cache
• trying other types such as int64

Am I missing anything? Thanks for any pointer in the right direction.

[lewtun]

Thanks a lot for the PR and quick fix @lhoestq!

[lhoestq]

Currently csv doesn’t support the features attribute (unlike json). What you can do for now is cast the features using the in-place transform cast_

from datasets import load_dataset

dataset = load_dataset('csv', data_files=file_dict, delimiter=';', column_names=['text', 'label'])
dataset.cast_(emotion_features)