How to choose proper download_mode in function load_dataset?

Hi, I am a beginner to datasets and I try to use datasets to load my csv file. my csv file looks like this

text,label
"Effective but too-tepid biopic",3
"If you sometimes like to go to the movies to have fun , Wasabi is a good place to start .",4
"Emerges as something rare , an issue movie that 's so honest and keenly observed that it does n't feel like one .",5

First I try to use this command to load my csv file .

dataset=load_dataset('csv', data_files=['sst_test.csv'])

It seems good, but when i try to overwrite the convert_options to convert ‘label’ columns from int64 to float32 like this.

import pyarrow as pa
from pyarrow import csv
read_options = csv.ReadOptions(block_size=1024*1024)
parse_options = csv.ParseOptions()
convert_options = csv.ConvertOptions(column_types={'text': pa.string(), 'label': pa.float32()})
dataset = load_dataset('csv', data_files=['sst_test.csv'], read_options=read_options,
                       parse_options=parse_options, convert_options=convert_options)

It keeps the same:

Dataset(features: {'text': Value(dtype='string', id=None), 'label': Value(dtype='int64', id=None)}, num_rows: 2210)

I think this issue is caused by the parameter “download_mode” Default to REUSE_DATASET_IF_EXISTS because after I delete the cache_dir, it seems right.

Is it a bug? How to choose proper download_mode to avoid this issue?

[lhoestq]

It’s no big deal, but since it can be confusing to users I think it’s worth renaming it, and deprecate GenerateMode until datasets 2.0 at least. IMO it’s confusing to have download_mode=GenerateMode.something

[lhoestq]

download_mode=datasets.GenerateMode.FORCE_REDOWNLOAD should work. This makes me think we we should rename this to DownloadMode.FORCE_REDOWNLOAD. Currently that’s confusing