Hive Connector: Ignore empty files when scanning files

Currently if the table(external table) are created as format e.g. Parquet, and there is an empty file(zero length) in the directory, an error will occur:

oss://xxxxxx/xxxx/empty_file is not a valid Parquet File

For the ease of use, we can safely skip all the empty files to avoid this error, how do you guys think? (The empty file use case will not occur if the files are managed by Hive, but could occur if the files are uploaded by user or any other program, when using Object Store Services).

[zhenxiao]

hi @xumingming @mbasmanova I feel if a Hive connector table with metadata specified as Parquet format, all files should be Parquet format. empty files seems invalid cases. Am not convinced to have ParquetReader skip non-parquet-empty files. What do you think?

[xumingming]

OK, sounds good to me 😃