Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
[SUPPORT] Trino hudi query slow when compared to parquet
See original GitHub issueDescribe the problem you faced
Presto query performance with hudi table takes ~2x extra time when compared to parquet for simple query . data stored in s3. hudi metadata store enabled. note, spark sql much faster with hudi dataset.
parquet query first attempt: 23 secs parquet query second attempt: 0.6 secs hudi query first attempt: 40 sec hudi query second attempt: 40 sec data size : 200gb
also, parquet query faster after first run. we are in poc stage to migrate parquet to Hudi data lake. it is one of the blocker. help appreciated.
Environment Description
-
Hudi version : master (4a5683d54a3910e1c860f26b291e1e99931b5949)
-
Spark version : 2.4.0
-
Hive version : 2.3.3
-
Hadoop version :2.8.5
-
Storage (HDFS/S3/GCS…) : S3
-
Running on Docker? (yes/no) : yes, presto runs on k8
Additional context Presto (trino 352) runs on k8. hudi bundle added.
Stacktrace
parquet Query (first attempt)
======================================
presto> select y, m, d, h, count(*) from hive.parquet_warehouse.sample_data where y=2021 and m=2 and d=20 and h between 0 and 4 group by y, m, d, h;
Query 20210227_043723_00000_52ygg, RUNNING, 8 nodes, 6,122 splits
0:23 [ 450M rows, 0B] [19.5M rows/s, 0B/s] [====================================>>> ] 85%
STAGES ROWS ROWS/s BYTES BYTES/s QUEUED RUN DONE
0.........R 0 0 0B 0B 0 16 0
1.......R 4.81K 207 685K 29.6K 0 256 0
2.....R 450M 19.4M 0B 0B 524 80 5246
y | m | d | h | _col4
------+---+----+---+-----------
2021 | 2 | 20 | 4 | 109245879
2021 | 2 | 20 | 0 | 76442859
2021 | 2 | 20 | 1 | 94755366
2021 | 2 | 20 | 2 | 101223162
2021 | 2 | 20 | 3 | 103258290
(5 rows)
Query 20210227_043723_00000_52ygg, FINISHED, 8 nodes
Splits: 6,183 total, 6,183 done (100.00%)
0:23 [485M rows, 0B] [20.9M rows/s, 0B/s]
parquet query (second attempt)
======================================
presto> select y, m, d, h, count(*) from hive.parquet_warehouse.sample_data where y=2021 and m=2 and d=20 and h between 0 and 4 group by y, m, d, h;
Query 20210227_043810_00001_52ygg, RUNNING, 10 nodes, 6,215 splits
0:05 [ 475M rows, 0B] [95.6M rows/s, 0B/s] [=======================================>>>] 91%
STAGES ROWS ROWS/s BYTES BYTES/s QUEUED RUN DONE
0.........R 0 0 0B 0B 0 16 0
1.......R 4.81K 966 686K 138K 0 320 0
2.....R 475M 95.3M 0B 0B 120 94 5665
y | m | d | h | _col4
------+---+----+---+-----------
2021 | 2 | 20 | 3 | 103258290
2021 | 2 | 20 | 2 | 101223162
2021 | 2 | 20 | 0 | 76442859
2021 | 2 | 20 | 1 | 94755366
2021 | 2 | 20 | 4 | 109245879
(5 rows)
Query 20210227_043810_00001_52ygg, FINISHED, 10 nodes
Splits: 6,215 total, 6,215 done (100.00%)
0:05 [485M rows, 0B] [94.2M rows/s, 0B/s]
hudi query (first attempt)
==========================================
presto> select y, m, d, h, count(*) from hive.hudi_warehouse.sample_data group by y, m, d, h;
Query 20210227_042722_00016_9dket, RUNNING, 10 nodes, 8,443 splits
0:40 [ 473M rows, 0B] [ 12M rows/s, 0B/s] [ <=> ]
STAGES ROWS ROWS/s BYTES BYTES/s QUEUED RUN DONE
0.........R 0 0 0B 0B 0 16 0
1.......R 3.26K 82 464K 11.7K 0 320 0
2.....S 473M 12M 0B 0B 43 18 8046
y | m | d | h | _col4
------+---+----+---+-----------
2021 | 2 | 20 | 1 | 94755289
2021 | 2 | 20 | 4 | 109245869
2021 | 2 | 20 | 0 | 76442808
2021 | 2 | 20 | 2 | 101223154
2021 | 2 | 20 | 3 | 103258285
(5 rows)
Query 20210227_042722_00016_9dket, FINISHED, 10 nodes
Splits: 8,572 total, 8,572 done (100.00%)
0:40 [485M rows, 0B] [12.1M rows/s, 0B/s]
presto logs for hudi Query
================================
2021-02-27T04:27:23.112Z DEBUG dispatcher-query-24 io.prestosql.execution.QueryStateMachine Query 20210227_042722_00016_9dket is QUEUED
2021-02-27T04:27:23.114Z DEBUG dispatcher-query-28 io.prestosql.execution.QueryStateMachine Query 20210227_042722_00016_9dket is WAITING_FOR_RESOURCES
2021-02-27T04:27:23.167Z DEBUG Query-20210227_042722_00016_9dket-664 io.prestosql.plugin.hive.metastore.thrift.ThriftHiveMetastoreClient Invocation of get_table_req(req=GetTableRequest(dbName:hudi_warehouse, tblName:sample_data_cluster_2, capabilities:ClientCapabilities(values:[INSERT_ONLY_TABLES]))) succeeded in 51.56ms
2021-02-27T04:27:23.194Z DEBUG dispatcher-query-25 io.prestosql.execution.QueryStateMachine Query 20210227_042722_00016_9dket is DISPATCHING
2021-02-27T04:27:23.194Z DEBUG dispatcher-query-26 io.prestosql.execution.QueryStateMachine Query 20210227_042722_00016_9dket is PLANNING
2021-02-27T04:27:23.276Z DEBUG Query-20210227_042722_00016_9dket-667 io.prestosql.plugin.hive.metastore.thrift.ThriftHiveMetastoreClient Invocation of get_partition_names_ps(db_name='hudi_warehouse', tbl_name='sample_data_cluster_2', part_vals=[, , , ], max_parts=-1) succeeded in 60.66ms
2021-02-27T04:27:23.364Z DEBUG hive-hive-16 io.prestosql.plugin.hive.metastore.thrift.ThriftHiveMetastoreClient Invocation of get_partitions_by_names(db_name='hudi_warehouse', tbl_name='sample_data_cluster_2', names=[y=2021/m=2/d=20/h=4, y=2021/m=2/d=20/h=3, y=2021/m=2/d=20/h=2, y=2021/m=2/d=20/h=1, y=2021/m=2/d=20/h=0]) succeeded in 58.46ms
2021-02-27T04:27:23.369Z DEBUG query-execution-13 io.prestosql.execution.StageStateMachine Stage 20210227_042722_00016_9dket.0 is PLANNED
2021-02-27T04:27:23.369Z DEBUG query-execution-13 io.prestosql.execution.StageStateMachine Stage 20210227_042722_00016_9dket.1 is PLANNED
2021-02-27T04:27:23.373Z DEBUG query-execution-14 io.prestosql.execution.StageStateMachine Stage 20210227_042722_00016_9dket.2 is PLANNED
2021-02-27T04:27:23.373Z DEBUG dispatcher-query-25 io.prestosql.execution.QueryStateMachine Query 20210227_042722_00016_9dket is STARTING
2021-02-27T04:27:23.378Z DEBUG query-execution-16 io.prestosql.execution.StageStateMachine Stage 20210227_042722_00016_9dket.2 is SCHEDULING
2021-02-27T04:27:23.379Z DEBUG query-execution-14 io.prestosql.execution.StageStateMachine Stage 20210227_042722_00016_9dket.1 is SCHEDULING
2021-02-27T04:27:23.420Z DEBUG dispatcher-query-26 io.prestosql.execution.QueryStateMachine Query 20210227_042722_00016_9dket is RUNNING
2021-02-27T04:27:23.425Z INFO hive-hive-19 org.apache.hudi.hadoop.utils.HoodieInputFormatUtils Reading hoodie metadata from path s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.426Z INFO hive-hive-19 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.427Z DEBUG query-execution-14 io.prestosql.execution.StageStateMachine Stage 20210227_042722_00016_9dket.1 is SCHEDULED
2021-02-27T04:27:23.427Z INFO hive-hive-19 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:23.428Z DEBUG query-execution-14 io.prestosql.execution.StageStateMachine Stage 20210227_042722_00016_9dket.0 is SCHEDULING
2021-02-27T04:27:23.428Z DEBUG query-execution-14 io.prestosql.execution.StageStateMachine Stage 20210227_042722_00016_9dket.1 is RUNNING
2021-02-27T04:27:23.429Z INFO hive-hive-17 org.apache.hudi.hadoop.utils.HoodieInputFormatUtils Reading hoodie metadata from path s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.429Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.430Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:23.432Z INFO hive-hive-18 org.apache.hudi.hadoop.utils.HoodieInputFormatUtils Reading hoodie metadata from path s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.433Z INFO hive-hive-16 org.apache.hudi.hadoop.utils.HoodieInputFormatUtils Reading hoodie metadata from path s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.433Z INFO hive-hive-18 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.433Z INFO hive-hive-16 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.434Z INFO hive-hive-18 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:23.434Z DEBUG query-execution-14 io.prestosql.execution.StageStateMachine Stage 20210227_042722_00016_9dket.0 is SCHEDULED
2021-02-27T04:27:23.437Z DEBUG query-execution-16 io.prestosql.execution.StageStateMachine Stage 20210227_042722_00016_9dket.0 is RUNNING
2021-02-27T04:27:23.437Z INFO hive-hive-16 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:23.473Z INFO hive-hive-19 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/hoodie.properties
2021-02-27T04:27:23.473Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/hoodie.properties
2021-02-27T04:27:23.474Z INFO hive-hive-16 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/hoodie.properties
2021-02-27T04:27:23.490Z INFO hive-hive-18 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/hoodie.properties
2021-02-27T04:27:23.502Z INFO hive-hive-16 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type COPY_ON_WRITE(version=1, baseFileFormat=PARQUET) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.503Z INFO hive-hive-16 org.apache.hudi.hadoop.utils.HoodieInputFormatUtils Found a total of 1 groups
2021-02-27T04:27:23.503Z INFO hive-hive-19 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type COPY_ON_WRITE(version=1, baseFileFormat=PARQUET) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.503Z INFO hive-hive-16 org.apache.hudi.common.table.view.FileSystemViewManager Creating InMemory based view for basePath s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.503Z INFO hive-hive-19 org.apache.hudi.hadoop.utils.HoodieInputFormatUtils Found a total of 1 groups
2021-02-27T04:27:23.503Z INFO hive-hive-19 org.apache.hudi.common.table.view.FileSystemViewManager Creating InMemory based view for basePath s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.523Z INFO hive-hive-19 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[20210227020224__commit__COMPLETED], [20210227021755__commit__COMPLETED], [20210227024056__commit__COMPLETED], [20210227030430__commit__COMPLETED], [20210227032808__commit__COMPLETED], [==>20210227035211__commit__INFLIGHT]]
2021-02-27T04:27:23.523Z INFO hive-hive-19 org.apache.hudi.common.table.view.AbstractTableFileSystemView Took 0 ms to read 0 instants, 0 replaced file groups
2021-02-27T04:27:23.529Z INFO hive-hive-16 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[20210227020224__commit__COMPLETED], [20210227021755__commit__COMPLETED], [20210227024056__commit__COMPLETED], [20210227030430__commit__COMPLETED], [20210227032808__commit__COMPLETED], [==>20210227035211__commit__INFLIGHT]]
2021-02-27T04:27:23.530Z INFO hive-hive-16 org.apache.hudi.common.table.view.AbstractTableFileSystemView Took 0 ms to read 0 instants, 0 replaced file groups
2021-02-27T04:27:23.539Z INFO hive-hive-18 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type COPY_ON_WRITE(version=1, baseFileFormat=PARQUET) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.540Z INFO hive-hive-18 org.apache.hudi.hadoop.utils.HoodieInputFormatUtils Found a total of 1 groups
2021-02-27T04:27:23.540Z INFO hive-hive-18 org.apache.hudi.common.table.view.FileSystemViewManager Creating InMemory based view for basePath s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.541Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type COPY_ON_WRITE(version=1, baseFileFormat=PARQUET) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.542Z INFO hive-hive-17 org.apache.hudi.hadoop.utils.HoodieInputFormatUtils Found a total of 1 groups
2021-02-27T04:27:23.542Z INFO hive-hive-17 org.apache.hudi.common.table.view.FileSystemViewManager Creating InMemory based view for basePath s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.561Z INFO hive-hive-19 org.apache.hudi.common.util.ClusteringUtils Found 0 files in pending clustering operations
2021-02-27T04:27:23.561Z INFO hive-hive-18 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[20210227020224__commit__COMPLETED], [20210227021755__commit__COMPLETED], [20210227024056__commit__COMPLETED], [20210227030430__commit__COMPLETED], [20210227032808__commit__COMPLETED], [==>20210227035211__commit__INFLIGHT]]
2021-02-27T04:27:23.561Z INFO hive-hive-19 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.562Z INFO hive-hive-17 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[20210227020224__commit__COMPLETED], [20210227021755__commit__COMPLETED], [20210227024056__commit__COMPLETED], [20210227030430__commit__COMPLETED], [20210227032808__commit__COMPLETED], [==>20210227035211__commit__INFLIGHT]]
2021-02-27T04:27:23.562Z INFO hive-hive-18 org.apache.hudi.common.table.view.AbstractTableFileSystemView Took 0 ms to read 0 instants, 0 replaced file groups
2021-02-27T04:27:23.562Z INFO hive-hive-19 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:23.562Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView Took 0 ms to read 0 instants, 0 replaced file groups
2021-02-27T04:27:23.567Z INFO hive-hive-16 org.apache.hudi.common.util.ClusteringUtils Found 0 files in pending clustering operations
2021-02-27T04:27:23.568Z INFO hive-hive-16 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.569Z INFO hive-hive-16 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:23.598Z INFO hive-hive-18 org.apache.hudi.common.util.ClusteringUtils Found 0 files in pending clustering operations
2021-02-27T04:27:23.598Z INFO hive-hive-18 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.599Z INFO hive-hive-17 org.apache.hudi.common.util.ClusteringUtils Found 0 files in pending clustering operations
2021-02-27T04:27:23.599Z INFO hive-hive-18 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:23.600Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.601Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:23.604Z INFO hive-hive-19 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/hoodie.properties
2021-02-27T04:27:23.606Z INFO hive-hive-16 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/hoodie.properties
2021-02-27T04:27:23.619Z INFO hive-hive-19 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type COPY_ON_WRITE(version=1, baseFileFormat=PARQUET) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.619Z INFO hive-hive-19 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:23.620Z INFO hive-hive-19 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:23.620Z INFO hive-hive-16 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type COPY_ON_WRITE(version=1, baseFileFormat=PARQUET) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.620Z INFO hive-hive-16 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:23.621Z INFO hive-hive-16 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:23.638Z INFO hive-hive-18 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/hoodie.properties
2021-02-27T04:27:23.639Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/hoodie.properties
2021-02-27T04:27:23.652Z INFO hive-hive-18 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type COPY_ON_WRITE(version=1, baseFileFormat=PARQUET) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.653Z INFO hive-hive-18 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:23.655Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type COPY_ON_WRITE(version=1, baseFileFormat=PARQUET) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:23.655Z INFO hive-hive-18 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:23.656Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:23.657Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:23.659Z INFO hive-hive-16 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/.hoodie/hoodie.properties
2021-02-27T04:27:23.670Z INFO hive-hive-19 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/.hoodie/hoodie.properties
2021-02-27T04:27:23.674Z INFO hive-hive-16 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type MERGE_ON_READ(version=1, baseFileFormat=HFILE) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:23.687Z INFO hive-hive-19 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type MERGE_ON_READ(version=1, baseFileFormat=HFILE) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:23.695Z INFO hive-hive-18 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/.hoodie/hoodie.properties
2021-02-27T04:27:23.696Z INFO hive-hive-16 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[0000000000000__deltacommit__COMPLETED], [20210227020224__deltacommit__COMPLETED], [20210227021755__deltacommit__COMPLETED], [20210227024056__deltacommit__COMPLETED], [20210227030430__deltacommit__COMPLETED], [20210227032808__deltacommit__COMPLETED]]
2021-02-27T04:27:23.697Z INFO hive-hive-16 org.apache.hudi.common.table.view.AbstractTableFileSystemView Took 0 ms to read 0 instants, 0 replaced file groups
2021-02-27T04:27:23.700Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/.hoodie/hoodie.properties
2021-02-27T04:27:23.710Z INFO hive-hive-19 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[0000000000000__deltacommit__COMPLETED], [20210227020224__deltacommit__COMPLETED], [20210227021755__deltacommit__COMPLETED], [20210227024056__deltacommit__COMPLETED], [20210227030430__deltacommit__COMPLETED], [20210227032808__deltacommit__COMPLETED]]
2021-02-27T04:27:23.710Z INFO hive-hive-19 org.apache.hudi.common.table.view.AbstractTableFileSystemView Took 0 ms to read 0 instants, 0 replaced file groups
2021-02-27T04:27:23.711Z INFO hive-hive-18 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type MERGE_ON_READ(version=1, baseFileFormat=HFILE) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:23.714Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type MERGE_ON_READ(version=1, baseFileFormat=HFILE) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:23.729Z INFO hive-hive-18 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[0000000000000__deltacommit__COMPLETED], [20210227020224__deltacommit__COMPLETED], [20210227021755__deltacommit__COMPLETED], [20210227024056__deltacommit__COMPLETED], [20210227030430__deltacommit__COMPLETED], [20210227032808__deltacommit__COMPLETED]]
2021-02-27T04:27:23.730Z INFO hive-hive-18 org.apache.hudi.common.table.view.AbstractTableFileSystemView Took 0 ms to read 0 instants, 0 replaced file groups
2021-02-27T04:27:23.732Z INFO hive-hive-17 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[0000000000000__deltacommit__COMPLETED], [20210227020224__deltacommit__COMPLETED], [20210227021755__deltacommit__COMPLETED], [20210227024056__deltacommit__COMPLETED], [20210227030430__deltacommit__COMPLETED], [20210227032808__deltacommit__COMPLETED]]
2021-02-27T04:27:23.733Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView Took 0 ms to read 0 instants, 0 replaced file groups
2021-02-27T04:27:23.737Z INFO hive-hive-16 org.apache.hudi.common.util.ClusteringUtils Found 0 files in pending clustering operations
2021-02-27T04:27:23.737Z INFO hive-hive-16 org.apache.hudi.common.table.view.AbstractTableFileSystemView Building file system view for partition (files)
2021-02-27T04:27:23.746Z INFO hive-hive-19 org.apache.hudi.common.util.ClusteringUtils Found 0 files in pending clustering operations
2021-02-27T04:27:23.747Z INFO hive-hive-19 org.apache.hudi.common.table.view.AbstractTableFileSystemView Building file system view for partition (files)
2021-02-27T04:27:23.767Z INFO hive-hive-18 org.apache.hudi.common.util.ClusteringUtils Found 0 files in pending clustering operations
2021-02-27T04:27:23.767Z INFO hive-hive-18 org.apache.hudi.common.table.view.AbstractTableFileSystemView Building file system view for partition (files)
2021-02-27T04:27:23.771Z INFO hive-hive-17 org.apache.hudi.common.util.ClusteringUtils Found 0 files in pending clustering operations
2021-02-27T04:27:23.772Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView Building file system view for partition (files)
2021-02-27T04:27:23.793Z INFO hive-hive-16 org.apache.hudi.common.table.view.AbstractTableFileSystemView #files found in partition (files) =7, Time taken =19
2021-02-27T04:27:23.795Z INFO hive-hive-16 org.apache.hudi.common.table.view.HoodieTableFileSystemView Adding file-groups for partition :files, #FileGroups=1
2021-02-27T04:27:23.795Z INFO hive-hive-16 org.apache.hudi.common.table.view.AbstractTableFileSystemView addFilesToView: NumFiles=7, NumFileGroups=1, FileGroupsCreationTime=2, StoreTimeTaken=0
2021-02-27T04:27:23.795Z INFO hive-hive-16 org.apache.hudi.common.table.view.AbstractTableFileSystemView Time to load partition (files) =58
2021-02-27T04:27:23.795Z INFO hive-hive-16 org.apache.hudi.common.table.view.AbstractTableFileSystemView Pending Compaction instant for (FileSlice {fileGroupId=HoodieFileGroupId{partitionPath='files', fileId='7e172a24-c399-4f87-a9a7-75b3d2879e0b-0'}, baseCommitTime=0000000000000, baseFile='HoodieBaseFile{fullPath=s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0-15-2332_0000000000000.hfile, fileLen=5696, BootstrapBaseFile=null}', logFiles='[HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093', fileLen=25302}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088', fileLen=23561}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049', fileLen=23243}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183', fileLen=21732}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553', fileLen=27253}]'}) is :Optional.empty
2021-02-27T04:27:23.805Z INFO hive-hive-19 org.apache.hudi.common.table.view.AbstractTableFileSystemView #files found in partition (files) =7, Time taken =20
2021-02-27T04:27:23.807Z INFO hive-hive-19 org.apache.hudi.common.table.view.HoodieTableFileSystemView Adding file-groups for partition :files, #FileGroups=1
2021-02-27T04:27:23.807Z INFO hive-hive-19 org.apache.hudi.common.table.view.AbstractTableFileSystemView addFilesToView: NumFiles=7, NumFileGroups=1, FileGroupsCreationTime=1, StoreTimeTaken=1
2021-02-27T04:27:23.807Z INFO hive-hive-19 org.apache.hudi.common.table.view.AbstractTableFileSystemView Time to load partition (files) =60
2021-02-27T04:27:23.807Z INFO hive-hive-19 org.apache.hudi.common.table.view.AbstractTableFileSystemView Pending Compaction instant for (FileSlice {fileGroupId=HoodieFileGroupId{partitionPath='files', fileId='7e172a24-c399-4f87-a9a7-75b3d2879e0b-0'}, baseCommitTime=0000000000000, baseFile='HoodieBaseFile{fullPath=s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0-15-2332_0000000000000.hfile, fileLen=5696, BootstrapBaseFile=null}', logFiles='[HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093', fileLen=25302}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088', fileLen=23561}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049', fileLen=23243}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183', fileLen=21732}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553', fileLen=27253}]'}) is :Optional.empty
2021-02-27T04:27:23.815Z INFO hive-hive-16 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[20210227020224__commit__COMPLETED], [20210227021755__commit__COMPLETED], [20210227024056__commit__COMPLETED], [20210227030430__commit__COMPLETED], [20210227032808__commit__COMPLETED], [==>20210227035211__commit__INFLIGHT]]
2021-02-27T04:27:23.827Z INFO hive-hive-18 org.apache.hudi.common.table.view.AbstractTableFileSystemView #files found in partition (files) =7, Time taken =22
2021-02-27T04:27:23.828Z INFO hive-hive-19 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[20210227020224__commit__COMPLETED], [20210227021755__commit__COMPLETED], [20210227024056__commit__COMPLETED], [20210227030430__commit__COMPLETED], [20210227032808__commit__COMPLETED], [==>20210227035211__commit__INFLIGHT]]
2021-02-27T04:27:23.829Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView #files found in partition (files) =7, Time taken =17
2021-02-27T04:27:23.829Z INFO hive-hive-18 org.apache.hudi.common.table.view.HoodieTableFileSystemView Adding file-groups for partition :files, #FileGroups=1
2021-02-27T04:27:23.829Z INFO hive-hive-18 org.apache.hudi.common.table.view.AbstractTableFileSystemView addFilesToView: NumFiles=7, NumFileGroups=1, FileGroupsCreationTime=1, StoreTimeTaken=0
2021-02-27T04:27:23.829Z INFO hive-hive-18 org.apache.hudi.common.table.view.AbstractTableFileSystemView Time to load partition (files) =62
2021-02-27T04:27:23.830Z INFO hive-hive-18 org.apache.hudi.common.table.view.AbstractTableFileSystemView Pending Compaction instant for (FileSlice {fileGroupId=HoodieFileGroupId{partitionPath='files', fileId='7e172a24-c399-4f87-a9a7-75b3d2879e0b-0'}, baseCommitTime=0000000000000, baseFile='HoodieBaseFile{fullPath=s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0-15-2332_0000000000000.hfile, fileLen=5696, BootstrapBaseFile=null}', logFiles='[HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093', fileLen=25302}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088', fileLen=23561}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049', fileLen=23243}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183', fileLen=21732}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553', fileLen=27253}]'}) is :Optional.empty
2021-02-27T04:27:23.831Z INFO hive-hive-17 org.apache.hudi.common.table.view.HoodieTableFileSystemView Adding file-groups for partition :files, #FileGroups=1
2021-02-27T04:27:23.831Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView addFilesToView: NumFiles=7, NumFileGroups=1, FileGroupsCreationTime=1, StoreTimeTaken=1
2021-02-27T04:27:23.831Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView Time to load partition (files) =59
2021-02-27T04:27:23.831Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView Pending Compaction instant for (FileSlice {fileGroupId=HoodieFileGroupId{partitionPath='files', fileId='7e172a24-c399-4f87-a9a7-75b3d2879e0b-0'}, baseCommitTime=0000000000000, baseFile='HoodieBaseFile{fullPath=s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0-15-2332_0000000000000.hfile, fileLen=5696, BootstrapBaseFile=null}', logFiles='[HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093', fileLen=25302}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088', fileLen=23561}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049', fileLen=23243}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183', fileLen=21732}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553', fileLen=27253}]'}) is :Optional.empty
2021-02-27T04:27:23.834Z INFO hive-hive-16 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[0000000000000__deltacommit__COMPLETED], [20210227020224__deltacommit__COMPLETED], [20210227021755__deltacommit__COMPLETED], [20210227024056__deltacommit__COMPLETED], [20210227030430__deltacommit__COMPLETED], [20210227032808__deltacommit__COMPLETED]]
2021-02-27T04:27:23.835Z INFO hive-hive-16 org.apache.hudi.metadata.HoodieBackedTableMetadata Metadata table is disabled.
2021-02-27T04:27:23.835Z INFO hive-hive-16 org.apache.hudi.common.table.view.AbstractTableFileSystemView Building file system view for partition (y=2021/m=2/d=20/h=4)
2021-02-27T04:27:23.849Z INFO hive-hive-18 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[20210227020224__commit__COMPLETED], [20210227021755__commit__COMPLETED], [20210227024056__commit__COMPLETED], [20210227030430__commit__COMPLETED], [20210227032808__commit__COMPLETED], [==>20210227035211__commit__INFLIGHT]]
2021-02-27T04:27:23.849Z INFO hive-hive-19 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[0000000000000__deltacommit__COMPLETED], [20210227020224__deltacommit__COMPLETED], [20210227021755__deltacommit__COMPLETED], [20210227024056__deltacommit__COMPLETED], [20210227030430__deltacommit__COMPLETED], [20210227032808__deltacommit__COMPLETED]]
2021-02-27T04:27:23.849Z INFO hive-hive-19 org.apache.hudi.metadata.HoodieBackedTableMetadata Metadata table is disabled.
2021-02-27T04:27:23.849Z INFO hive-hive-17 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[20210227020224__commit__COMPLETED], [20210227021755__commit__COMPLETED], [20210227024056__commit__COMPLETED], [20210227030430__commit__COMPLETED], [20210227032808__commit__COMPLETED], [==>20210227035211__commit__INFLIGHT]]
2021-02-27T04:27:23.850Z INFO hive-hive-19 org.apache.hudi.common.table.view.AbstractTableFileSystemView Building file system view for partition (y=2021/m=2/d=20/h=3)
2021-02-27T04:27:23.869Z INFO hive-hive-17 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[0000000000000__deltacommit__COMPLETED], [20210227020224__deltacommit__COMPLETED], [20210227021755__deltacommit__COMPLETED], [20210227024056__deltacommit__COMPLETED], [20210227030430__deltacommit__COMPLETED], [20210227032808__deltacommit__COMPLETED]]
2021-02-27T04:27:23.870Z INFO hive-hive-17 org.apache.hudi.metadata.HoodieBackedTableMetadata Metadata table is disabled.
2021-02-27T04:27:23.870Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView Building file system view for partition (y=2021/m=2/d=20/h=1)
2021-02-27T04:27:23.872Z INFO hive-hive-18 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[0000000000000__deltacommit__COMPLETED], [20210227020224__deltacommit__COMPLETED], [20210227021755__deltacommit__COMPLETED], [20210227024056__deltacommit__COMPLETED], [20210227030430__deltacommit__COMPLETED], [20210227032808__deltacommit__COMPLETED]]
2021-02-27T04:27:23.873Z INFO hive-hive-18 org.apache.hudi.metadata.HoodieBackedTableMetadata Metadata table is disabled.
2021-02-27T04:27:23.873Z INFO hive-hive-18 org.apache.hudi.common.table.view.AbstractTableFileSystemView Building file system view for partition (y=2021/m=2/d=20/h=2)
2021-02-27T04:27:23.952Z INFO hive-hive-16 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:23.953Z INFO hive-hive-16 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:23.972Z INFO hive-hive-17 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:23.973Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:23.983Z INFO hive-hive-19 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:23.983Z INFO hive-hive-19 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:23.995Z INFO hive-hive-16 org.apache.hudi.metadata.HoodieBackedTableMetadata Opened metadata base file from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0-15-2332_0000000000000.hfile at instant 0000000000000
2021-02-27T04:27:23.995Z INFO hive-hive-18 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:23.995Z INFO hive-hive-18 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:23.996Z INFO hive-hive-16 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:23.997Z INFO hive-hive-16 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:24.034Z INFO hive-hive-17 org.apache.hudi.metadata.HoodieBackedTableMetadata Opened metadata base file from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0-15-2332_0000000000000.hfile at instant 0000000000000
2021-02-27T04:27:24.036Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:24.037Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:24.050Z INFO hive-hive-16 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/.hoodie/hoodie.properties
2021-02-27T04:27:24.051Z INFO hive-hive-19 org.apache.hudi.metadata.HoodieBackedTableMetadata Opened metadata base file from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0-15-2332_0000000000000.hfile at instant 0000000000000
2021-02-27T04:27:24.052Z INFO hive-hive-19 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:24.053Z INFO hive-hive-19 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:24.068Z INFO hive-hive-18 org.apache.hudi.metadata.HoodieBackedTableMetadata Opened metadata base file from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0-15-2332_0000000000000.hfile at instant 0000000000000
2021-02-27T04:27:24.070Z INFO hive-hive-18 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:24.070Z INFO hive-hive-16 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type MERGE_ON_READ(version=1, baseFileFormat=HFILE) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:24.071Z INFO hive-hive-18 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:24.075Z INFO hive-hive-16 org.apache.hudi.common.util.collection.DiskBasedMap Spilling to file location /tmp/view_map/de468b42-7a11-47fb-a360-fb966c8f8b15 in host (100.97.190.93) with hostname (presto-coordinator-5fbd7bd6bd-bhx56)
2021-02-27T04:27:24.096Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/.hoodie/hoodie.properties
2021-02-27T04:27:24.106Z INFO hive-hive-19 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/.hoodie/hoodie.properties
2021-02-27T04:27:24.111Z INFO hive-hive-18 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/.hoodie/hoodie.properties
2021-02-27T04:27:24.112Z INFO hive-hive-16 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553', fileLen=0}
2021-02-27T04:27:24.114Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type MERGE_ON_READ(version=1, baseFileFormat=HFILE) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:24.114Z INFO hive-hive-17 org.apache.hudi.common.util.collection.DiskBasedMap Spilling to file location /tmp/view_map/287eb55d-674d-4f7f-82c1-7a766c596e64 in host (100.97.190.93) with hostname (presto-coordinator-5fbd7bd6bd-bhx56)
2021-02-27T04:27:24.122Z INFO hive-hive-19 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type MERGE_ON_READ(version=1, baseFileFormat=HFILE) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:24.123Z INFO hive-hive-19 org.apache.hudi.common.util.collection.DiskBasedMap Spilling to file location /tmp/view_map/2a5b2fb5-5287-4b3b-8c9c-8ce4ebfed736 in host (100.97.190.93) with hostname (presto-coordinator-5fbd7bd6bd-bhx56)
2021-02-27T04:27:24.125Z INFO hive-hive-18 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type MERGE_ON_READ(version=1, baseFileFormat=HFILE) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:24.125Z INFO hive-hive-16 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553 at instant 20210227020224
2021-02-27T04:27:24.126Z INFO hive-hive-18 org.apache.hudi.common.util.collection.DiskBasedMap Spilling to file location /tmp/view_map/52ad2374-1441-48f2-8856-f4838ca8bec5 in host (100.97.190.93) with hostname (presto-coordinator-5fbd7bd6bd-bhx56)
2021-02-27T04:27:24.140Z INFO hive-hive-16 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183', fileLen=0}
2021-02-27T04:27:24.145Z INFO hive-hive-19 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553', fileLen=0}
2021-02-27T04:27:24.151Z INFO hive-hive-18 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553', fileLen=0}
2021-02-27T04:27:24.155Z INFO hive-hive-16 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183', fileLen=0}
2021-02-27T04:27:24.156Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553', fileLen=0}
2021-02-27T04:27:24.164Z INFO hive-hive-19 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553 at instant 20210227020224
2021-02-27T04:27:24.168Z INFO hive-hive-18 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553 at instant 20210227020224
2021-02-27T04:27:24.171Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553 at instant 20210227020224
2021-02-27T04:27:24.171Z INFO hive-hive-19 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183', fileLen=0}
2021-02-27T04:27:24.176Z INFO hive-hive-18 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183', fileLen=0}
2021-02-27T04:27:24.177Z INFO hive-hive-16 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183 at instant 20210227021755
2021-02-27T04:27:24.178Z INFO hive-hive-16 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.179Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183', fileLen=0}
2021-02-27T04:27:24.184Z INFO hive-hive-19 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183', fileLen=0}
2021-02-27T04:27:24.188Z INFO hive-hive-16 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.189Z INFO hive-hive-16 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.190Z INFO hive-hive-18 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183', fileLen=0}
2021-02-27T04:27:24.191Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183', fileLen=0}
2021-02-27T04:27:24.191Z INFO hive-hive-16 org.apache.hudi.common.util.collection.ExternalSpillableMap Estimated Payload size => 792
2021-02-27T04:27:24.198Z INFO hive-hive-16 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049', fileLen=0}
2021-02-27T04:27:24.200Z INFO hive-hive-19 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183 at instant 20210227021755
2021-02-27T04:27:24.200Z INFO hive-hive-19 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.204Z INFO hive-hive-18 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183 at instant 20210227021755
2021-02-27T04:27:24.204Z INFO hive-hive-18 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.207Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183 at instant 20210227021755
2021-02-27T04:27:24.208Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.213Z INFO hive-hive-19 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.213Z INFO hive-hive-19 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.215Z INFO hive-hive-16 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049', fileLen=0}
2021-02-27T04:27:24.216Z INFO hive-hive-19 org.apache.hudi.common.util.collection.ExternalSpillableMap Estimated Payload size => 792
2021-02-27T04:27:24.218Z INFO hive-hive-18 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.218Z INFO hive-hive-18 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.220Z INFO hive-hive-18 org.apache.hudi.common.util.collection.ExternalSpillableMap Estimated Payload size => 792
2021-02-27T04:27:24.223Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.223Z INFO hive-hive-19 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049', fileLen=0}
2021-02-27T04:27:24.223Z INFO hive-hive-17 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.228Z INFO hive-hive-18 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049', fileLen=0}
2021-02-27T04:27:24.233Z INFO hive-hive-16 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049 at instant 20210227024056
2021-02-27T04:27:24.233Z INFO hive-hive-16 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.235Z INFO hive-hive-17 org.apache.hudi.common.util.collection.ExternalSpillableMap Estimated Payload size => 792
2021-02-27T04:27:24.239Z INFO hive-hive-19 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049', fileLen=0}
2021-02-27T04:27:24.244Z INFO hive-hive-16 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.245Z INFO hive-hive-16 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.245Z INFO hive-hive-18 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049', fileLen=0}
2021-02-27T04:27:24.247Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049', fileLen=0}
2021-02-27T04:27:24.255Z INFO hive-hive-16 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088', fileLen=0}
2021-02-27T04:27:24.258Z INFO hive-hive-19 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049 at instant 20210227024056
2021-02-27T04:27:24.258Z INFO hive-hive-19 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.262Z INFO hive-hive-18 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049 at instant 20210227024056
2021-02-27T04:27:24.262Z INFO hive-hive-18 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.270Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049', fileLen=0}
2021-02-27T04:27:24.271Z INFO hive-hive-19 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.271Z INFO hive-hive-19 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.275Z INFO hive-hive-18 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.275Z INFO hive-hive-18 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.281Z INFO hive-hive-19 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088', fileLen=0}
2021-02-27T04:27:24.285Z INFO hive-hive-18 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088', fileLen=0}
2021-02-27T04:27:24.288Z INFO hive-hive-16 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088', fileLen=0}
2021-02-27T04:27:24.289Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049 at instant 20210227024056
2021-02-27T04:27:24.289Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.297Z INFO hive-hive-19 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088', fileLen=0}
2021-02-27T04:27:24.301Z INFO hive-hive-18 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088', fileLen=0}
2021-02-27T04:27:24.302Z INFO hive-hive-16 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088 at instant 20210227030430
2021-02-27T04:27:24.302Z INFO hive-hive-16 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.303Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.303Z INFO hive-hive-17 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.313Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088', fileLen=0}
2021-02-27T04:27:24.314Z INFO hive-hive-19 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088 at instant 20210227030430
2021-02-27T04:27:24.314Z INFO hive-hive-19 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.315Z INFO hive-hive-16 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.316Z INFO hive-hive-16 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.326Z INFO hive-hive-16 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093', fileLen=0}
2021-02-27T04:27:24.326Z INFO hive-hive-18 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088 at instant 20210227030430
2021-02-27T04:27:24.326Z INFO hive-hive-18 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.327Z INFO hive-hive-19 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.327Z INFO hive-hive-19 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.333Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088', fileLen=0}
2021-02-27T04:27:24.338Z INFO hive-hive-19 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093', fileLen=0}
2021-02-27T04:27:24.340Z INFO hive-hive-16 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093', fileLen=0}
2021-02-27T04:27:24.348Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088 at instant 20210227030430
2021-02-27T04:27:24.348Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.353Z INFO hive-hive-16 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093 at instant 20210227032808
2021-02-27T04:27:24.354Z INFO hive-hive-16 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.356Z INFO hive-hive-19 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093', fileLen=0}
2021-02-27T04:27:24.356Z INFO hive-hive-18 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.357Z INFO hive-hive-18 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.372Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.372Z INFO hive-hive-17 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.373Z INFO hive-hive-19 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093 at instant 20210227032808
2021-02-27T04:27:24.373Z INFO hive-hive-19 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.382Z INFO hive-hive-16 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.382Z INFO hive-hive-16 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.383Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093', fileLen=0}
2021-02-27T04:27:24.384Z INFO hive-hive-16 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Merging the final data blocks
2021-02-27T04:27:24.384Z INFO hive-hive-16 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.387Z INFO hive-hive-19 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.387Z INFO hive-hive-19 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.390Z INFO hive-hive-19 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Merging the final data blocks
2021-02-27T04:27:24.390Z INFO hive-hive-19 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.397Z INFO hive-hive-18 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093', fileLen=0}
2021-02-27T04:27:24.398Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093', fileLen=0}
2021-02-27T04:27:24.413Z INFO hive-hive-18 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093', fileLen=0}
2021-02-27T04:27:24.416Z INFO hive-hive-16 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.416Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093 at instant 20210227032808
2021-02-27T04:27:24.416Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.416Z INFO hive-hive-16 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.418Z INFO hive-hive-16 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Number of log files scanned => 5
2021-02-27T04:27:24.419Z INFO hive-hive-16 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner MaxMemoryInBytes allowed for compaction => 0
2021-02-27T04:27:24.419Z INFO hive-hive-16 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Number of entries in MemoryBasedMap in ExternalSpillableMap => 6
2021-02-27T04:27:24.419Z INFO hive-hive-16 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Total size in bytes of MemoryBasedMap in ExternalSpillableMap => 4752
2021-02-27T04:27:24.419Z INFO hive-hive-16 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Number of entries in DiskBasedMap in ExternalSpillableMap => 0
2021-02-27T04:27:24.419Z INFO hive-hive-16 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Size of file spilled to disk => 0
2021-02-27T04:27:24.419Z INFO hive-hive-16 org.apache.hudi.metadata.HoodieBackedTableMetadata Opened metadata log files from [s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093] at instant 20210227032808(dataset instant=20210227032808, metadata instant=20210227032808)
2021-02-27T04:27:24.422Z INFO hive-hive-19 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.422Z INFO hive-hive-19 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.425Z INFO hive-hive-19 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Number of log files scanned => 5
2021-02-27T04:27:24.425Z INFO hive-hive-19 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner MaxMemoryInBytes allowed for compaction => 0
2021-02-27T04:27:24.425Z INFO hive-hive-19 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Number of entries in MemoryBasedMap in ExternalSpillableMap => 6
2021-02-27T04:27:24.425Z INFO hive-hive-19 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Total size in bytes of MemoryBasedMap in ExternalSpillableMap => 4752
2021-02-27T04:27:24.425Z INFO hive-hive-19 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Number of entries in DiskBasedMap in ExternalSpillableMap => 0
2021-02-27T04:27:24.426Z INFO hive-hive-19 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Size of file spilled to disk => 0
2021-02-27T04:27:24.426Z INFO hive-hive-19 org.apache.hudi.metadata.HoodieBackedTableMetadata Opened metadata log files from [s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093] at instant 20210227032808(dataset instant=20210227032808, metadata instant=20210227032808)
2021-02-27T04:27:24.430Z INFO hive-hive-18 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093 at instant 20210227032808
2021-02-27T04:27:24.431Z INFO hive-hive-18 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.435Z INFO hive-hive-16 org.apache.hudi.metadata.HoodieBackedTableMetadata Metadata read for key y=2021/m=2/d=20/h=4 took [open, baseFileRead, logMerge] [467, 16, 0] ms
2021-02-27T04:27:24.436Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.436Z INFO hive-hive-17 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.439Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Merging the final data blocks
2021-02-27T04:27:24.439Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.440Z INFO hive-hive-19 org.apache.hudi.metadata.HoodieBackedTableMetadata Metadata read for key y=2021/m=2/d=20/h=3 took [open, baseFileRead, logMerge] [443, 14, 0] ms
2021-02-27T04:27:24.445Z INFO hive-hive-16 org.apache.hudi.metadata.BaseTableMetadata Listed file in partition from metadata: partition=y=2021/m=2/d=20/h=4, #files=761
2021-02-27T04:27:24.445Z INFO hive-hive-16 org.apache.hudi.common.table.view.AbstractTableFileSystemView #files found in partition (y=2021/m=2/d=20/h=4) =761, Time taken =493
2021-02-27T04:27:24.450Z INFO hive-hive-19 org.apache.hudi.metadata.BaseTableMetadata Listed file in partition from metadata: partition=y=2021/m=2/d=20/h=3, #files=699
2021-02-27T04:27:24.450Z INFO hive-hive-19 org.apache.hudi.common.table.view.AbstractTableFileSystemView #files found in partition (y=2021/m=2/d=20/h=3) =699, Time taken =467
2021-02-27T04:27:24.452Z INFO hive-hive-18 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.452Z INFO hive-hive-18 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.459Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.460Z INFO hive-hive-17 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.462Z INFO hive-hive-18 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Merging the final data blocks
2021-02-27T04:27:24.462Z INFO hive-hive-18 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:24.462Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Number of log files scanned => 5
2021-02-27T04:27:24.463Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner MaxMemoryInBytes allowed for compaction => 0
2021-02-27T04:27:24.463Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Number of entries in MemoryBasedMap in ExternalSpillableMap => 6
2021-02-27T04:27:24.463Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Total size in bytes of MemoryBasedMap in ExternalSpillableMap => 4752
2021-02-27T04:27:24.463Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Number of entries in DiskBasedMap in ExternalSpillableMap => 0
2021-02-27T04:27:24.463Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Size of file spilled to disk => 0
2021-02-27T04:27:24.463Z INFO hive-hive-17 org.apache.hudi.metadata.HoodieBackedTableMetadata Opened metadata log files from [s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093] at instant 20210227032808(dataset instant=20210227032808, metadata instant=20210227032808)
2021-02-27T04:27:24.480Z INFO hive-hive-17 org.apache.hudi.metadata.HoodieBackedTableMetadata Metadata read for key y=2021/m=2/d=20/h=1 took [open, baseFileRead, logMerge] [491, 17, 0] ms
2021-02-27T04:27:24.487Z INFO hive-hive-18 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:24.487Z INFO hive-hive-18 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:24.491Z INFO hive-hive-17 org.apache.hudi.metadata.BaseTableMetadata Listed file in partition from metadata: partition=y=2021/m=2/d=20/h=1, #files=647
2021-02-27T04:27:24.491Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView #files found in partition (y=2021/m=2/d=20/h=1) =647, Time taken =519
2021-02-27T04:27:24.501Z INFO hive-hive-16 org.apache.hudi.common.table.view.HoodieTableFileSystemView Adding file-groups for partition :y=2021/m=2/d=20/h=4, #FileGroups=761
2021-02-27T04:27:24.501Z INFO hive-hive-16 org.apache.hudi.common.table.view.AbstractTableFileSystemView addFilesToView: NumFiles=761, NumFileGroups=761, FileGroupsCreationTime=54, StoreTimeTaken=1
2021-02-27T04:27:24.501Z INFO hive-hive-16 org.apache.hudi.common.table.view.AbstractTableFileSystemView Time to load partition (y=2021/m=2/d=20/h=4) =666
2021-02-27T04:27:24.526Z INFO hive-hive-18 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Number of log files scanned => 5
2021-02-27T04:27:24.527Z INFO hive-hive-18 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner MaxMemoryInBytes allowed for compaction => 0
2021-02-27T04:27:24.527Z INFO hive-hive-18 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Number of entries in MemoryBasedMap in ExternalSpillableMap => 6
2021-02-27T04:27:24.527Z INFO hive-hive-18 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Total size in bytes of MemoryBasedMap in ExternalSpillableMap => 4752
2021-02-27T04:27:24.527Z INFO hive-hive-18 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Number of entries in DiskBasedMap in ExternalSpillableMap => 0
2021-02-27T04:27:24.527Z INFO hive-hive-18 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Size of file spilled to disk => 0
2021-02-27T04:27:24.531Z INFO hive-hive-18 org.apache.hudi.metadata.HoodieBackedTableMetadata Opened metadata log files from [s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093] at instant 20210227032808(dataset instant=20210227032808, metadata instant=20210227032808)
2021-02-27T04:27:24.551Z INFO hive-hive-18 org.apache.hudi.metadata.HoodieBackedTableMetadata Metadata read for key y=2021/m=2/d=20/h=2 took [open, baseFileRead, logMerge] [537, 19, 0] ms
2021-02-27T04:27:24.560Z INFO hive-hive-18 org.apache.hudi.metadata.BaseTableMetadata Listed file in partition from metadata: partition=y=2021/m=2/d=20/h=2, #files=691
2021-02-27T04:27:24.560Z INFO hive-hive-18 org.apache.hudi.common.table.view.AbstractTableFileSystemView #files found in partition (y=2021/m=2/d=20/h=2) =691, Time taken =566
2021-02-27T04:27:24.563Z INFO hive-hive-19 org.apache.hudi.common.table.view.HoodieTableFileSystemView Adding file-groups for partition :y=2021/m=2/d=20/h=3, #FileGroups=699
2021-02-27T04:27:24.563Z INFO hive-hive-19 org.apache.hudi.common.table.view.AbstractTableFileSystemView addFilesToView: NumFiles=699, NumFileGroups=699, FileGroupsCreationTime=113, StoreTimeTaken=0
2021-02-27T04:27:24.563Z INFO hive-hive-19 org.apache.hudi.common.table.view.AbstractTableFileSystemView Time to load partition (y=2021/m=2/d=20/h=3) =713
2021-02-27T04:27:24.584Z INFO hive-hive-17 org.apache.hudi.common.table.view.HoodieTableFileSystemView Adding file-groups for partition :y=2021/m=2/d=20/h=1, #FileGroups=647
2021-02-27T04:27:24.584Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView addFilesToView: NumFiles=647, NumFileGroups=647, FileGroupsCreationTime=92, StoreTimeTaken=1
2021-02-27T04:27:24.584Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView Time to load partition (y=2021/m=2/d=20/h=1) =714
2021-02-27T04:27:24.609Z INFO hive-hive-19 org.apache.hudi.hadoop.utils.HoodieInputFormatUtils Total paths to process after hoodie filter 699
2021-02-27T04:27:24.615Z INFO hive-hive-16 org.apache.hudi.hadoop.utils.HoodieInputFormatUtils Total paths to process after hoodie filter 761
2021-02-27T04:27:24.631Z INFO hive-hive-17 org.apache.hudi.hadoop.utils.HoodieInputFormatUtils Total paths to process after hoodie filter 647
2021-02-27T04:27:24.670Z INFO hive-hive-18 org.apache.hudi.common.table.view.HoodieTableFileSystemView Adding file-groups for partition :y=2021/m=2/d=20/h=2, #FileGroups=691
2021-02-27T04:27:24.670Z INFO hive-hive-18 org.apache.hudi.common.table.view.AbstractTableFileSystemView addFilesToView: NumFiles=691, NumFileGroups=691, FileGroupsCreationTime=110, StoreTimeTaken=0
2021-02-27T04:27:24.671Z INFO hive-hive-18 org.apache.hudi.common.table.view.AbstractTableFileSystemView Time to load partition (y=2021/m=2/d=20/h=2) =798
2021-02-27T04:27:24.714Z INFO hive-hive-18 org.apache.hudi.hadoop.utils.HoodieInputFormatUtils Total paths to process after hoodie filter 691
2021-02-27T04:27:45.364Z INFO hive-hive-17 org.apache.hudi.hadoop.utils.HoodieInputFormatUtils Reading hoodie metadata from path s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:45.364Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:45.365Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:45.410Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/hoodie.properties
2021-02-27T04:27:45.426Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type COPY_ON_WRITE(version=1, baseFileFormat=PARQUET) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:45.427Z INFO hive-hive-17 org.apache.hudi.hadoop.utils.HoodieInputFormatUtils Found a total of 1 groups
2021-02-27T04:27:45.427Z INFO hive-hive-17 org.apache.hudi.common.table.view.FileSystemViewManager Creating InMemory based view for basePath s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:45.453Z INFO hive-hive-17 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[20210227020224__commit__COMPLETED], [20210227021755__commit__COMPLETED], [20210227024056__commit__COMPLETED], [20210227030430__commit__COMPLETED], [20210227032808__commit__COMPLETED], [==>20210227035211__commit__INFLIGHT]]
2021-02-27T04:27:45.453Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView Took 0 ms to read 0 instants, 0 replaced file groups
2021-02-27T04:27:45.492Z INFO hive-hive-17 org.apache.hudi.common.util.ClusteringUtils Found 0 files in pending clustering operations
2021-02-27T04:27:45.492Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:45.493Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:45.531Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/hoodie.properties
2021-02-27T04:27:45.547Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type COPY_ON_WRITE(version=1, baseFileFormat=PARQUET) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data
2021-02-27T04:27:45.547Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:45.548Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:45.585Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/.hoodie/hoodie.properties
2021-02-27T04:27:45.599Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type MERGE_ON_READ(version=1, baseFileFormat=HFILE) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:45.618Z INFO hive-hive-17 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[0000000000000__deltacommit__COMPLETED], [20210227020224__deltacommit__COMPLETED], [20210227021755__deltacommit__COMPLETED], [20210227024056__deltacommit__COMPLETED], [20210227030430__deltacommit__COMPLETED], [20210227032808__deltacommit__COMPLETED]]
2021-02-27T04:27:45.618Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView Took 0 ms to read 0 instants, 0 replaced file groups
2021-02-27T04:27:45.652Z INFO hive-hive-17 org.apache.hudi.common.util.ClusteringUtils Found 0 files in pending clustering operations
2021-02-27T04:27:45.652Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView Building file system view for partition (files)
2021-02-27T04:27:45.709Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView #files found in partition (files) =7, Time taken =18
2021-02-27T04:27:45.710Z INFO hive-hive-17 org.apache.hudi.common.table.view.HoodieTableFileSystemView Adding file-groups for partition :files, #FileGroups=1
2021-02-27T04:27:45.710Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView addFilesToView: NumFiles=7, NumFileGroups=1, FileGroupsCreationTime=1, StoreTimeTaken=0
2021-02-27T04:27:45.711Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView Time to load partition (files) =59
2021-02-27T04:27:45.711Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView Pending Compaction instant for (FileSlice {fileGroupId=HoodieFileGroupId{partitionPath='files', fileId='7e172a24-c399-4f87-a9a7-75b3d2879e0b-0'}, baseCommitTime=0000000000000, baseFile='HoodieBaseFile{fullPath=s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0-15-2332_0000000000000.hfile, fileLen=5696, BootstrapBaseFile=null}', logFiles='[HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093', fileLen=25302}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088', fileLen=23561}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049', fileLen=23243}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183', fileLen=21732}, HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553', fileLen=27253}]'}) is :Optional.empty
2021-02-27T04:27:45.732Z INFO hive-hive-17 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[20210227020224__commit__COMPLETED], [20210227021755__commit__COMPLETED], [20210227024056__commit__COMPLETED], [20210227030430__commit__COMPLETED], [20210227032808__commit__COMPLETED], [==>20210227035211__commit__INFLIGHT]]
2021-02-27T04:27:45.756Z INFO hive-hive-17 org.apache.hudi.common.table.timeline.HoodieActiveTimeline Loaded instants [[0000000000000__deltacommit__COMPLETED], [20210227020224__deltacommit__COMPLETED], [20210227021755__deltacommit__COMPLETED], [20210227024056__deltacommit__COMPLETED], [20210227030430__deltacommit__COMPLETED], [20210227032808__deltacommit__COMPLETED]]
2021-02-27T04:27:45.756Z INFO hive-hive-17 org.apache.hudi.metadata.HoodieBackedTableMetadata Metadata table is disabled.
2021-02-27T04:27:45.756Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView Building file system view for partition (y=2021/m=2/d=20/h=0)
2021-02-27T04:27:45.899Z INFO hive-hive-17 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:45.899Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:45.953Z INFO hive-hive-17 org.apache.hudi.metadata.HoodieBackedTableMetadata Opened metadata base file from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0-15-2332_0000000000000.hfile at instant 0000000000000
2021-02-27T04:27:45.954Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableMetaClient Loading HoodieTableMetaClient from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:45.954Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [hdfs://hdfs-service], Config:[Configuration: ], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7dabbda3]
2021-02-27T04:27:46.012Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableConfig Loading table properties from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/.hoodie/hoodie.properties
2021-02-27T04:27:46.030Z INFO hive-hive-17 org.apache.hudi.common.table.HoodieTableMetaClient Finished Loading Table of type MERGE_ON_READ(version=1, baseFileFormat=HFILE) from s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata
2021-02-27T04:27:46.035Z INFO hive-hive-17 org.apache.hudi.common.util.collection.DiskBasedMap Spilling to file location /tmp/view_map/c949fce3-e000-4ef4-b7c6-17ee139de8e3 in host (100.97.190.93) with hostname (presto-coordinator-5fbd7bd6bd-bhx56)
2021-02-27T04:27:46.064Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553', fileLen=0}
2021-02-27T04:27:46.082Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553 at instant 20210227020224
2021-02-27T04:27:46.086Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183', fileLen=0}
2021-02-27T04:27:46.102Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183', fileLen=0}
2021-02-27T04:27:46.121Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183 at instant 20210227021755
2021-02-27T04:27:46.121Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:46.134Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:46.134Z INFO hive-hive-17 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:46.137Z INFO hive-hive-17 org.apache.hudi.common.util.collection.ExternalSpillableMap Estimated Payload size => 792
2021-02-27T04:27:46.140Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049', fileLen=0}
2021-02-27T04:27:46.154Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049', fileLen=0}
2021-02-27T04:27:46.168Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049 at instant 20210227024056
2021-02-27T04:27:46.168Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:46.179Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:46.180Z INFO hive-hive-17 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:46.184Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088', fileLen=0}
2021-02-27T04:27:46.200Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088', fileLen=0}
2021-02-27T04:27:46.214Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088 at instant 20210227030430
2021-02-27T04:27:46.214Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:46.225Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:46.226Z INFO hive-hive-17 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:46.230Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieLogFormatReader Moving to the next reader for logfile HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093', fileLen=0}
2021-02-27T04:27:46.245Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Scanning log file HoodieLogFile{pathStr='s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093', fileLen=0}
2021-02-27T04:27:46.264Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Reading a data block from file s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093 at instant 20210227032808
2021-02-27T04:27:46.265Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:46.276Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:46.277Z INFO hive-hive-17 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:46.279Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Merging the final data blocks
2021-02-27T04:27:46.279Z INFO hive-hive-17 org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner Number of remaining logblocks to merge 1
2021-02-27T04:27:46.291Z INFO hive-hive-17 org.apache.hudi.common.fs.FSUtils Hadoop Configuration: fs.defaultFS: [file:///], Config:[Configuration: core-default.xml, core-site.xml, hdfs-default.xml, hdfs-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml], FileSystem: [org.apache.hadoop.fs.PrestoFileSystemCache$FileSystemWrapper@7f02c117]
2021-02-27T04:27:46.291Z INFO hive-hive-17 org.apache.hadoop.hbase.io.hfile.CacheConfig Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=12497808, freeSize=12174472304, maxSize=12186970112, heapSize=12497808, minSize=11577621504, minFactor=0.95, multiSize=5788810752, multiFactor=0.5, singleSize=2894405376, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2021-02-27T04:27:46.293Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Number of log files scanned => 5
2021-02-27T04:27:46.293Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner MaxMemoryInBytes allowed for compaction => 0
2021-02-27T04:27:46.293Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Number of entries in MemoryBasedMap in ExternalSpillableMap => 6
2021-02-27T04:27:46.293Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Total size in bytes of MemoryBasedMap in ExternalSpillableMap => 4752
2021-02-27T04:27:46.293Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Number of entries in DiskBasedMap in ExternalSpillableMap => 0
2021-02-27T04:27:46.293Z INFO hive-hive-17 org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner Size of file spilled to disk => 0
2021-02-27T04:27:46.294Z INFO hive-hive-17 org.apache.hudi.metadata.HoodieBackedTableMetadata Opened metadata log files from [s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.1_0-45-8553, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.2_0-85-17183, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.3_0-125-26049, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.4_0-168-35088, s3a://my-test-bucket/tmp/ramesh/hudi_0_7_cl2/sample_data/.hoodie/metadata/files/.7e172a24-c399-4f87-a9a7-75b3d2879e0b-0_0000000000000.log.5_0-211-44093] at instant 20210227032808(dataset instant=20210227032808, metadata instant=20210227032808)
2021-02-27T04:27:46.309Z INFO hive-hive-17 org.apache.hudi.metadata.HoodieBackedTableMetadata Metadata read for key y=2021/m=2/d=20/h=0 took [open, baseFileRead, logMerge] [396, 15, 0] ms
2021-02-27T04:27:46.314Z INFO hive-hive-17 org.apache.hudi.metadata.BaseTableMetadata Listed file in partition from metadata: partition=y=2021/m=2/d=20/h=0, #files=623
2021-02-27T04:27:46.315Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView #files found in partition (y=2021/m=2/d=20/h=0) =623, Time taken =417
2021-02-27T04:27:46.337Z INFO hive-hive-17 org.apache.hudi.common.table.view.HoodieTableFileSystemView Adding file-groups for partition :y=2021/m=2/d=20/h=0, #FileGroups=623
2021-02-27T04:27:46.338Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView addFilesToView: NumFiles=623, NumFileGroups=623, FileGroupsCreationTime=22, StoreTimeTaken=1
2021-02-27T04:27:46.338Z INFO hive-hive-17 org.apache.hudi.common.table.view.AbstractTableFileSystemView Time to load partition (y=2021/m=2/d=20/h=0) =582
2021-02-27T04:27:46.360Z INFO hive-hive-17 org.apache.hudi.hadoop.utils.HoodieInputFormatUtils Total paths to process after hoodie filter 623
2021-02-27T04:28:02.931Z DEBUG query-execution-16 io.prestosql.execution.StageStateMachine Stage 20210227_042722_00016_9dket.2 is SCHEDULED
2021-02-27T04:28:02.934Z DEBUG query-execution-16 io.prestosql.execution.StageStateMachine Stage 20210227_042722_00016_9dket.2 is RUNNING
2021-02-27T04:28:03.016Z DEBUG query-execution-16 io.prestosql.execution.StageStateMachine Stage 20210227_042722_00016_9dket.2 is FINISHED
2021-02-27T04:28:03.024Z DEBUG query-execution-16 io.prestosql.execution.StageStateMachine Stage 20210227_042722_00016_9dket.1 is FINISHED
2021-02-27T04:28:03.026Z DEBUG query-execution-16 io.prestosql.execution.StageStateMachine Stage 20210227_042722_00016_9dket.0 is FINISHED
2021-02-27T04:28:03.026Z DEBUG dispatcher-query-26 io.prestosql.execution.QueryStateMachine Query 20210227_042722_00016_9dket is FINISHING
2021-02-27T04:28:03.027Z DEBUG dispatcher-query-26 io.prestosql.execution.QueryStateMachine Query 20210227_042722_00016_9dket is FINISHED
2021-02-27T04:28:03.053Z INFO dispatcher-query-26 io.prestosql.event.QueryMonitor TIMELINE: Query 20210227_042722_00016_9dket :: Transaction:[13b23144-6f70-42e6-b876-4a69ca8744f3] :: elapsed 39914ms :: planning 179ms :: waiting 80ms :: scheduling 1724ms :: running 37991ms :: finishing 20ms :: begin 2021-02-27T04:27:23.112Z :: end 2021-02-27T04:28:03.026Z
Issue Analytics
- State:
- Created 3 years ago
- Reactions:1
- Comments:19 (8 by maintainers)
Top Results From Across the Web
[GitHub] [hudi] rshanmugam1 opened a new ... - The Mail Archive
... new issue #2609: [SUPPORT] Presto hudi query slow when compared to parquet ... **Additional context** Presto (trino 352) runs on k8. hudi...
Read more >Hive connector storage caching — Trino 403 Documentation
Querying object storage with the Hive connector is a very common use case for Trino. ... and your local cache storage is at...
Read more >Querying Data - Apache Hudi
Hudi tables can be queried via the Spark datasource with a simple spark.read.parquet . See the Spark Quick Start for more examples of...
Read more >Apache Hudi — The Streaming Data Lake Platform - Medium
Hudi provides snapshot isolation between writers and readers and allows for any table snapshot to be queries consistently from all major lake ...
Read more >Build your Apache Hudi data lake on AWS using Amazon EMR
CoW table supports snapshot query and incremental queries. MoR offers two ways to query the same underlying storage: ReadOptimized tables and ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
@rshanmugam1 see https://prestodb.io/blog/2020/08/04/prestodb-and-hudi and https://github.com/prestodb/presto/commit/9fd2459d98efd0809023b175ba53775466b74cc6 and https://github.com/prestodb/presto/commit/cfb2e7aa077954a02c048e81c97a47994d329852
@njalan not trino (aka prestosql), it only in prestodb