Metrics "Short-circuit Read" and "Cache Hit Rate" for Alluxio Local does not update.

Alluxio Version: Alluxio-2.0

Describe the bug Metrics like short circuit read and Cache Hit Rate for Alluxio Local does not update or more precisely it always gives zero on running ./alluxio fsadmin report metrics.

So even if it’s caching or not I don’t know because it’s updating other metrics like Under Filesystem Read and Cache Hit Rate for Alluxio Remote. But my Spark-UI shows PROCESS_LEVEL and NODE_LEVEL of data locality.

Note :- flag alluxio.user.metrics.collection.enabled=true is already there in alluxio-site.conf

To Reproduce

val df = spark.read.parquet("alluxio:///path/to/parquet/file")
df.agg(sum("XYZ")).show

Expected behavior If my spark UI shows the data locality level at process-local and node-local then Alluxio metric must show some short circuit read overall.

Additional context To make sure of data-locality I’m running my setup with a single worker node and spark and alluxio-worker both running on that same node.

I also needed one clarification -

spark.read.parquet("alluxio:///path/to/parquet/file") I think by the time spark does this transformation, it gathers node information where it’s going to deploy executors according to data locality. Is it so?

If not then when and how spark get to know the partition location on worker nodes?

[jinhang]

I also encountered this problem, did you solve it? @bf8086 Total IO: Short-circuit Read 0B Short-circuit Read (Domain Socket) 0B From Remote Instances 787.84GB Under Filesystem Read 617.89GB Alluxio Write 32.60GB Alluxio Write (Domain Socket) 0B Under Filesystem Write 0B Total IO Throughput (Last Minute): Short-circuit Read 0B Short-circuit Read (Domain Socket) 0B From Remote Instances 7.37KB Under Filesystem Read 0B Alluxio Write 2623B Alluxio Write (Domain Socket) 0B Under Filesystem Write 0B Cache Hit Rate (Percentage): Alluxio Local 0.00 Alluxio Remote 100 Miss 0.00

[yuzhu]

fixed in 2.7.0