Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Read a 1 GB file from alluxio-fuse 2min but alluxio shell 10sec

See original GitHub issue

Alluxio Version:

2.6.1

Describe the bug

We read a full cached file in alluxio with two way, the duration is pretty different.

To Reproduce

Env: alluxio on K8s, use memdisk as cache storage for workers
Step 1, make sure a 1G file was cached totally in alluxio.
Step 2, make sure the fuse pod is not co-located with worker pod. Using alluxio/integration/fuse/bin/alluxio-fuse mount -okernel_cache,allow_other,max_read=131072,attr_timeout=7200,entry_timeout=7200 -o ro /tmp/alluxio-fuse / to mount alluxio fuse.
time cat /tmp/alluxio-fuse/1g-file > /dev/null // 2min
time bin/alluxio fs cat /1g-file > /dev/null // 10sec

Expected behavior

cat a 1g file from fuse cost little than 30 sec would be accpeted.

Urgency

Urgency. it blocked our user to use our alluxio cluster on production env.

Are you planning to fix it

Not sure.

Additional context

Issue Analytics

State:
Created 2 years ago
Comments:6 (5 by maintainers)

Top GitHub Comments

2reactions

LuQQiucommented, Jan 13, 2022

Could be because gRPC overhead. Could be because fuse issues much more gRPC calls.

0reactions

LuQQiucommented, Oct 5, 2022

@maobaolong can you help validate if direct_io option solve the issue, thanks!

Top Results From Across the Web

List of Configuration Properties - Alluxio v2.9.0 (stable ...

Property Name Default Description alluxio.conf.dynamic.update.enabled false Whether to support dynamic update pro... alluxio.fuse.fs.name alluxio‑fuse The FUSE file system name. alluxio.fuse.special.command.enabled false

High CPU usage on Master node if the cache had been filled ...

we thought it was caused by storing metastore on ROCKS, we then turnover to HEAP, but it did not help us solve this...

参考- 配置项列表- 《Alluxio 社区版v2.6官方文档》 - 书栈网· BookStack

所有Alluxio配置属性都属于以下六类之一：共有配置项（由Master和Worker共享）， ... alluxio.fuse.cached.paths.max, 500, 用于FUSE conversion的Alluxio缓存路径的 ...

IsarError: Could not initialize IsarCore library for processor ...

**Version** - Platform: iOS, XCode 13.2.1 - Flutter version: 3.0.5 - Isar version: from ... Read a 1 GB file from alluxio-fuse 2min...

Alluxio - access existing files in underfs - Stack Overflow

I am running a small Alluxio (1.7.0) cluster using Swift as the underfs. I've confirmed Alluxio works great writing and reading files and...