ERROR AlluxioJniFuseFileSystem - File is not completed
See original GitHub issueAlluxio Version: 2.5.0-1
Describe the bug After writing a file to an Alluxio FUSE mounted folder I start seeing the following message repeated in the alluxio-fuse logs:
ERROR AlluxioJniFuseFileSystem - File /test.txt is not completed
When trying to read the file from the folder I get the following local error:
$ cat ${MOUNT_POINT}/test.txt
cat: /mnt/fuse_parent_volume/alluxio_fuse/test.txt: Input/output error
And see the following in the alluxio-fuse process logs:
alluxio.exception.FileIncompleteException: Cannot read from /test.txt because it is incomplete. Wait for the file to be marked as complete by the writing thread or application.
at alluxio.client.file.BaseFileSystem.openFile(BaseFileSystem.java:395)
at alluxio.client.file.BaseFileSystem.openFile(BaseFileSystem.java:383)
at alluxio.client.file.FileSystem.openFile(FileSystem.java:498)
at alluxio.fuse.AlluxioJniFuseFileSystem.openInternal(AlluxioJniFuseFileSystem.java:330)
at alluxio.fuse.AlluxioJniFuseFileSystem.lambda$open$4(AlluxioJniFuseFileSystem.java:321)
at alluxio.fuse.AlluxioFuseUtils.call(AlluxioFuseUtils.java:278)
at alluxio.fuse.AlluxioJniFuseFileSystem.open(AlluxioJniFuseFileSystem.java:321)
at alluxio.jnifuse.AbstractFuseFileSystem.openCallback(AbstractFuseFileSystem.java:171)
Also ls ${MOUNT_POINT}
takes much longer than usual, and is in the order of multiple seconds each time.
The weird thing is the file is in Alluxio (from the Master dashboard):
To Reproduce
- Launch Alluxio Master and Worker
- Launch Alluxio fuse
- Write simple test file to mount point, e.g.
echo "test" > ${MOUNT_POINT}/test.txt
Expected behavior I expect Alluxio FUSE to work
Urgency Low
Additional context
- Running in Docker.
- FUSE config:
alluxio.fuse.jnifuse.enabled=true
alluxio.user.file.readtype.default=CACHE
alluxio.user.file.writetype.default=ASYNC_THROUGH
alluxio.fuse.maxwrite.bytes=131072
alluxio.fuse.shared.caching.reader.enabled=false
alluxio.user.metadata.cache.enabled=true
Issue Analytics
- State:
- Created 2 years ago
- Comments:8 (8 by maintainers)
Top Results From Across the Web
FUSE-based POSIX API - Alluxio v2.9.0 (stable) Documentation
The Alluxio POSIX API is based on the Filesystem in Userspace (FUSE) project. Most basic file system operations are supported. However, given the...
Read more >List of Configuration Properties - Alluxio v2.9.0 (stable ...
Property Name Default Description
alluxio.conf.dynamic.update.enabled false Whether to support dynamic update pro...
alluxio.fuse.fs.name alluxio‑fuse The FUSE file system name.
alluxio.fuse.special.command.enabled false
Read more >Uses of Class alluxio.exception.status.AlluxioStatusException
Exception indicating that some requested entity (e.g., file or directory) was not found. class, OutOfRangeException. Exception indicating that and operation ...
Read more >StackFS (Alluxio Parent 2.6.2 API)
Stack FS implements the FUSE callbacks defined by jni-fuse without interactions with Alluxio clients/servers. Stack FS mounts a local filesystem path to ...
Read more >List of Metrics - Alluxio v2.9.0 (stable) Documentation
Name Type Description
Master.AbsentCacheHits GAUGE Number of cache hits on the absent cache
Master.AbsentCacheMisses GAUGE Number of cache misses on the absent cache
Master.AbsentCacheSize GAUGE Size...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@peterroelants Metadata cache should be enabled only in read-only workloads with data/metadata remains unchanged (no file modification through either alluxio or UFS directly). Otherwise AlluxioFuse will cache the wrong metadata and will not go to the Alluxio servers or UFS to fetch the updated metadata information. In your case, AlluxioFuse cached the file metadata which shows that the file is not completed. Even though the metadata is updated in Alluxio servers (when you do ls, you see the correct metadata), AlluxioFuse keeps the old metadata and doesn’t update those metadata.
@apc999 Thank you. Just so you know the urgency for me personally became quite low atm since I can get away with
alluxio.user.metadata.cache.enabled=false
most of the time.