Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Active Syncing is not supported on this UFS type: hdfs

See original GitHub issue

Alluxio Version: Alluxio version: v2.8.0

Other tools Version: k8s version: v1.20.11 docker version: 20.10.11 hdfs version: 3.3.2

Some background to add hdfs was deployed by binary file hdfs was started by user root:supergroup alluxio was deployed in k8s by helm run user root:root

Describe the bug [bug1] I can’t sync metadata and raw data from hdfs to alluxio. output exception: Active Syncing is not supported on this UFS type: hdfs [bug2] I can’t write data in ReadWriteMany PVC created by alluxio csi output xxx.txt" E514: write error (file system full?) To Reproduce

start sync alluxio dir in alluxio master pod
error log in alluxio master

I can touch new file in alluxio pvc, but can not write data in when write something into touched new file, error occured: or can not write into swap file
error log in alluxio master

Problem

I found out hdfs version 3.3.2 have enabled support for ACL, can not understand Active Syncing is not supported on this UFS type: hdfs
I do not understand the procession of write data, .swap file id needed, but alluxio can not sync it.

Issue Analytics

State:
Created a year ago
Comments:5

Top GitHub Comments

1reaction

Crazybean-lwbcommented, Aug 29, 2022

Make sure the parent path of your startSync path /liuweibin in the mount table specifies the hdfs version. you can use “alluxio fs mount --option alluxio.underfs.version=xxx”

I have tried many times mount hdfs with under.version, no differences happend.

I found sollution to [1. start sync alluxio dir in alluxio master pod] alluxio latest version surpport hdfs version 3.3.1 by default, while my hdfs version is 3.3.2. I build alluxio-underfs-hdfs jar following the document, I can resolve the problem

bug2 still can not be resolved. I still can not write data in ReadWriteMany PVC created by alluxio csi output xxx.txt" E514: write error (file system full?)

0reactions

Crazybean-lwbcommented, Aug 31, 2022

Make sure the parent path of your startSync path /liuweibin in the mount table specifies the hdfs version. you can use “alluxio fs mount --option alluxio.underfs.version=xxx”

I have tried many times mount hdfs with under.version, no differences happend.

I found sollution to [1. start sync alluxio dir in alluxio master pod]

alluxio latest version surpport hdfs version 3.3.1 by default, while my hdfs version is 3.3.2. I build alluxio-underfs-hdfs jar following the document, I can resolve the problem bug2 still can not be resolved. I still can not write data in ReadWriteMany PVC created by alluxio csi output xxx.txt" E514: write error (file system full?)

get some reply for bugs 2

Top Results From Across the Web

Two Ways to Keep Files in Sync Between Alluxio and HDFS

Sync Proactively Currently, Active UFS Sync is only supported between Alluxio and HDFS 2.7 or later. To use this feature, the user running ......

Two Ways to Keep Files in Sync Between Alluxio and HDFS

Currently, Active UFS Sync is only supported between Alluxio and HDFS 2.7 or later. To use this feature, the user running Alluxio must...

alluxio.underfs.UnderFileSystem.getUnderFSType java code ...

supportsActiveSync()) { throw new UnsupportedOperationException("Active Sync is not supported on this UFS type: " + ufsResource.get().getUnderFSType());.

HCAT_SYNC_OBJECTS stored procedure - IBM

Note: If automatic syncing has been disabled for whatever reason, tables in the Db2® Big SQL catalog might no longer match the table...

Issues - Alluxio - Projects - JIRA

Key Summary Assignee Reporter ALLUXIO‑3401 Add a query parameter to job list interface. Unassigned fangyan... ALLUXIO‑3400 EMR Bootstrap integration script fixes. Nakkul Sreenivas Alex Ma ALLUXIO‑3395...