Can alluxio mount oss bucket directly?
See original GitHub issueAlluxio Version: What version of Alluxio are you using?
2.7.3-SNAPSHOT
Describe the bug A clear and concise description of what the bug is.
To Reproduce Steps to reproduce the behavior (as minimally and precisely as possible)
I tried to mount oss bucket test
to alluxio file system, and run ls
. There are no files
alluxio fs mount --option fs.oss.endpoint=oss-cn-shanghai-internal.aliyuncs.com --option fs.oss.accessKeyId=abc --option fs.oss.accessKeySecret=efg /abc oss://test/
Mounted oss://wdl-shanghai/ at /abc
alluxio fs ls /abc
alluxio fs umount /abc
Unmounted /abc
Instead I tried with the directory oss://test/tfrecords
under the ossbucket, it works.
alluxio fs mount --option fs.oss.endpoint=oss-cn-shanghai-internal.aliyuncs.com --option fs.oss.accessKeyId=abc --option fs.oss.accessKeySecret=efg /abc oss://test/tfrecords
Mounted oss://wdl-shanghai/tfrecords at /abc
alluxio fs ls /abc
0 PERSISTED 01-19-2022 07:28:08:958 DIR /abc/transformed_metadata
1180 PERSISTED 12-28-2020 12:12:08:000 0% /abc/transformed_metadata.tar.gz
0 PERSISTED 01-19-2022 07:28:08:960 DIR /abc/train
I am wondering if it’s as design.
Expected behavior A clear and concise description of what you expected to happen.
Urgency Describe the impact and urgency of the bug.
Are you planning to fix it Please indicate if you are already working on a PR.
Additional context Add any other context about the problem here.
Issue Analytics
- State:
- Created 2 years ago
- Comments:8 (7 by maintainers)
Top Results From Across the Web
Aliyun Object Storage Service - Alluxio v2.8.0 (stable ...
Edit conf/alluxio-site.properties file to set the under storage address to the OSS bucket and the OSS directory you want to mount to Alluxio....
Read more >Aliyun Object Storage Service - 《Alluxio CE v1.8 ... - 书栈网
An OSS location can be either mounted at the root of the Alluxio ... Specify an existing OSS bucket and directory as the...
Read more >E-MapReduce:FAQ - Alluxio - Alibaba Cloud
You can run the following command to accelerate access to data in OSS by caching data: alluxio fs mount --option fs.oss.
Read more >Failed to distribute load small files · Issue #11859 - GitHub
Alluxio Version: alluxio version -r 2.3.1-SNAPSHOT-fd10ac016c6315275f1a91f845a545e28b2602ad Describe the bug The OSS bucket is 92G, ...
Read more >How to Set Up Object Storage | JuiceFS Document Center
--bucket https://bucketName.oss-cn-hangzhou.aliyuncs.com \ ... Newly mounted clients will use the new credentials directly, and all clients ...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
synced with @fuzhengjia offline, we’re on it and we will get back to you guys soon once we have a solution.
https://github.com/Alluxio/alluxio/pull/15361 fix this issue