question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Can alluxio mount oss bucket directly?

See original GitHub issue

Alluxio Version: What version of Alluxio are you using?

2.7.3-SNAPSHOT

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior (as minimally and precisely as possible)

I tried to mount oss bucket test to alluxio file system, and run ls. There are no files

alluxio fs mount --option fs.oss.endpoint=oss-cn-shanghai-internal.aliyuncs.com --option fs.oss.accessKeyId=abc --option fs.oss.accessKeySecret=efg /abc oss://test/
Mounted oss://wdl-shanghai/ at /abc
alluxio fs ls /abc
alluxio fs umount /abc
Unmounted /abc

Instead I tried with the directory oss://test/tfrecords under the ossbucket, it works.

alluxio fs mount --option fs.oss.endpoint=oss-cn-shanghai-internal.aliyuncs.com --option fs.oss.accessKeyId=abc --option fs.oss.accessKeySecret=efg /abc oss://test/tfrecords
Mounted oss://wdl-shanghai/tfrecords at /abc
alluxio fs ls /abc
              0       PERSISTED 01-19-2022 07:28:08:958  DIR /abc/transformed_metadata
           1180       PERSISTED 12-28-2020 12:12:08:000   0% /abc/transformed_metadata.tar.gz
              0       PERSISTED 01-19-2022 07:28:08:960  DIR /abc/train

I am wondering if it’s as design.

Expected behavior A clear and concise description of what you expected to happen.

Urgency Describe the impact and urgency of the bug.

Are you planning to fix it Please indicate if you are already working on a PR.

Additional context Add any other context about the problem here.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:8 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
beinancommented, Feb 24, 2022

synced with @fuzhengjia offline, we’re on it and we will get back to you guys soon once we have a solution.

0reactions
HelloHorizoncommented, May 9, 2022
Read more comments on GitHub >

github_iconTop Results From Across the Web

Aliyun Object Storage Service - Alluxio v2.8.0 (stable ...
Edit conf/alluxio-site.properties file to set the under storage address to the OSS bucket and the OSS directory you want to mount to Alluxio....
Read more >
Aliyun Object Storage Service - 《Alluxio CE v1.8 ... - 书栈网
An OSS location can be either mounted at the root of the Alluxio ... Specify an existing OSS bucket and directory as the...
Read more >
E-MapReduce:FAQ - Alluxio - Alibaba Cloud
You can run the following command to accelerate access to data in OSS by caching data: alluxio fs mount --option fs.oss.
Read more >
Failed to distribute load small files · Issue #11859 - GitHub
Alluxio Version: alluxio version -r 2.3.1-SNAPSHOT-fd10ac016c6315275f1a91f845a545e28b2602ad Describe the bug The OSS bucket is 92G, ...
Read more >
How to Set Up Object Storage | JuiceFS Document Center
--bucket https://bucketName.oss-cn-hangzhou.aliyuncs.com \ ... Newly mounted clients will use the new credentials directly, and all clients ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found