Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Lack of document about consistency.
See original GitHub issuePage https://www.alluxio.io/blog/4-different-ways-to-write-to-alluxio/
Summary
A main different between HDFS and S3 is consistency, this blog mentioned it, but details are lack. I hope in the summary table, there could be a column Consistency just like Write Speed and Fault Tolerance.
What I especially want to know is, what consistency Alluxio gives when under storage is S3, and Write Type is one of MUST_CACHE,THROUGH,CACHE_THROUGH,ASYNC_THROUGH, and what will happen when a worker crashes.
Issue Analytics
- State:
- Created 3 years ago
- Comments:8 (5 by maintainers)
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
No, the data will only be written to the UFS once
Sure thing I can clarify that part.
If you are writing with
ASYNC_THROUGHand the worker crashes before you persist the data, then you will incur a data loss. However, we do provide a feature we call “durable writes” which will split the write among 1 or more workers simultaneously during the write. This prevents data loss in the case of one or more machines. You can enable durable writes by setting the propertyalluxio.user.file.replication.durable. By default it is one, but if you want to ensure the availability of data in the case of a node loss, you may increase this to 2, 3, or more to produce replicas synchronously during the write.