question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

PrestoExtendedFileSystemCache have not take LocalFileSystem into consideration

See original GitHub issue

We know that in Presto, Hadoop’s FileSystem#cache is replaced by PrestoExtendedFileSystemCache, and in PrestoExtendedFileSystemCache, all the FileSytems are wrapped by HadoopExtendedFileSystem:

    // PrestoExtendedFileSystemCache.java
    @Override
    protected FileSystem createPrestoFileSystemWrapper(FileSystem original)
    {
        return new HadoopExtendedFileSystem(original);
    }

The code has not take LocalFileSystem into consideration, many FileSystem implementation utilize LocalFileSystem to implement multipart upload(write) data into the FileSystem. When we get a LocalFileSystem, the type cast will fail:

// FileSystem.java
  public static LocalFileSystem getLocal(Configuration conf)
    throws IOException {
    return (LocalFileSystem)get(LocalFileSystem.NAME, conf);
  }

Issue Analytics

  • State:open
  • Created 3 years ago
  • Reactions:1
  • Comments:9 (8 by maintainers)

github_iconTop GitHub Comments

3reactions
highkercommented, Jun 12, 2020

@jainxrohit, could you help to take a look?

0reactions
xumingmingcommented, Jun 7, 2021

@Naveen007 @highker do you have any suggestion for this issue?

Read more comments on GitHub >

github_iconTop Results From Across the Web

oozie error: Accessing local file system is not allowed
I have followed the following steps. 1.Started oozie server. 2.edited job.properties and workflow.xml files. 3.copied workflow.xml into hdfs. 4.
Read more >
Activating storage to local file system is not working - Node-RED
Dear all, I'm currently struggeling with the activation ot the local file storage as described in the following links: Working with context ...
Read more >
Use Cloud Storage as a mounted local file system in Vertex AI ...
This blog post introduces the feature Cloud Storage FUSE to Vertex AI and AI Platform users. This feature enables the training jobs on...
Read more >
How to work with files on Databricks
You can work with files on DBFS, the local driver node of the cluster, cloud object storage, external locations, and in Databricks Repos....
Read more >
Chapter 1. Overview of available file systems
If the speed of access is of prime importance, then use NFS to export a local file system like XFS. Shared storage file...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found