Error using s3 protocol for marathon application resource: hadoop command not found
See original GitHub issueWhen deploying a marathon application with a s3 uri the mesos task fails to staging with the following error in the sandbox:
E0605 14:18:44.685948 28351 shell.hpp:93] Command 'hadoop version 2>&1' failed; this is the output:
sh: hadoop: command not found
Failed to fetch 's3://bucket/object.tar.gz': Failed to create HDFS client: Failed to execute 'hadoop version 2>&1'; the command was either not found or exited with a non-zero exit status: 127
- Ansible version (
ansible --version
): ansible 1.9.6 - Python version (
python --version
): Python 2.7.10 - Git commit hash or branch: 1.1
- Cloud Environment: AWS
- Terraform version (
terraform version
): Terraform v0.6.15
Issue Analytics
- State:
- Created 7 years ago
- Comments:6 (2 by maintainers)
Top Results From Across the Web
Troubleshooting - Apache Hadoop
hadoop.fs.s3a.S3AFileSystem. These are Hadoop filesystem client classes, found in the hadoop-aws JAR. An exception reporting this class as missing means ...
Read more >Spark + s3 - error - java.lang.ClassNotFoundException ...
S3AFileSystem These are Hadoop filesystem client classes, found in the `hadoop-aws` JAR. An exception reporting this class as missing means that ...
Read more >Resolve HTTP 403 "Access Denied" AmazonS3Exception ...
First, check the credentials or role specified in your application code. Run the following command on the EMR cluster's master node. Replace s3...
Read more >Solved: I run a Hadoop job, but it got stucked and nothing...
Solved: I run this command - hadoop jar ... Job: Running job: job_1479831488614_0002 ... log4j:ERROR Could not find value for key log4j.appender.
Read more >May 18, 2022•Knowledge 000153675 - Search
ClassNotFoundException:Class com.amazon.ws.emr.hadoop.fs.EmrFileSystem not found]" while running Amazon S3 mapping on Spark engine with EMR ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
From the Mesos Fetcher documentation:
Also running into the same error. Task has nothing to do with Hadoop (it’s just a “hello world” container) and I’m getting the output: