Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

java.lang.IllegalArgumentException: Can not create a Path from a null string

See original GitHub issue

I’m unable to insert data to HBase, because my job is failing with exception:

App > 16/07/06 12:16:58 task-result-getter-1 WARN TaskSetManager: Lost task 3.0 in stage 0.0 (TID 3, ip-172-31-19-56.eu-west-1.compute.internal): java.lang.IllegalArgumentException: Can not create a Path from a null string
App > at org.apache.hadoop.fs.Path.checkPathArg(Path.java:125)
App > at org.apache.hadoop.fs.Path.(Path.java:137)
App > at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1$$anonfun$13.apply(PairRDDFunctions.scala:1197)
App > at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1$$anonfun$13.apply(PairRDDFunctions.scala:1190)
App > at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
App > at org.apache.spark.scheduler.Task.run(Task.scala:89)
App > at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
App > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
App > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
App > at java.lang.Thread.run(Thread.java:745)

my hbase-site.xml is:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
</property>
<property>
    <name>zookeeper.recovery.retry</name>
    <value>3</value>
</property>
<property>
    <name>hbase.regionserver.info.port</name>
    <value>16030</value>
</property>
<property>
    <name>hbase.zookeeper.quorum</name>
    <value>remote_IP</value>
</property>
<property>
    <name>hbase.rootdir</name>
    <value>hdfs://remote_IP:9000/hbase</value>
</property>
<property>
    <name>hbase.fs.tmp.dir</name>
    <value>/user/${user.name}/hbase-staging</value>
</property>
<property>
    <name>hadoop.tmp.dir</name>
    <value>/tmp/hadoop-${user.name}</value>
</property>
    <!-- Put any other property here, it will be used -->
</configuration>

my Hbase version: HBase Version 1.0.0, revision=984db9a1cae088b996e997db9ce83f6d4bd565ad

Any suggesions?

Issue Analytics

State:
Created 7 years ago
Comments:7 (3 by maintainers)

Top GitHub Comments

7reactions

weiqingycommented, Jul 28, 2017

@mavencode01 The issue is hitting https://issues.apache.org/jira/browse/SPARK-21549. For reference: https://issues.apache.org/jira/browse/SPARK-18191.

Workaround is to explicitly set the property mapreduce.output.fileoutputformat.outputdir to a dummy value (which is valid and writable by user - say /tmp).

0reactions

mavencode01commented, Aug 9, 2017

@weiqingy sweet!!! Thanks for the quick reply. Appreciate