question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

RuntimeError: Core module unavailable. Trying to import from the source directory?

See original GitHub issue

I installed with sudo pip2.7 install pydoop. But when I try to import and test any hdfs function, I get the following runtime error:

In [11]: import pydoop.hdfs as hdfs

In [12]: hdfs.ls("/user")
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-12-b865d4bfdb3c> in <module>()
----> 1 hdfs.ls("/user")

/home/hadoop/manish/pydoop/pydoop/hdfs/__init__.pyc in ls(hdfs_path, user, recursive)
    319     items are hdfs paths instead of dictionaries of properties.
    320     """
--> 321     dir_list = lsl(hdfs_path, user, recursive)
    322     return [d["name"] for d in dir_list]
    323

/home/hadoop/manish/pydoop/pydoop/hdfs/__init__.pyc in lsl(hdfs_path, user, recursive)
    298     """
    299     host, port, path_ = path.split(hdfs_path, user)
--> 300     fs = hdfs(host, port, user)
    301     if not recursive:
    302         dir_list = fs.list_directory(path_)

/home/hadoop/manish/pydoop/pydoop/hdfs/fs.pyc in __init__(self, host, port, user, groups)
    154             self.__status = self.__lookup((host, port, user))
    155         except KeyError:
--> 156             h, p, u, fs = _get_connection_info(host, port, user)
    157             aliasing_info = [] if user else [("user", u, user)]
    158             if h != "":

/home/hadoop/manish/pydoop/pydoop/hdfs/fs.pyc in _get_connection_info(host, port, user)
     71
     72 def _get_connection_info(host, port, user):
---> 73     fs = core_hdfs_fs(host, port, user)
     74     res = urlparse(fs.get_working_directory())
     75     if res.scheme == "file":

/home/hadoop/manish/pydoop/pydoop/hdfs/core/__init__.pyc in core_hdfs_fs(host, port, user)
     43         else:
     44             msg = "Check that Pydoop is correctly installed"
---> 45         raise RuntimeError("Core module unavailable. %s" % msg)
     46     return _CORE_MODULE.CoreHdfsFs(host, port, user)

RuntimeError: Core module unavailable. Trying to import from the source directory?

By the way, I installed it in Cloudera CDH 5 distribution.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:9 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
simleocommented, Mar 16, 2018

Your machine does not have to be part of the Hadoop cluster. It just needs to be able to connect to the cluster nodes via TCP/IP. Please refer to http://hadoop.apache.org/ for details on how to configure your Hadoop cluster.

0reactions
Atrbearcommented, Mar 16, 2018

THX, I think it may be the problem of the hadoop settings.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Py.test No module named * - Stack Overflow
In my test_app.py file , I have a line to import my app module. When I run py.test on the root folder, I...
Read more >
Error when importing TensorFlow | Apple Developer Forums
When I try to import tensorflow: import tensorflow. I get the following error: RuntimeError: module compiled against API version 0xe but this version...
Read more >
importlib — The implementation of import — Python 3.11.1 ...
This is useful if you have edited the module source file using an external editor and want to try out the new version...
Read more >
Source code for oggm.shop.rgitopo
Source code for oggm.shop.rgitopo. import logging import os import shutil try: import salem except ImportError: pass try: import rasterio except ...
Read more >
How to Fix PyCharm Import Error and Setup Your Interpreter
This will show you how to fix common pycharm import errors when trying to import python modules. Please watch through the entire video...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found