Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Installing extension modules in setup.py?

See original GitHub issue

Is there a reason for not building the extension modules in the setup.py file, but generate them on the fly when needed?

This seems to give me permission errors when working with a python interpreter that I do not have write access to:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.5/site-packages/pyximport/pyximport.py", line 216, in load_module
    inplace=build_inplace, language_level=language_level)
  File "/opt/conda/lib/python3.5/site-packages/pyximport/pyximport.py", line 192, in build_module
    reload_support=pyxargs.reload_support)
  File "/opt/conda/lib/python3.5/site-packages/pyximport/pyxbuild.py", line 102, in pyx_to_dll
    dist.run_commands()
  File "/opt/conda/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/opt/conda/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.5/site-packages/Cython/Distutils/old_build_ext.py", line 185, in run
    _build_ext.build_ext.run(self)
  File "/opt/conda/lib/python3.5/distutils/command/build_ext.py", line 338, in run
    self.build_extensions()
  File "/opt/conda/lib/python3.5/site-packages/Cython/Distutils/old_build_ext.py", line 192, in build_extensions
    ext.sources = self.cython_sources(ext.sources, ext)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Distutils/old_build_ext.py", line 345, in cython_sources
    full_module_name=module_name)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Compiler/Main.py", line 684, in compile
    return compile_single(source, options, full_module_name)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Compiler/Main.py", line 634, in compile_single
    return run_pipeline(source, options, full_module_name)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Compiler/Main.py", line 494, in run_pipeline
    err, enddata = Pipeline.run_pipeline(pipeline, source)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Compiler/Pipeline.py", line 340, in run_pipeline
    data = phase(data)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Compiler/Pipeline.py", line 53, in generate_pyx_code_stage
    module_node.process_implementation(options, result)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Compiler/ModuleNode.py", line 137, in process_implementation
    self.generate_c_code(env, options, result)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Compiler/ModuleNode.py", line 387, in generate_c_code
    f = open_new_file(result.c_file)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Utils.py", line 65, in open_new_file
    return codecs.open(path, "w", encoding="ISO-8859-1")
  File "/opt/conda/lib/python3.5/codecs.py", line 895, in open
    file = builtins.open(filename, mode, buffering)
PermissionError: [Errno 13] Permission denied: '/opt/conda/lib/python3.5/site-packages/umi_tools/_dedup_umi.c'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.5/site-packages/umi_tools/network.py", line 20, in <module>
    from umi_tools._dedup_umi import edit_distance
  File "/opt/conda/lib/python3.5/site-packages/pyximport/pyximport.py", line 445, in load_module
    language_level=self.language_level)
  File "/opt/conda/lib/python3.5/site-packages/pyximport/pyximport.py", line 232, in load_module
    raise exc.with_traceback(tb)
  File "/opt/conda/lib/python3.5/site-packages/pyximport/pyximport.py", line 216, in load_module
    inplace=build_inplace, language_level=language_level)
  File "/opt/conda/lib/python3.5/site-packages/pyximport/pyximport.py", line 192, in build_module
    reload_support=pyxargs.reload_support)
  File "/opt/conda/lib/python3.5/site-packages/pyximport/pyxbuild.py", line 102, in pyx_to_dll
    dist.run_commands()
  File "/opt/conda/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/opt/conda/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.5/site-packages/Cython/Distutils/old_build_ext.py", line 185, in run
    _build_ext.build_ext.run(self)
  File "/opt/conda/lib/python3.5/distutils/command/build_ext.py", line 338, in run
    self.build_extensions()
  File "/opt/conda/lib/python3.5/site-packages/Cython/Distutils/old_build_ext.py", line 192, in build_extensions
    ext.sources = self.cython_sources(ext.sources, ext)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Distutils/old_build_ext.py", line 345, in cython_sources
    full_module_name=module_name)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Compiler/Main.py", line 684, in compile
    return compile_single(source, options, full_module_name)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Compiler/Main.py", line 634, in compile_single
    return run_pipeline(source, options, full_module_name)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Compiler/Main.py", line 494, in run_pipeline
    err, enddata = Pipeline.run_pipeline(pipeline, source)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Compiler/Pipeline.py", line 340, in run_pipeline
    data = phase(data)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Compiler/Pipeline.py", line 53, in generate_pyx_code_stage
    module_node.process_implementation(options, result)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Compiler/ModuleNode.py", line 137, in process_implementation
    self.generate_c_code(env, options, result)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Compiler/ModuleNode.py", line 387, in generate_c_code
    f = open_new_file(result.c_file)
  File "/opt/conda/lib/python3.5/site-packages/Cython/Utils.py", line 65, in open_new_file
    return codecs.open(path, "w", encoding="ISO-8859-1")
  File "/opt/conda/lib/python3.5/codecs.py", line 895, in open
    file = builtins.open(filename, mode, buffering)
ImportError: Building module umi_tools._dedup_umi failed: ["PermissionError: [Errno 13] Permission denied: '/opt/conda/lib/python3.5/site-packages/umi_tools/_dedup_umi.c'\n"]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.5/site-packages/umi_tools/dedup.py", line 279, in <module>
    import umi_tools.network as network
  File "/opt/conda/lib/python3.5/site-packages/umi_tools/network.py", line 24, in <module>
    from _dedup_umi import edit_distance
ImportError: No module named '_dedup_umi'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/opt/conda/bin/umi_tools", line 11, in <module>
    load_entry_point('umi-tools==0.4.3', 'console_scripts', 'umi_tools')()
  File "/opt/conda/lib/python3.5/site-packages/umi_tools/umi_tools.py", line 43, in main
    module = imp.load_module(command, file, pathname, description)
  File "/opt/conda/lib/python3.5/imp.py", line 234, in load_module
    return load_source(name, filename, file)
  File "/opt/conda/lib/python3.5/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 693, in _load
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 665, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/opt/conda/lib/python3.5/site-packages/umi_tools/dedup.py", line 281, in <module>
    import network
ImportError: No module named 'network'

Issue Analytics

State:
Created 6 years ago
Comments:6 (3 by maintainers)

Top GitHub Comments

1reaction

IanSudberycommented, Aug 21, 2017

Dear @croth1 and @andreas-wilm, we think that these issues have been solved in 0.5.

Extensions are now compiled at install time
Errors should now not show up as import errors of unrelated modules.

We are closing this for now, please feel free to reopen if you are still having issues.

1reaction

IanSudberycommented, Jun 30, 2017

We will try and get this done in 0.5

Top Results From Across the Web

2. Writing the Setup Script — Python 3.11.1 documentation

The setup script is the centre of all activity in building, distributing, and installing modules using the Distutils. The main purpose of the...

Building Extension Modules - Setuptools

Setuptools can build C/C++ extension modules. The keyword argument ext_modules of setup() should be a list of instances of the setuptools.Extension class.

How to use setuptools packages and ext_modules with the ...

The problem is that when I install it, my Python C Extension module is overridden by fastfilepackage/version.py and ...

Writing setup.py for extension module question - Google Groups

Hi, I want to build and install a package which contains a cython based extension module, some other python modules and a shared...

3. Building C and C++ Extensions with distutils - Read the Docs

Building an extension module using distutils requires that distutils is installed on the build machine, which is included in Python 2.x and available ......