Add support for pyenv

I’m trying to build a standalone project of a simple test file importing numpy:

python -m nuitka --follow-imports --standalone --plugin-enable=numpy --plugin-enable=pylint-warnings test.py

I get the following error:

undefined symbols for architecture x86_64:████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 994/995
  "_libintl_bind_textdomain_codeset", referenced from:
      _PyIntl_bind_textdomain_codeset in libpython3.7m.a(_localemodule.o)
  "_libintl_bindtextdomain", referenced from:
      _PyIntl_bindtextdomain in libpython3.7m.a(_localemodule.o)
  "_libintl_dcgettext", referenced from:
      _PyIntl_dcgettext in libpython3.7m.a(_localemodule.o)
  "_libintl_dgettext", referenced from:
      _PyIntl_dgettext in libpython3.7m.a(_localemodule.o)
  "_libintl_gettext", referenced from:
      _PyIntl_gettext in libpython3.7m.a(_localemodule.o)
  "_libintl_setlocale", referenced from:
      _PyLocale_setlocale in libpython3.7m.a(_localemodule.o)
      _PyLocale_localeconv in libpython3.7m.a(_localemodule.o)
  "_libintl_textdomain", referenced from:
      _PyIntl_textdomain in libpython3.7m.a(_localemodule.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
                                                                                                                                                                      scons: *** [/Users/loretoparisi/Documents/Projects/AI/nuitka-test/test.dist/test] Error 1   

python --version
Python 3.7.10

mbploreto:nuitka-test loretoparisi$  python -m nuitka --version
0.6.14.4
Python: 3.7.10 (default, Apr 29 2021, 22:08:48) 
Executable: /Users/loretoparisi/.pyenv/versions/3.7.10/bin/python
OS: Darwin
Arch: x86_64

The whole compilation logs

Nuitka-Options:INFO: Used command line options: --follow-imports --standalone --plugin-enable=numpy --plugin-enable=pylint-warnings test.py
Nuitka-Options:INFO: Following all imports is the default for standalone mode and need not be specified.
Nuitka:INFO: Starting Python compilation.
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'tkinter' added '_tkinter'.                                                                     
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'socket' added '_socket'.
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'ctypes' added '_ctypes'.                                                                       
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'numpy' added 'numpy.compat,numpy.lib,numpy.linalg,numpy.fft,numpy.polynomial,numpy.random,numpy.random._bounded_integers,numpy.random._common,numpy.random._generator,numpy.random._mt19937,numpy.random._pcg64,numpy.random._philox,numpy.random._sfc64,numpy.random.bit_generator,numpy.random.mtrand,numpy.ctypeslib,numpy.ma,numpy.matrixlib'.
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'pkg_resources.extern' added 'pkg_resources._vendor.packaging,pkg_resources._vendor.pyparsing,pkg_resources._vendor.six,pkg_resources._vendor.appdirs'.                                                                                                                
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'pkg_resources._vendor.packaging' added 'pkg_resources._vendor.packaging.version,pkg_resources._vendor.packaging.specifiers,pkg_resources._vendor.packaging.requirements'.
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'brotli._brotli' added '_cffi_backend'.
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'requests.packages' added 'urllib3,urllib3._collections,urllib3.connection,urllib3.connectionpool,urllib3.contrib,urllib3.contrib.appengine,urllib3.exceptions,urllib3.fields,urllib3.filepost,urllib3.packages,urllib3.packages.six,urllib3.packages.ssl_match_hostname,urllib3.poolmanager,urllib3.request,urllib3.response,urllib3.util,urllib3.util.connection,urllib3.util.queue,urllib3.util.request,urllib3.util.response,urllib3.util.retry,urllib3.util.ssl_,urllib3.util.timeout,urllib3.util.url,urllib3.util.wait,urllib.error,urllib.parse,urllib.request,urllib.response'.
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'cryptography' added '_cffi_backend'.
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'urllib3' added 'urllib3._collections,urllib3.connection,urllib3.connectionpool,urllib3.contrib,urllib3.contrib.appengine,urllib3.exceptions,urllib3.fields,urllib3.filepost,urllib3.packages,urllib3.packages.six,urllib3.packages.ssl_match_hostname,urllib3.poolmanager,urllib3.request,urllib3.response,urllib3.util,urllib3.util.connection,urllib3.util.queue,urllib3.util.request,urllib3.util.response,urllib3.util.retry,urllib3.util.ssl_,urllib3.util.timeout,urllib3.util.url,urllib3.util.wait,urllib.error,urllib.parse,urllib.request,urllib.response'.                                               
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'matplotlib.backends' added 'matplotlib.backends._backend_agg,matplotlib.backends._tkagg,matplotlib.backends.backend_tkagg,matplotlib.backends.backend_agg'.
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'matplotlib.backends.backend_webagg' added 'matplotlib.backends.backend_webagg_core,tornado'.
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'matplotlib.backends.backend_webagg_core' added 'matplotlib.backends.backend_webagg,tornado'.   
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'matplotlib' added 'matplotlib.backend_managers,matplotlib.backend_bases,mpl_toolkits'.
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'pycparser.c_parser' added 'pycparser.yacctab,pycparser.lextab'.
Nuitka-Plugins:INFO: pyzmq: Implicit dependencies of module 'zmq.backend' added 'zmq.backend.cython'.                                                                  
Nuitka-Plugins:WARNING: Use '--plugin-enable=pkg-resources' for: Resolve pkg_resources version information.
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'scipy._lib' added 'scipy._lib.messagestream'.                                                  
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'numpy.random' added 'numpy.random._bounded_integers,numpy.random._common,numpy.random._generator,numpy.random._mt19937,numpy.random._pcg64,numpy.random._philox,numpy.random._sfc64,numpy.random.bit_generator,numpy.random.mtrand'.
Nuitka-Plugins:INFO: implicit-imports: Implicit dependencies of module 'numpy.core' added 'numpy.core._dtype_ctypes,numpy.core._multiarray_tests'.                     
                                                                                                                                                                      Nuitka:INFO: Completed Python level compilation and optimization.                                                                                                       
Nuitka:INFO: Generating source code for C backend compiler.
Nuitka:INFO: Running data composer tool for optimal constant value handling.                                                                                           
Nuitka:INFO: Running C level backend compilation via Scons.
Nuitka-Scons:INFO: Backend C compiler: clang (clang).
Nuitka-Scons:INFO: Slow C compilation detected, used 60s so far, this might indicate scalability problems.                                                             
Nuitka-Scons:INFO: Running clang -o module.asn1crypto.core.o -c -std=c11 -fvisibility=hidden -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -O3 -pipe -D_XOPEN_SOURCE -D_NUITKA_STANDALONE -D__NUITKA_NO_ASSERT__ -D_NUITKA_CONSTANTS_FROM_INCBIN -D_NUITKA_FROZEN=165 -D_NUITKA_EXE -I/Users/loretoparisi/.pyenv/versions/3.7.10/include/python3.7m -I. -I/Users/loretoparisi/.pyenv/versions/3.7.10/lib/python3.7/site-packages/nuitka/build/include -I/Users/loretoparisi/.pyenv/versions/3.7.10/lib/python3.7/site-packages/nuitka/build/static_src module.asn1crypto.core.c took 60.24 seconds
Nuitka-Scons:INFO: Slow C compilation detected, used 60s so far, this might indicate scalability problems.
Nuitka-Scons:INFO: Running clang -o module.matplotlib.axes._axes.o -c -std=c11 -fvisibility=hidden -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -O3 -pipe -D_XOPEN_SOURCE -D_NUITKA_STANDALONE -D__NUITKA_NO_ASSERT__ -D_NUITKA_CONSTANTS_FROM_INCBIN -D_NUITKA_FROZEN=165 -D_NUITKA_EXE -I/Users/loretoparisi/.pyenv/versions/3.7.10/include/python3.7m -I. -I/Users/loretoparisi/.pyenv/versions/3.7.10/lib/python3.7/site-packages/nuitka/build/include -I/Users/loretoparisi/.pyenv/versions/3.7.10/lib/python3.7/site-packages/nuitka/build/static_src module.matplotlib.axes._axes.c took 98.03 seconds
Nuitka-Scons:INFO: Slow C compilation detected, used 60s so far, this might indicate scalability problems.
Nuitka-Scons:INFO: Running clang -o module.matplotlib.patches.o -c -std=c11 -fvisibility=hidden -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -O3 -pipe -D_XOPEN_SOURCE -D_NUITKA_STANDALONE -D__NUITKA_NO_ASSERT__ -D_NUITKA_CONSTANTS_FROM_INCBIN -D_NUITKA_FROZEN=165 -D_NUITKA_EXE -I/Users/loretoparisi/.pyenv/versions/3.7.10/include/python3.7m -I. -I/Users/loretoparisi/.pyenv/versions/3.7.10/lib/python3.7/site-packages/nuitka/build/include -I/Users/loretoparisi/.pyenv/versions/3.7.10/lib/python3.7/site-packages/nuitka/build/static_src module.matplotlib.patches.c took 64.55 seconds
Nuitka-Scons:INFO: Slow C compilation detected, used 60s so far, this might indicate scalability problems.
Nuitka-Scons:INFO: Running clang -o module.pyparsing.o -c -std=c11 -fvisibility=hidden -fwrapv -w -fvisibility=hidden -fvisibility-inlines-hidden -O3 -pipe -D_XOPEN_SOURCE -D_NUITKA_STANDALONE -D__NUITKA_NO_ASSERT__ -D_NUITKA_CONSTANTS_FROM_INCBIN -D_NUITKA_FROZEN=165 -D_NUITKA_EXE -I/Users/loretoparisi/.pyenv/versions/3.7.10/include/python3.7m -I. -I/Users/loretoparisi/.pyenv/versions/3.7.10/lib/python3.7/site-packages/nuitka/build/include -I/Users/loretoparisi/.pyenv/versions/3.7.10/lib/python3.7/site-packages/nuitka/build/static_src module.pyparsing.c took 90.84 seconds

[jtran]

I had the same error, but as mentioned in this comment, if you reinstall Python using this, it works.

env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 3.7.10

I always use pyenv for development. I’ve been using Nuitka with it for over a year now, and it (mostly) works with pyenv, despite claiming not to 🙂

[kayhayen]

This is now on the develop branch, please give it a spin, esp. on macOS and let me know how it goes with your existing setups, no need to change anything just yet.