Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Scons: Argument list too long

See original GitHub issue

$ python -m nuitka --version
0.6.2
Python: 3.6.7 (default, Feb 26 2019, 11:59:03)
Executable: /home/sridhar/devel/python-3.6.7/malenv3/bin/python
OS: Linux
Arch: x86_64

I’m trying to convert a sklearn/tensorflow based project into a standalone executable. The compilations exits because of the size of the argument list:

$python -m nuitka --follow-imports --standalone --follow-stdlib   -j 6 --show-progress run_server.py
...
Nuitka:INFO:Optimizing module '_codecs_iso2022', 3 more modules to go after that.
Nuitka:INFO:Optimizing module '_codecs_hk', 2 more modules to go after that.
Nuitka:INFO:Optimizing module '_codecs_cn', 1 more modules to go after that.
Nuitka:INFO:Optimizing module '_bz2', 0 more modules to go after that.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/packages/six.py:74' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/ipython_genutils/importstring.py:31' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/ipython_genutils/importstring.py:39' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/jedi/evaluate/compiled/access.py:140' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/joblib/_memmapping_reducer.py:403' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/joblib/externals/cloudpickle/cloudpickle.py:965' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/joblib/externals/cloudpickle/cloudpickle.py:1000' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/joblib/externals/cloudpickle/cloudpickle.py:1196' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/numpy/core/function_base.py:453' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/numpy/lib/utils.py:366' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/numpy/lib/utils.py:865' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/numpy/lib/utils.py:923' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/pandas/compat/pickle_compat.py:160' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/pygments/formatters/__init__.py:39' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/pygments/lexers/__init__.py:41' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/pygments/styles/__init__.py:64' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/requests/packages.py:7' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/scipy/_lib/_util.py:160' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/scipy/_lib/six.py:74' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/scipy/optimize/optimize.py:3041' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/sklearn/externals/joblib/_memmapping_reducer.py:403' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:944' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:979' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:1179' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/sklearn/externals/six.py:76' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/sklearn/utils/testing.py:663' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/traitlets/utils/importstring.py:34' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/traitlets/utils/importstring.py:42' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved '__import__' call at '/home/sridhar/devel/python-3.6.7/malenv3/lib/python3.6/site-packages/urllib3/packages/six.py:82' may require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:INFO:Total memory usage before running scons: 4.70 GB (5047599104 bytes):
scons: *** [run_server.dist/run_server] sh: Argument list too long

Is it related to the warnings? How could I handle them? “–include-plugin-*” option is not available in nuitka.

Issue Analytics

State:
Created 5 years ago
Comments:19 (14 by maintainers)

Top GitHub Comments

2reactions

kayhayencommented, May 13, 2019

The next release meant is 0.6.4, the fix was considered too involved for a hotfix. And that is still not released. Notice the “develop” tag. That means the fix is only there.

1reaction

kayhayencommented, Apr 13, 2019

I pushed pushed the fix to develop, it will be part of the next pre-release package set, and subsequent release.

Top Results From Across the Web

Argument list too long error with makefile - Unix StackExchange

I'm on Xubuntu 18.10. Running make . hgignore with this, I no longer get the "Argument list too long" error, but the generated...

[SERVER-13829] Build failure: scons - argument list too long

Without pasting full build log here (its long), I run into issues while linking mongod since arguments passed to scons are super long....

Why do I get "/bin/sh: Argument list too long" when passing ...

The argument list gets expanded in the current shell when double quotes are used as evidenced by the fact that Bash is the...

Chapter 10. Controlling a Build From the Command Line

Option-arguments separated from long options by whitespace, rather than by an = , cannot be correctly resolved by SCons. While --input=ARG is clearly...

The “Argument List Too Long” Error in Linux Commands

In this tutorial, we'll look at the “argument list too long” problem, often encountered while working with a large number of files.