huge packages with long module names problem (sh: Argument list too long)
See original GitHub issueHi,
huge packages with long module names can not linked correctly.
System
-
Nuitka version, full Python version, flavor, OS, etc. as output by this command.
python -m nuitka --version
1.0.6 Commercial: None Python: 3.9.0 (default, Nov 15 2020, 14:28:56) Flavor: Anaconda Python Executable: /home/user/anaconda39/envs/py39/bin/python OS: Linux Arch: x86_64 Distribution: Ubuntu (based on Debian) 20.04.4
- How did you install Nuitka and Python
pip in ananconda env
python -m pip freeze
Nuitka==1.0.6
## Example
### create mymodule
"""
python -m nuitka --module m.py --include-package=mymodule
"""
def create_mymodule(n=2000):
if not os.path.exists("mymodule"):
os.mkdir("mymodule")
with open(os.path.join("mymodule", "__init__.py"), "w") as fh:
i = 1
fh.write(f"""
import module{i:0200d}
print(module{i:0200d}.f())
""")
for i in range(n):
with open(os.path.join("mymodule", f"module{i:0200d}.py"), "[Shell Command Line Length Limitations](https://www.cyberciti.biz/faq/argument-list-too-long-error-solution/)w") as fh:
fh.write(f"def f():return f'module{i:0200d}'")
if __name__ == '__main__':
create_mymodule(2000)
create module m.py
import mymodule.module000001
print(mymodule.module000001.f())
nuitka command
python -m nuitka --module m.py --include-package=mymodule
Error
Nuitka-Options:INFO: Used command line options: --module m.py --include-package=mymodule
Nuitka:INFO: Starting Python compilation with Nuitka '1.0.6' on Python '3.9' commercial grade 'not installed'.
Nuitka:INFO: Completed Python level compilation and optimization.
Nuitka:INFO: Generating source code for C backend compiler.
Nuitka:INFO: Running data composer tool for optimal constant value handling.
Nuitka:INFO: Running C compilation via Scons.
Nuitka-Scons:INFO: Backend C compiler: gcc (gcc).
Nuitka-Scons:INFO: Backend linking program with 2010 files (no progress information available).
scons: *** [/home/user/project/tests/m.cpython-39-x86_64-linux-gnu.so] sh: Argument list too long```
### Shell Command Line Length Limitation
```getconf ARG_MAX
2097152
Issue Analytics
- State:
- Created a year ago
- Comments:7 (7 by maintainers)
Top Results From Across the Web
kernel - "Argument list too long": How do I deal with it, without ...
The standard advice is to change the command I use, to avoid requiring so much space for arguments (e.g., use find and xargs...
Read more >The “Argument List Too Long” Error in Linux Commands
In this tutorial, we'll look at the “argument list too long” problem, often encountered while working with a large number of files.
Read more >Linux/UNIX: Argument list too long error for rm, cp, mv ...
I'm getting “Argument list too long” message. How do I solve this problem when executing commands in the bash/ksh/csh/tcsh/zsh shell?
Read more >Why do I get "/bin/sh: Argument list too long" when passing ...
Just for completeness: /bin/bash -c '/bin/true "$(seq 1 100000)"' breaks too because of how the shell tokenizes strings. +1 for solving for Y...
Read more >300867 – net-libs/webkit-gtk crash: make argument list is too ...
Could you now try again as sys-devel/make now comes with the patch that should fix the argument list too long issue.
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
So this now works on Linux for me, but sadly for mingw it’s not a string, but a generator with code behind that. Might have to monkey patch around there.
Ah, so LINKCOM is used in scons for programs, and SHLINKCOM for modules. That one will have to be patched in much the same way.