question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

huge packages with long module names problem (sh: Argument list too long)

See original GitHub issue

Hi,

huge packages with long module names can not linked correctly.

System

  • Nuitka version, full Python version, flavor, OS, etc. as output by this command.

    python -m nuitka --version

1.0.6 Commercial: None Python: 3.9.0 (default, Nov 15 2020, 14:28:56) Flavor: Anaconda Python Executable: /home/user/anaconda39/envs/py39/bin/python OS: Linux Arch: x86_64 Distribution: Ubuntu (based on Debian) 20.04.4

  • How did you install Nuitka and Python

pip in ananconda env

python -m pip freeze

Nuitka==1.0.6


## Example
### create mymodule

"""
python -m nuitka --module m.py --include-package=mymodule
"""
def create_mymodule(n=2000):
    if not os.path.exists("mymodule"):
        os.mkdir("mymodule")

    with open(os.path.join("mymodule", "__init__.py"), "w") as fh:
        i = 1
        fh.write(f"""
import module{i:0200d}

print(module{i:0200d}.f())
""")

    for i in range(n):
        with open(os.path.join("mymodule", f"module{i:0200d}.py"), "[Shell Command Line Length Limitations](https://www.cyberciti.biz/faq/argument-list-too-long-error-solution/)w") as fh:
            fh.write(f"def f():return f'module{i:0200d}'")



if __name__ == '__main__':
    create_mymodule(2000)

create module m.py


import mymodule.module000001

print(mymodule.module000001.f())

nuitka command

python -m nuitka --module m.py --include-package=mymodule

Error

Nuitka-Options:INFO: Used command line options: --module m.py --include-package=mymodule
Nuitka:INFO: Starting Python compilation with Nuitka '1.0.6' on Python '3.9' commercial grade 'not installed'.
Nuitka:INFO: Completed Python level compilation and optimization.                                                                                                                                                                       
Nuitka:INFO: Generating source code for C backend compiler.
Nuitka:INFO: Running data composer tool for optimal constant value handling.                                                                                                                                                            
Nuitka:INFO: Running C compilation via Scons.
Nuitka-Scons:INFO: Backend C compiler: gcc (gcc).
Nuitka-Scons:INFO: Backend linking program with 2010 files (no progress information available).
scons: *** [/home/user/project/tests/m.cpython-39-x86_64-linux-gnu.so] sh: Argument list too long```

### Shell Command Line Length Limitation

```getconf ARG_MAX
2097152

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:7 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
kayhayencommented, Sep 6, 2022

So this now works on Linux for me, but sadly for mingw it’s not a string, but a generator with code behind that. Might have to monkey patch around there.

1reaction
kayhayencommented, Sep 6, 2022

Ah, so LINKCOM is used in scons for programs, and SHLINKCOM for modules. That one will have to be patched in much the same way.

Read more comments on GitHub >

github_iconTop Results From Across the Web

kernel - "Argument list too long": How do I deal with it, without ...
The standard advice is to change the command I use, to avoid requiring so much space for arguments (e.g., use find and xargs...
Read more >
The “Argument List Too Long” Error in Linux Commands
In this tutorial, we'll look at the “argument list too long” problem, often encountered while working with a large number of files.
Read more >
Linux/UNIX: Argument list too long error for rm, cp, mv ...
I'm getting “Argument list too long” message. How do I solve this problem when executing commands in the bash/ksh/csh/tcsh/zsh shell?
Read more >
Why do I get "/bin/sh: Argument list too long" when passing ...
Just for completeness: /bin/bash -c '/bin/true "$(seq 1 100000)"' breaks too because of how the shell tokenizes strings. +1 for solving for Y...
Read more >
300867 – net-libs/webkit-gtk crash: make argument list is too ...
Could you now try again as sys-devel/make now comes with the patch that should fix the argument list too long issue.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found