Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

"Unable to open fd 8 for writing, quitting" on Linux

See original GitHub issue

Relevant older issue: where I believe users also reported it happening on Linux but it was closed without any resolution (e.g. via suggested use of temporary files).

In my case I have a stock Debian (testing/unstable mix), python 3.8.3rc1 and I observe the same error with our DataLad which recently was fixed up to work (again) with argcomplete and it works for everyone else on the team but (un)lucky me. In the session you below can see that I am getting the exception, and the process has no 8 fd among its fds:

$> _ARC_DEBUG=1 _ARGCOMPLETE=1 datalad --pdb -l 1 --help
[DEBUG  ] Command line args 1st pass. Parsed: Namespace() Unparsed: ['--pdb', '--help'] 
[DEBUG  ] Discovering plugins 
[DEBUG  ] Loading entrypoints 
[DEBUG  ] Generating detailed description for the parser 
[Level 5] Finished setup_parser 
> /home/yoh/proj/misc/argcomplete/argcomplete/
-> try:
(Pdb) l
183  	        except:
184  	            debug_stream = sys.stderr
186  	        if output_stream is None:
187  	            import pdb; pdb.set_trace()
188  ->	            try:
189  	                output_stream = os.fdopen(8, "wb")
190  	            except:
191  	                debug("Unable to open fd 8 for writing, quitting")
192  	                exit_method(1)
(Pdb) os.getpid()
[3]  + 1092474 suspended  _ARC_DEBUG=1 _ARGCOMPLETE=1 datalad --pdb -l 1 --help

$> ls -l /proc/1092474/fd 
total 0
lrwx------ 1 yoh yoh 64 May 19 13:48 0 -> /dev/pts/16
lrwx------ 1 yoh yoh 64 May 19 13:48 1 -> /dev/pts/16
lrwx------ 1 yoh yoh 64 May 19 13:48 2 -> /dev/pts/16
lr-x------ 1 yoh yoh 64 May 19 13:48 3 -> pipe:[41232747]
lr-x------ 1 yoh yoh 64 May 19 13:48 5 -> pipe:[41232748]

$> fg
[3]    1092474 continued  _ARC_DEBUG=1 _ARGCOMPLETE=1 datalad --pdb -l 1 --help
194  	        # print("", stream=debug_stream)
196  	        #     print(v, os.environ[v], stream=debug_stream)
198  	        ifs = os.environ.get("_ARGCOMPLETE_IFS", "\013")
199  	        if len(ifs) != 1:
200  	            debug("Invalid value for IFS, quitting [{v}]".format(v=ifs))
201  	            exit_method(1)
203  	        comp_line = os.environ["COMP_LINE"]
204  	        comp_point = int(os.environ["COMP_POINT"])
(Pdb) n
> /home/yoh/proj/misc/argcomplete/argcomplete/
-> output_stream = os.fdopen(8, "wb")
OSError: [Errno 9] Bad file descriptor
> /home/yoh/proj/misc/argcomplete/argcomplete/
-> output_stream = os.fdopen(8, "wb")
> /home/yoh/proj/misc/argcomplete/argcomplete/
-> except:
(Pdb) fg
*** NameError: name 'fg' is not defined
Traceback (most recent call last):
  File "/home/yoh/proj/misc/argcomplete/argcomplete/", line 189, in __call__
    output_stream = os.fdopen(8, "wb")
  File "/usr/lib/python3.8/", line 1023, in fdopen
    return, *args, **kwargs)
OSError: [Errno 9] Bad file descriptor

in the shell I do get calls which do not explicitly create fd 8 failing:

$> python -c 'import os; os.fdopen(8)'     
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.8/", line 1023, in fdopen
    return, *args, **kwargs)
OSError: [Errno 9] Bad file descriptor
$> python -c 'import os; os.fdopen(8)' 8>&1

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:7 (1 by maintainers)

github_iconTop GitHub Comments

yarikopticcommented, Jul 24, 2020

THANK YOU @evanunderscore for the guidance – I am ready to declare “success” in troubleshooting !

What version of argcomplete are you running?

since it is argcomplete which generates it, may be it should embed argcomplete version which generated it somewhere in the comment

TL;DR summary: it is interaction between

  • argcomplete setting IFS=$'\013' which is later ‘exported’ into the underlying process:
_python_argcomplete() {
    local IFS=$'\013'
    local SUPPRESS_SPACE=0
    if compopt +o nospace 2> /dev/null; then
    COMPREPLY=( $(IFS="$IFS" \
                  COMP_LINE="$COMP_LINE" \
                  COMP_POINT="$COMP_POINT" \
                  COMP_TYPE="$COMP_TYPE" \
                  _ARGCOMPLETE=1 \
                  __python_argcomplete_run "$1") )
  • git-annex standalone (filed fresh bug report) to @joeyh build shim showing intolerance to that IFS causing it to segfault.
$> IFS=$'\013' /usr/lib/git-annex.linux/git version
[1]    1039928 segmentation fault (core dumped)  IFS=$'\013' /usr/lib/git-annex.linux/git version

That also explains why other @datalad/developers didn’t experience this – since they typically do not use standalone git-annex build.

I guess there is nothing wrong done on argcomplete side, so feel free to close this issue, unless you see that you could avoid setting IFS and set some other “argcomplete specific” variable so it might not cause disturbance in some weak tools underneath?

Some additional notes which are no longer pertinent probably

How did you generate it?

re-doing everything now after pip install --upgrade argcomplete to 1.12.0:

$> venvs/dev3/bin/register-python-argcomplete --shell bash `which datalad` >| /tmp/datalad-argcomplete
$> bash  # since I was in zsh
$> which datalad
$> source /tmp/datalad-argcomplete

and verified that it still doesn’t work by trying to complete datalad inst<TAB>

You can try manually removing 1>/dev/null 2>/dev/null from the line …

*$> sed -e 's, 1>/dev/null 2>&1,,g' /tmp/datalad-argcomplete > /tmp/datalad-argcomplete-noswallow
$> diff /tmp/datalad-argcomplete /tmp/datalad-argcomplete-noswallow
<         "$@" 8>&1 9>&2 1>/dev/null 2>&1
>         "$@" 8>&1 9>&2
evanunderscorecommented, Aug 2, 2020

Thanks for the detailed notes @yarikoptic. Glad you managed to find the source of your problem!

Embedding the argcomplete version in the generated wrapper is a good idea, though currently argcomplete doesn’t have a __version__ and keeping one in sync with the package version number is a workflow problem that would need to be left to @kislyuk.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Problems reading\writing to devices for embedded Linux project
I have two file descriptors (fd) to a COM port on an embedded Linux board, one for writing and one for reading. They...
Read more >
What permissions are needed to write a PID file in /var/run?
By default, you can only write to /var/run as a user with an effective user ID of 0 (ie as root). This is...
Read more >
lsof(8) - Linux manual page -
+d s causes lsof to search for all open instances of directory s and the files and directories it contains at its top...
Read more >
How to Fix the 'Too Many Open Files' Error in Linux?
It means that a process has opened too many files (file descriptors) and cannot open new ones. On Linux, the “max open file...
Read more >
Linux Creating a Partition Size Larger Than 2TB - nixCraft
Frankly speaking, you cannot create a Linux partition larger than 2 TB using the fdisk command. The fdisk won't create partitions larger ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found