question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Passing info about currently executing shiv to child subprocesses

See original GitHub issue

I have a particular python script that uses subprocess to create child processes that execute another script. What is the recommended way of passing down the info about the shiv’s site directory down to the child?

As an example, suppose I made a shiv like this:

$ shiv requests -o foo.shiv

I then make a parent.py that creates a child process:

import os
import subprocess
import sys

print("Parent sys.executable:", sys.executable)
print("Parent sys.path:", sys.path)
print("Parent sys.argv", sys.argv)
print("Parent PYTHONPATH", os.environ.get("PYTHONPATH"))

child = subprocess.run([
    sys.executable,
    'child.py',
],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)

print(child)

And then I make a child.py that makes use of requests, like so:

import requests

print(requests)

When I run it, I get output that looks like this:

$ ./foo.shiv parent.py
Parent sys.executable: /Users/kedo/.pyenv/versions/3.6.5/bin/python
Parent sys.path: ['./foo.shiv', '/Users/kedo/.pyenv/versions/3.6.5/lib/python36.zip', '/Users/kedo/.pyenv/versions/3.6.5/lib/python3.6', '/Users/kedo/.pyenv/versions/3.6.5/lib/python3.6/lib-dynload', '/Users/kedo/.local/lib/python3.6/site-packages', '/Users/kedo/.pyenv/versions/3.6.5/lib/python3.6/site-packages', '/Users/kedo/Workspace/shiv/src', '/Users/kedo/.pyenv/versions/3.6.5/lib/python3.6/site-packages/importlib_resources-0.8-py3.6.egg', '/Users/kedo/.pyenv/versions/3.6.5/lib/python3.6/site-packages/click-6.7-py3.6.egg', '/Users/kedo/.pyenv/versions/3.6.5/lib/python3.6/site-packages/wheel-0.31.1-py3.6.egg', '/Users/kedo/.shiv/foo_0550dd79-b895-48c0-91fa-11306e18f9cd/site-packages']
Parent sys.argv ['parent.py']
Parent PYTHONPATH None
CompletedProcess(args=['/Users/kedo/.pyenv/versions/3.6.5/bin/python', 'child.py'], returncode=1, stdout=b'', stderr=b'Traceback (most recent call last):\n  File "child.py", line 1, in <module>\n    import requests\nModuleNotFoundError: No module named \'requests\'\n')

Is there a non-hacky way for parent to know that it’s running via a shiv, and that it needs to pass down info about the shiv to the child? Could we set some sort of environment variable in https://github.com/linkedin/shiv/blob/master/src/shiv/bootstrap/__init__.py#L86 that parent.py could check and use?

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:10 (8 by maintainers)

github_iconTop GitHub Comments

1reaction
pfmoorecommented, Oct 30, 2018

Sounds good. I’ll pull something together tonight.

0reactions
lorencarvalhocommented, Oct 30, 2018

fixed in #76

thanks @pfmoore !

Read more comments on GitHub >

github_iconTop Results From Across the Web

Python Multiprocessing: Handling Child Errors in Parent
The problem I currently have is that, if a child process runs into an unhandled Exception, this is not recognized by the parent...
Read more >
Interacting with a long-running child process in Python
In this post I want to discuss a variation of this task that is less directly addressed - long-running child processes. Think about...
Read more >
Thread Example In Java - shiva tutorials
Thread is a collection of light-weight subprocess to execute the program. ... the state of currently executed thread to ready state and pass...
Read more >
How macOS Broke Python - We Fear Change
You also don't need to pass data from the parent to the child. You just keep running code in the child stanza of...
Read more >
20 ps Command Examples to Monitor Linux Processes
To get an overview of all the running processes on your Linux ... You can list a group process using the ps command...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found