Subprocess not been killed when main python script got SIGINT
  • 31-May-2023
Lightrun Team
Author Lightrun Team
Subprocess not been killed when main python script got SIGINT

Subprocess not been killed when main python script got SIGINT

Lightrun Team
Lightrun Team

Explanation of the problem

The following script,, is intended to continuously print the message “hello” to a file named “tmp”:

while true
    echo "hello" >> tmp

However, when this script is run and interrupted with a SIGINT signal (e.g., by pressing Ctrl+C), it fails to terminate gracefully. The objective is to modify the script to ensure that it can be properly terminated after receiving a SIGINT signal.


Troubleshooting with the Lightrun Developer Observability Platform

Getting a sense of what’s actually happening inside a live application is a frustrating experience, one that relies mostly on querying and observing whatever logs were written during development.
Lightrun is a Developer Observability Platform, allowing developers to add telemetry to live applications in real-time, on-demand, and right from the IDE.

  • Instantly add logs to, set metrics in, and take snapshots of live applications
  • Insights delivered straight to your IDE or CLI
  • Works where you do: dev, QA, staging, CI/CD, and production

Start for free today

Problem solution for Subprocess not been killed when main python script got SIGINT

In the first answer, it is explained that the reason why the second solution works is that the bash command is executed in the foreground process group, within the same process group and session as the Python process. This allows the SIGINT signal sent from the terminal to be received by both the script and the Python script. On the other hand, the first solution fails because when a process is launched using sh, it creates its own process session with _new_session=True, making it disconnected from the terminal. In this case, the responsibility lies with the user to send signals directly to the subprocess in order to terminate it.

The second answer builds upon the first one, suggesting a possible solution to terminate a process launched with sh in the background. The recommended approach involves running the process in the background using the _bg=True argument and handling the KeyboardInterrupt exception in the Python script. The code snippet provided demonstrates how to achieve this by using process.wait() to wait for the subprocess to finish, and in the event of a KeyboardInterrupt, sending a kill signal to the subprocess using process.kill().


Other popular problems with sh

Problem: Inability to Capture Subprocess Output

One common issue encountered when using the sh module is the inability to capture the output of subprocesses. By default, when executing a command using sh, the output is directly printed to the console, making it difficult to capture and manipulate programmatically. This limitation can hinder tasks that require processing or analyzing the output of subprocesses.


To overcome this problem, the sh module provides a solution by allowing users to redirect the output of subprocesses to a variable or file. By utilizing the ret parameter when executing a command, users can capture the output and access it as a string.

Problem: Inconsistent Behavior Across Platforms

Another challenge faced when using the sh module is inconsistent behavior across different operating systems or platforms. Due to variations in the underlying shell implementations, certain commands or options may behave differently or produce unexpected results when executed using sh. This can lead to issues when developing cross-platform applications or scripts.


To mitigate this problem, it is recommended to thoroughly test the scripts or commands across the target platforms. By conducting platform-specific testing, developers can identify any inconsistencies or differences in behavior and adapt the code accordingly. Additionally, it may be necessary to employ conditional statements or platform-specific code to handle variations between different operating systems.

Problem: Lack of Advanced Shell Features

The sh module, although convenient and user-friendly, may not offer the full range of advanced shell features and functionalities available in traditional shell environments. Some complex shell constructs, such as piping, input/output redirection, or complex conditionals, may not be directly supported or may require additional workarounds when using sh. This can limit the flexibility and capabilities of shell scripting when using the sh module.


When advanced shell features are required, it may be more appropriate to use the subprocess module from the Python standard library. Unlike sh, subprocess allows for more fine-grained control and access to the underlying shell features. By utilizing the subprocess module, developers can leverage the full power of shell scripting, including complex constructs and advanced functionalities. However, it’s important to note that working with subprocess requires a deeper understanding of process management and may involve more complex code compared to the simplified interface provided by the sh module.

A brief introduction to sh

The sh module is a Python library that provides a high-level interface for executing shell commands and managing subprocesses. It aims to simplify the process of interacting with the shell environment by offering a more intuitive and Pythonic approach. With sh, users can execute shell commands and capture their output, handle command arguments, and perform various operations such as input/output redirection, command composition, and command substitution.

The module facilitates seamless integration of shell commands into Python scripts, enabling developers to leverage the power of the shell environment while utilizing the flexibility and ease-of-use of Python. sh abstracts away the complexities of working with subprocesses and command execution, providing a straightforward and consistent interface for interacting with shell commands. By encapsulating common shell operations within Python functions, sh eliminates the need to manually handle subprocess creation, communication, and termination, saving time and effort in the development process.

Most popular use cases for sh

  • Executing Shell Commands: The primary use of the sh module is to execute shell commands directly from Python. It provides a simplified interface for executing commands and capturing their output. Users can utilize the sh.command_name syntax to execute various shell commands, passing arguments as needed. For example, the following code block demonstrates executing the ls command to list files in a directory:
import sh

output ="-l", "/path/to/directory")
  • Interacting with Subprocesses: sh allows for seamless interaction with subprocesses, providing functionality to manage the execution, communication, and termination of subprocesses within a Python script. It abstracts away the complexities of subprocess management, enabling users to focus on the task at hand. Users can launch subprocesses, pass input, retrieve output, and handle errors through the sh module.
  • Scripting Shell Operations: The sh module can be used to script complex shell operations and workflows using Python. It allows for the composition of multiple shell commands, input/output redirection, piping, and other advanced shell features. Users can combine multiple sh commands together to create powerful shell scripts within a Python environment.

It’s Really not that Complicated.

You can actually understand what’s going on inside your live applications. It’s a registration form away.

Get Lightrun

Lets Talk!

Looking for more information about Lightrun and debugging?
We’d love to hear from you!
Drop us a line and we’ll get back to you shortly.

By submitting this form, I agree to Lightrun’s Privacy Policy and Terms of Use.