08-Sep-2021| Best Practices, Debugging, Observability |10 min read

Debugging is the process of identifying, analyzing and removing errors in the software. It is a process that can start at any stage of the software development, even as early as the software has been written.

Sometimes, remote debugging is necessary. In the simplest terms, remote debugging is debugging an application running in a remote environment like production and staging. To perform remote debugging, you need to be able to remotely connect to a running application from your local development environment.

This article will focus on a debugging strategy known as forward analysis using remote debugging for Python. We’ll cover the different tools and libraries that will allow you to debug a live application remotely. We’ll go over the setup and configuration steps for debugging code using Visual Studio Code, one of the most accessible and powerful IDEs available.

Debugging Basics

There are a couple of strategies you can employ when debugging issues.

  • Backward analysis: Tracing the problem from an error log or message in order to identify the region of the code with the failure.
  • Forward analysis: Tracing the problem by using breakpoints at different parts of the programming execution.

From there, there’s a basic workflow to follow.

  1. *Identify the error: Error reports might be specific or vague depending on the part of the software, environment, and stage of development they present on. The first step to effective debugging is to make sure you identify the actual error.
  2. Find the error location: Once you have identified the error correctly, you need to narrow down the code’s location to where the bug is present.
  3. Analyze the error: Now that you have a better idea of where the error might be located, your next step is to analyze the conditions and the underlying cause to solve the issue.
  4. Fix and validate: Finally, you need to take steps to change the code and validate that your solution is correct. Typically, this is done by writing unit tests that validate the conditions of the original software defect.

Prerequisites

For this tutorial, you will need Python 3, venv, Docker Compose and the Flask web framework. You will also need Visual Studio Code, preferably with the official Python extension installed.

You’ll use a simple sudoku script as your debugging target. To download the demo application, follow this link. Alternatively, you can also use your own target Python 3 application, as the libraries you’ll use should be compatible.

Remote Debugging Architecture

Remote Debugging Diagram

At a high level, a remote debugging process requires:

  1. A connection between the server and your local development environment
  2. A debugging package installed on the remote application compatible with the IDE
  3. An IDE configured to allow for remote debugging
  4. Matching code between the IDE and the version deployed on the server.

Remote debugging has several benefits, but mainly it allows you to inspect and debug the application directly on the system. In many situations, this might be the only option available to developers to trace a software defect.

Start with debugging your sudoku solver locally to get familiar with the debugging tools and interface. Later, you’ll modify that initial configuration for remote debugging.

Setting Up Local Debugging with VS Code

The official Python extension provided by VS Code supports debugging several types of Python applications, from simple scripts like your sudoku solver to more complex applications like Flask and Django.

Create the Initial Configuration

With the sudoku solver script open, click the Run tab on the sidebar.

Run view

A dialog asks you to customize the Run and Debug configuration.

Run Configuration

Click create a launch.json file to open a dialog where you will select the debug configuration.

Debug Configuration

Out of the box, Visual Studio Code gives you a couple of presets to start, including Django and Flask. Go ahead and select the Python file configuration from the dropdown menu. VScode will immediately create and open a sample launch.json configuration file.

Additionally, the sidebar will change to show the following sections:

  • Variables: Variable values and expression evaluation from the selected stack frame in the Call Stack section.
  • Watch: List of expressions to watch, helpful in setting conditional breakpoints.
  • Call Stack: This shows the order in which methods, modules, and functions are getting called.
  • Breakpoints: Defined stopping or pausing places in the application execution.

Let’s take a closer look and define a couple of breakpoints in your file:

Breakpoints

Finally, go ahead and run the program through the debug configuration.

Debug options

The program will initialize and stop at your first breaking point on line 9 and correctly show the call stack that leads to the program’s execution. Additionally, you can inspect all the variables and their state at the current step in the program execution.

How to Set Up Remote Debugging

Now, with a better understanding of your IDE’s overall layout and debugging capabilities, you can move forward and set up for remote debugging. In order to properly illustrate the capabilities and value of remote debugging, you need a more complex application than a simple sudoku solver. Fortunately, Microsoft has a sample Flask project that you can leverage for your tutorial.

The Flask Sample Project has a simple web app and a matching Docker file that will allow you to simulate the remote debugging scenario. If you’d like to set up and build this application from scratch, feel free to visit the official tutorial.

Step 1: Application and Docker Setup

Start by checking out and setting up the sample project:

git clone git@github.com:microsoft/python-sample-vscode-flask-tutorial.git 
cd python-sample-vscode-flask-tutorial
python3 -m venv env
source ./env/bin/activate
python3 -m pip install --upgrade pip
pip install -r requirements.txt
code .

The above Bash snippet performs the following actions:

  • Clone the official VS Code Flask tutorial.
  • Create a Python Virtual Environment.
  • Activate the virtual environment.
  • Make sure pip is up to date.
  • Install required libraries specified in requirements.txt.
  • Open VS Code IDE. This will automatically load the Flask project with the active directory set to python-sample-vscode-flask-tutorial.

Note that on the first run, VS Code will ask you to select the relevant Python interpreter. Make sure to select the one from your virtual environment.

Virtual env

Next, make a few changes inside the python-sample-vscode-flask-tutorial project to simulate a remotely running application with Docker. Open the Dockerfile and uncomment the last three lines of the file.

# Pull a pre-built alpine Docker image with NGINX and Python 3 installed
FROM tiangolo/uwsgi-nginx:python3.8-alpine-2020-12-19

# Set the port on which the app runs; make both values the same.
#
# IMPORTANT: When deploying to the Azure App Service, go to the App Service on the Azure 
# portal, navigate to the Applications settings blade, and create a setting named
# WEBSITES_PORT with a value that matches the port here (the Azure default is 80).
# You can also create a setting through the App Service Extension in VS Code.
ENV LISTEN_PORT=5000
EXPOSE 5000

# Indicate where uwsgi.ini lives
ENV UWSGI_INI uwsgi.ini

# Tell NGINX where static files live. Typically, developers place static files for
# multiple apps in a shared folder, but for the purposes of this tutorial you can use the one
# app's folder. Note that when multiple apps share a folder, you should create subfolders
# with the same name as the app underneath "static", so there aren't any collisions
# when all those static files are collected together.
ENV STATIC_URL /hello_app/static

# Set the folder where uwsgi looks for the app
WORKDIR /hello_app

# Copy the app contents to the image
COPY . /hello_app

# If you have additional requirements beyond Flask (which is included in the
# base image), generate a requirements.txt file with pip freeze and uncomment
# the next three lines.
COPY requirements.txt /
RUN pip install --no-cache-dir -U pip
RUN pip install --no-cache-dir -r /requirements.txt

Next create a docker-compose.yml that will allow you to launch your Docker container.

./python-sample-vscode-flask-tutorial/docker-compose.yml

# docker-compose.yml
version: "3.4"

services:
  flask-server:
    image: flask
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - 5000:5000
    volumes:
      - .:/hello_app:cached

You can verify that you can run your Flask application by running the following command and visiting localhost:5000.

docker-compose up

If everything is working correctly, you should see the following in your browser:

Flask Application

Step 2: Debugger Setup

Now that you have the sample Flask application up and running, set up Visual Studio Code for remote debugging.

As before, create a launch configuration for attaching to your remote debugger. You can do this by adding or modifying your launch.json file as following:

./python-sample-vscode-flask-tutorial/.vscode/launch.json

{
  "configurations": [
        {
            "name": "Python: Attach",
            "type": "python",
            "request": "attach",
            "localRoot": "${workspaceFolder}",
            "remoteRoot": "/hello_app",
            "port": 9000,
            "secret": "my_secret",
            "host": "localhost"
        },
  ]
}

Make sure the localRoot and remoteRoot mappings are correct to your project setup and system. These mappings are used to link the files being executed on the remote server with your local copy.

Install debugpy

Next, install and set up debugpy, a Python module that allows you to create a debugger for your Python applications. To install debugpy, make sure to add debugpy to the

requirements.txt

. If you’re using PyCharm too you can install Lightrun for PyCharm debugging.

./python-sample-vscode-flask-tutorial/requirements.txt

Flask
debugpy

Instantiate debugpy Inside Your Application

Create the following file inside your hello_app application:

./python-sample-vscode-flask-tutorial/hello_app/debugger.py

from os import getenv

def initialize_debugger():
    if getenv("DEBUG") == "True":
        import multiprocessing

        if multiprocessing.current_process().pid > 1:
            import debugpy

            debugpy.listen(("0.0.0.0", 9000))
            print("Debugger is ready to be attached, press F5", flush=True)
            debugpy.wait_for_client()
            print("Visual Studio Code debugger is now attached", flush=True)

Instantiate the Debugger

./python-sample-vscode-flask-tutorial/hello_app/webapp.py

# Entry point for the application.
from . import app    # For application discovery by the 'flask' command. 
from . import views  # For import side effects of setting up routes. 
from . import debugger  # For import side effects of setting up debugger.

# Time-saver: output a URL to the VS Code terminal so you can easily Ctrl+click to open a browser
# print('http://127.0.0.1:5000/hello/VSCode')


./python-sample-vscode-flask-tutorial/hello_app/\_\_init\_\_.py

from flask import Flask  # Import the Flask class
from .debugger import initialize_debugger

initialize_debugger()

app = Flask(__name__)    # Create an instance of the class for your use

Expose the Debugging Port

Finally make sure your debugging port is exposed on your Docker container. Open the docker-compose.yml and add the following line:

# docker-compose.yml
version: "3.4"

services:
  flask-server:
    image: flask
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - 5000:5000
      - 9000:9000
    volumes:
      - .:/hello_app:cached

This will allow you to attach your remote debugging session to the container.

Launch the Container with a Debugger Enabled

Next, launch your Docker container in debug mode. The main thing you’re doing here is passing the DEBUG variable, which will then instantiate your debugpy listener.

docker-compose run --rm -e FLASK_APP=hello_app/webapp.py -e FLASK_ENV=development -e DEBUG=True --service-ports flask-server flask run --host 0.0.0.0

Finally, the last step is to attach to a debug session directly from Visual Studio Code. The following is a GIF of the debugger working against the remote application:

Remote Debugger

In summary, this is what’s happening:

  • In a terminal, run the Docker command with the DEBUG variable set to true.
  • When prompted, the debugger can now be attached; run the Python Attach launch configuration.
  • Add a breakpoint to the line you want to debug by clicking left of its number.
  • Call the corresponding route (localhost:5000/hello/flask in the GIF).
  • The debugger stops the code at your breakpoint.

Conclusion

As you’ve probably realized, a powerful remote debugging setup can take some effort to incorporate effectively. Trying to replicate the exact conditions that a customer experiences can add another layer of difficulty.

Fortunately, Lightrun can save you hassle, bringing you far less friction and, overall, a better customer experience with its real-time logs and metrics.

Allan MacGregor
Allan MacGregor is a software engineer and entrepreneur based in Toronto, Canada. He has fifteen years of professional experience building innovative solutions both inside and out of the e-commerce space.

All the Lightrun News That’s Fit to Print(f)!

Get the latest posts, news,
feature releases, updates, and articles directly to your inbox