Python has become one of the most popular programming languages globally and is particularly popular in data science and artificial intelligence. Python’s popularity can be attributed to its ease of use and readability and the large ecosystem of libraries and frameworks built around it. Python is also popular among developers working on cloud-based applications, as they can use it to orchestrate complex workflows.
Netflix, for example, is one of the largest online streaming services in the world and relies heavily on Python for its content management system and recommendation engine. Python is also used extensively throughout Netflix’s back-end infrastructure, from system administration tools to deployment scripts. Major companies and institutions such as IBM, Pixar, Facebook, Spotify, and NASA also use Python.
What is Python?
Python is a programming language permeating every industry due to its versatility and large developer community. It is a general-purpose programming language employed in software development, scripting, and automation. It is also popular for scientific computing and data analysis, as it has many libraries and tools for these tasks.
Python is easy to learn for beginners because of its high readability and simple syntax construction. According to Statista, Python claimed the top spot for the most popular programming language in early March 2020. The open-source language also has an estimated market value of over $10 billion. In addition, TechRepublic reports Python as one of the most in-demand and employable skill sets from 2021 onwards.
While knowing how to code in Python is a great skill to have, knowing how to debug Python to figure out what’s wrong with your code is equally important.
Debugging is a staple requirement of every programming job. To debug means to walk through the program step-by-step to figure out the origins of an error. When we need to debug Python, it’s usually because something is not working as expected.
In this tutorial, we will go over how to debug Python using VSCode’s built-in debugging tools to help us figure out and pinpoint where our errors are originating.
Prerequisites
For this tutorial, here are the prerequisites and assumptions:
You have a working knowledge and understanding of Python
You have a Python interpreter installed
Opening a Python project in VSCode
If you don’t have a Python project to debug, here is one we prepared for you. It is a simple piece of Python with an error built into the do_something_cool method.
class User:
def __init__(self, name, email):
self._name = name
self._email = email
def get_name(self):
return self._name
def get_email(self):
return self._email
def do_something_cool(self):
print ("Sign up to Lightrun " + self._namee)
def __str__(self):
return self._name + " , " + self._email
users = [ User("Lightrun Demo", "help@lightrun.com"), User("Debugger", "debugger@debugger.com")]
for user in users:
user.do_something_cool()
Let’s get into the process of debugging this code.
Initial configuration for Python debugging
Start by clicking on the debug icon on the left-hand panel in your VSCode. Click Create a launch.json file, which will give you a drop-down of all your currently installed debuggers.
By default, a Python debugger is not installed in VSCode. You can identify this in the missing Python option in the dropdown pictured above.
To download and install a Python debugger, you can do one of the following:
Go back to the left-hand menu and click the Extensions button, then type Python in the search box, and a list of results should show up.
Click Install an extension for Python.
Regardless of what you choose, it will bring up similar, if not the same results. Select the first one, Microsoft’s official Python extension for Visual Studio Code. Click the Install button to add it to your VSCode instance. (Sidenote: if you have time, check out other extensions for Python in VS Code.)
Under normal circumstances, you shouldn’t have to restart VSCode after installing the Python debug extension.
Go back to the Debug menu. When you click Create a launch json file again, a Python option should become available. Alternatively, VSCode may simply make the selection for you and automatically generate a JSONfile.
Debugging current Python file
Open up the Python file you want to debug. In our case, it is the file with the code example above. A faded red dot will appear when you hover over on the left-hand side of the line number. If you click on this dot, it will set a breakpoint.
A breakpoint is a point in the code where the program’s execution will stop temporarily so that the program’s state can be examined. Breakpoints are helpful for debugging purposes because they allow the programmer to check the values of variables and see how the program is executing at that point.
When you click Run and Debug for the first time, you will get the option to select a debug configuration. For this tutorial, select Python File.
When you click the Play button located in the top left above the VARIABLES panel, the Python debugger will run through the code and stop at the breakpoint.
A debugging toolbar will also become available at this point. The debugging toolbar contains buttons that let you go forward, backward, and jump into different parts of the code based on where you currently are in your breakpoint. Metadata about your code will also appear in the VARIABLES panel, where locals and globals can be watched and monitored for changes.
Here is a rundown on what each icon in the toolbar does:
Continue (F5) will run past the breakpoint and continue with the rest of the program until it hits the next breakpoint.
Step over (F10) will take the debugger down to the following line.
Step into (F11) will take the debugger into the following function.
Step out (F12) will take the debugger out of the function and into the next step.
Restart (Ctrl+Shift+F5) will restart the entire debugging process.
Stop (shift+F5) will stop the debugging process and exit it.
Let’s run our debugger through, and we see that the variable for name doesn’t exist.
What if you have a problem that you can’t reproduce in your local environment? In a real-world development setting, especially with large applications, bugs often only show up under load in production environments. How do you debug that?
Fortunately, there are now tools available to make debugging in production possible and painless.
Take a look at Lightrun: our next-gen remote debugger for your production environment. With Lightrun, you can inject logs without changing code or redeploying, or add snapshots: breakpoints that don’t stop your production application. Lightrun supports Python, Java, and Node.js applications, and you can start using it for free.
Black Friday, the traditional kickoff to the holiday shopping season, is set to make waves in 2024 with projected sales reaching an impressive $10.8 billion—a 9.9% increase from last year according to Statistics.blackfriday analysts. According to the same team, Cyber Monday sales in 2024 are expected to reach $13.2 billion—a 6.1% increase from 2023.
Both events in sum are expected to generate $24 billion in sales. A major contributor to this growth is the increasing number of consumers who prefer to shop online. As consumer behaviors evolve to favor online shopping, retailers are expected to invest more in their online infrastructure, user experience, and platform reliability to meet this projected demand.
More traffic means more potential revenue, but it also means more queries, more transactions, and more potential problems. As an online retailer, you need to be well-prepared from a technical standpoint to ensure that you don’t miss out on any potential sales.
Amazon Prime Day 2018: A Prime Example For How Things Can Go Wrong
Q4 is a critical time for businesses, and developers are often under immense pressure to ensure that their systems are up and running. The last thing you want is for your website to crash during the busiest shopping season of the year. Poor performance means lost revenue, and problems can quickly escalate into a PR nightmare.
You’ve likely heard about the infamous Amazon Prime Day crash in 2018, which reportedly cost the company an estimated $100 million in lost sales. Despite the charm of the “Dogs of Amazon” error pages, they couldn’t mitigate the damage. Cute error messages are no substitute for a robust, well-prepared platform.
Some unofficial post-mortems of the Prime Day crash suggest that the root cause was a failure in the auto-scaling system, which left Amazon scrambling to manually add server capacity. The internal system “Sable,” Amazon’s e-commerce storage and computation platform, struggled under the surge—which led to a failure across the platform. Some experts speculate that misconfigurations or software bugs in scaling systems magnified the issue, forcing Amazon to temporarily shut down international traffic to the site according to CNBC.
In all cases, the lesson is clear: impredictable issues can arise, and you need to be prepared.
If you google “How to prepare for Black Friday for developers,” you’ll find a plethora of articles that will tell you to “test your system,” “monitor it,” “add autoscalability to your infrastructure,” and “secure your code.” While these may be good pieces of advice, they touch only the surface of the problem as they are not shifting left enough to prevent the issues from happening in the first place.
In sports, for example, achieving peak performance isn’t the result of a single training session but rather a dedicated regimen of practice, evaluation, and adjustment over time. Similarly, preparing for Black Friday is not a one-time event, but rather a continuous process that requires iterative improvements based on feedback. The earlier this feedback is received, the better.
Shift-Left Everything: The Early Bird Catches the Worm
Performance should be a first-class citizen in your development process. That’s why it should not be an afterthought, but rather a core part of your development cycle. A software development lifecycle includes several stages, such as planning, coding, testing, and deployment. You should be thinking about performance from the early stages of development. The problem is that performance testing is often associated with the staging and production environments—which is too late in the development cycle.
By shifting performance testing to the left, you can identify and fix performance issues early in the development process. This approach allows you to catch performance bottlenecks before they become critical issues.
Performance problems can be triggered by a variety of factors, such as bad capacity planning, downsized infrastructure, inefficient code, slow database queries, extreme edge cases, or code that doesn’t scale well. All these issues can be identified early in the SDLC if you have the right tools and processes in place.
For example, with Lightrun, you can dynamically add a log to a specific function suspected of causing slow database queries, without halting or redeploying the application. This enables you to pinpoint the exact query, analyze its execution time, and resolve the issue during the development phase, long before it reaches staging or production.
Using Lightrun’s snapshot feature, you can also capture the state of your application at a specific point in time. Let’s say you have an issue that only occurs under certain conditions (e.g., a specific event or input). You can capture the state of your application after reproducing the issue using a snapshot that can be added directly from the Lightrun IDE plugin. This snapshot not only captures the trace of the application but also the state of the variables at that point in time. This allows you to deeply analyze the issue from the inside and fix it accordingly. You can also export the snapshot and share it with your team members for further analysis. All of this happens from the comfort of your IDE while you’re coding. The best part is that you capture logs and traces from any environment, including your development, testing, staging, and production environments without any code changes on the remote server.
Another crucial, often overlooked practice you should consider is observability.
Observability is the ability to understand what’s happening inside your system based on external outputs. An e-commerce platform is responsible for different types of operations, including user authentication, product search, transaction processing, and more. Observability allows you to monitor these operations and understand how they are performing from the user’s perspective. Using logs, metrics, and traces, developers can gain useful and actionable understanding of their applications’ and infrastructure’s behavior.
Nowadays, observability is a key practice, but its implementation is often not quite right in many organizations. First, not understanding what to “observe” can lead to a lot of noise and irrelevant data – a sheer volume of data makes it challenging to identify the root cause of a problem. Second, observability is often associated with the production environment, which is a late phase in the SDLC. The first problem creates a siloed approach to observability where developers are less likely to use the data to improve their code. The second problem leaves developers out of the loop when it comes to understanding how their code behaves in production.
Instead of this approach, observability should be a part of the development process from the beginning. But here’s the catch: if developers have to wait for a new deployment on the testing or staging environment to add a log or a metric, a new bottleneck is created. A solution to this, besides having a continuous deployment pipeline, is to use a tool that allows them to instrument their code on the fly without the need to redeploy their applications. This is also known as “dynamic instrumentation” and it’s a key feature of Lightrun.
According to a 2018 Stripe report, developers spend an average of 17.3 hours per week on maintenance and debugging issues. This amounts to $300 billion in lost developer productivity every year. By shifting left observability, performance testing, and debugging, you can reduce this number and save valuable time and resources.
Debugging in Production: No Longer a Meme
Shifting left observability, performance testing, and debugging is the secret sauce to a successful Black Friday. Small incremental changes with continuous feedback and early feedback are the ingredients to a successful recipe. However, incidents can still happen, and when they do, you need to be able to debug in production as close to the problem as possible.
Some years ago when someone mentioned “debugging in production“, it was often seen as a joke or the worst-case scenario. Today, it’s a regular practice for many organizations. The reason is simple: the production environment is the most complex and the most realistic environment. If a bug or a performance issue escapes your team despite all the early stages of testing, the early feedback, and the continuous improvement, you need to be able to debug it in production. Chances are that the issue is an edge case that only occurs in production.
Traditionally, debugging in production was a scary and unsafe process. The fastest way to do it was SSHing into the server and adding a print statement in the code. This is not only unsafe but also inefficient. It’s unsafe because you’re modifying the code in the production environment, which can lead to unexpected results. It’s inefficient because you need to redeploy the application to see the changes, which can take time and may disrupt the user experience.
Here’s when Lightrun once again comes to the rescue. A developer can add a line of log to the code in the live production environment without halting the application or redeploying it. Imagine the following scenario: you have a performance issue that only occurs in production. You can add a log to the suspected function or line of code, capture the logs, and analyze them in real-time to identify the root cause of the problem. Similarly, you can add a snapshot to capture the state of the application at a specific point in time and under specific conditions.
All you have to do is to install the Lightrun plugin in your IDE – available for IntelliJ, PyCharm, WebStorm, VS Code and select web IDEs – connect it to your production environment, and start debugging. It’s that simple.
Reduce the Mean Time to Resolution (MTTR)
It may be too late to prevent all performance issues from happening, but you can still reduce the time it takes to resolve them. The Mean Time to Resolution (MTTR) is a key KPI that measures the average time it takes to resolve an issue. The longer it takes to resolve an issue, the more revenue you lose.
To understand how to calculate the MTTR, let’s consider the following formula:
MTTR = (Total time to resolve issues) / (Total number of repairs
If you have a high MTTR, it means that you’re spending too much time resolving issues. Serious engineering teams aim to reduce the MTTR as much as possible. In general, there’s no good or bad value for the MTTR; this depends on the context of the organization, the criticality of the application, and the complexity of the issues. But as a rule of thumb, if your team is spending more than 3 hours per issue, you may want to consider improving your processes. This is especially true during Black Friday when every minute counts. According to ECDB, as an example, online shopping peak hours in some countries (e.g., Germany, UK) are between 8 pm and 10 pm. If your MTTR is 3 hours and your application crashes at 8 pm, your team will only start to resolve the issue at 11 pm. This means that you were down during all the peak hours.
Lightrun can help you reduce your MTTR. For example, Lightrun capabilities have enabled InsideTracker developers to diagnose and resolve application issues in real-time, saving valuable debugging time. Their MTTR improved by 50%, resulting in dozens of hours saved each month. Prior to using Lightrun, InsideTracker developers were unable to troubleshoot or access remote Kubernetes environments from their local machines in an easy, straightforward way. This led to long and inefficient debugging sessions that required hotfixes and redeployments taking hours.
What’s Next?
Lightrun provides a suite of powerful features that help developers quickly identify and resolve issues in production environments but before reaching production, Lightrun can also help you shift-left observability, testing, and debugging.
The tool offers a range of features that can help you improve your development process, including:
Live debugging: Developers can add logging statements and metrics to their code in real-time, without having to redeploy or restart their applications.
On-demand snapshots: Lightrun Snapshots provide virtual breakpoints but without stopping the execution of your application. You can add conditions, evaluate expressions and inspect any code.
Custom metrics: Right from the IDE, developers can create custom metrics to track specific aspects of the system’s behavior, such as response time or error rates. All tasks are completed without the need to redeploy any code.
Integration with existing developers and enterprise tools: Lightrun integrates with existing cloud-native tools, such as Prometheus, Grafana, VSCode, public cloud providers, and more, making it easy to integrate into existing workflows.
Lightrun empowers not only developers but also DevOps, SREs, Platform Engineers, and other stakeholders to improve their development process and as a result, catch issues early, reduce the MTTR, and improve the overall performance of their applications.
Ready for Black Friday 2024? If not, Lightrun is here to help:
Get a Lightrun demo and learn how to easily debug microservices, Kubernetes, Docker Swarm, ECS, Big Data, serverless, and more: https://lightrun.com/get-a-lightrun-demo/
