Dynamic Observability Tools for API Live Debugging

Intro

Application Programming Interfaces (APIs) are a crucial building block in modern software development, allowing applications to communicate with each other and share data consistently. APIs are used to exchange data inside and between organizations, and the widespread adoption of microservices and asynchronous patterns boosted API adoption inside the application itself.

The central role of APIs is also evident with the emergence of the API-first approach, where the application’s design and implementation start with the API, thus treating APIs as first-class citizens and developing reusable and consistent APIs.

In the last decade, Representational state transfer (REST) APIs have come to dominate the scene, becoming the predominant API technology on the web. REST is more of an architectural approach than a strict specification: This free-formedness is probably the key to REST success as it has been essential in making REST popular and one of the critical enablers of a loose coupling between API providers and consumers. However, sometimes this bites back as a lack of consistency in the API behavior and interface. This is sometimes alleviated using specification frameworks like OpenAPI or JSON Schema.

Also, it’s worth pointing out the role of developers in designing and consuming APIs, as frequently, the development of an API requires strict collaboration between backend developers, frontend developers, and mobile developers since the role of API is the integration of different applications and systems.

Challenges in API integration

Despite being central to modern application development, API integration remains challenging. Those challenges mainly originate from the fact that the systems connected by APIs form a distributed system, with the usual complexities involved in distributed computing. Also, the connected systems are mostly heterogeneous (different tech stacks, data models, ownership, hosting, etc.), leading to integration challenges. Here are the most common ones:

• Incorrect data. Improper data formatting or conversion errors (due to inaccurate data type or incompatible data structures) can cause issues with the exchanged data. This often results in malformed JSON, errors in deserialization, and type casting errors.
• Lack of proper documentation. Poorly documented endpoints may require extensive debugging to infer data format or API behavior. This is particularly problematic when dealing with third-party services without access to the source code or the architecture.
• Incorrect or unexpected logic or behavior. The loosely defined REST model does not allow for specifying the callee behavior formally, or such behavior can be undocumented or implemented wrong for some edge cases.
• Poor query parameter handling. Query parameters are the way for the callee to modify the provided results. Often, edge cases arise where parameters are not handled correctly, requiring a trial-and-error debugging process.
• Error handling. Even if HTTP provides the basic mechanism of response codes for error handling, each API implementation tends to customize it, either using custom codes or adding JSON error messages. Error handling is not always coherent, even between different endpoints on the same system, and it may be undocumented.
• Authentication and authorization errors. The way in which authorization is handled on the API producer can generate errors and unexpected behavior, sometimes manifesting incoherence between different endpoints on the same system.

Errors can be present on the provider side or the consumer side. On the provider side, we often cannot intervene in the implementation, which necessitates implementing workarounds on the consumer side.

For errors on the consumer (wrong deserialization, incorrect handling of pagination, or states, etc.), troubleshooting usually involves examining logs for request/response patterns and adding logs to examine parameters and payloads.

Lightrun Dynamic Observability for API debugging

Ligthrun‘s Developer Observability Platform implements a new approach to observability by overcoming the difficulties of troubleshooting applications in a live setting. It enables developers to dynamically instrument logs for applications that run remotely on a production server by adding logs, metrics, and virtual breakpoints, without the need to code changes, redeployment, or application restarts.

In the context of API debugging, the possibility of debugging on the production environment provides significant advantages, as developers do not need to reproduce locally the entire API ecosystem surrounding the application, which can result difficult: think, for example, to the need to authenticate to third-parties API, or to provide a realistic database to operate the application locally. Also, it is only sometimes possible to reproduce realistic API calls locally, as the local development environment tends to be simplified with respect to the production one.

Lightrun allows debugging API-providing and consuming applications directly on the live environment, in real-time and on-demand, regardless of the application execution environment. In particular, Lightrun makes it possible to:

• Add dynamic logs. Adding new logs without stopping the application allows obtaining the relevant information for the API exchange (request/response/state) without leaving the IDE and without losing the state (for example, authentication tokens, complex API interactions, pagination, and real query parameters). It’s also possible to log conditionally only when a specific code-level condition is true, for example, to debug a particular API edge case taken out from a high number of API requests.
• Take snapshots. Adding virtual breakpoints that can be triggered on a specific code condition to show the change in time of request parameters and response payloads.
• Add Lightrun metrics for method duration and other insights. It makes it possible to measure the execution times of APIs and count the time a specific endpoint is being called.

Lightrun is integrated with developer IDEs, making it ideal for developers, as it allows them to stay focused on their local environment. Doing so, Lightrun works as a debugger that works everywhere the application is deployed, allowing for a faster feedback loop during the API development and debugging phases.

Bottom Line

Troubleshooting APIs returning incorrect data or behaving erratically is essential to ensure reliable communication between systems and applications. By understanding the common causes of this issue and using the right tools and techniques, developers can quickly identify and fix API problems, delivering a better user experience and ensuring smooth software operations. Lightrun is a developer observability platform giving backend and frontend developers the ability to add telemetry to live API applications, thus representing an excellent resolution to API integration challenges. Try it now on the playground, or book a demo!