question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Inconsistent normalization of Transaction/Span data

See original GitHub issue

Summary

  • Transaction.data is too easily truncated
  • Span.data is never normalized

image

Details

Introduction

The JS SDKs have a feature to normalize events before serialization that prevents serializing very deeply nested objects.

https://github.com/getsentry/sentry-javascript/blob/4e552ec0dc7d96a249d0f0ab7df7f4796623a521/packages/types/src/options.ts#L97-L106

This is a great feature, however it does not apply consistently to Transaction/Span data.

Problem

  • Transaction data is stored in Event.contexts.trace.data.
  • Span data is stored in Event.spans[].data.

Unlike for breadcrumbs that normalization applies only to the .data property of each breadcrumb, normalization is applied to Event.contexts as a whole:

https://github.com/getsentry/sentry-javascript/blob/4e552ec0dc7d96a249d0f0ab7df7f4796623a521/packages/core/src/baseclient.ts#L303-L323

What that means in practice is that, without changes to normalizeDepth, it is impossible to store objects in Transaction.data and have them sent to Sentry. Transaction.data is indirectly forced to be a flat object.

Users unaware of normalizeDepth can hack it and JSON.stringify the values they send, but that gives worse user experience 2 times:

  1. Seeing [Object] where the expect some data; fix the frustration with JSON.stringify and then
  2. Having a string instead of proper JSON means we cannot pretty print the value in the UI:

Manually stringified object image

Regular JSON object image

Possible solutions

Considering that all documented context types are flat objects, it is possible that we could relax the way how normalization applies to Event.contexts.

The new contexts.trace is the only key under contexts that may have nested objects under contexts.trace.data.

Option 1: liberal

An obvious option to consistently handle Transaction/Span data is to remove normalization from contexts altogether.

Option 2: restrictive

Alternatively, normalization should:

  • not apply to contexts as a whole; and
  • apply only to contexts.trace.data; and
  • apply to spans[].data.

Option 3: Improve Contexts normalization

  • not apply to contexts as a whole; and
  • Instead, walk through all the keys in contexts and apply it to that values of the key

edit: this is not an option to fix the inconsistency, it would still treat Transaction.data and Span.data differently.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:1
  • Comments:9 (9 by maintainers)

github_iconTop GitHub Comments

1reaction
HazATcommented, Jun 5, 2020

@dashed nope

1reaction
HazATcommented, Jun 5, 2020

Option 3️⃣ Reasoning: Option 1 changes the whole behavior of the SDK again -> We can drop whole normalization once we decided what to do with the payload issue.

Option 2 I don’t want to apply this now to span.data since it wasn’t there in the first place

Option 3 It fixes the current problem while leaving most of the stuff the same as it was before

Read more comments on GitHub >

github_iconTop Results From Across the Web

Reducing Data Inconsistencies with POI Normalization
Data normalization is an elegant technique that reduces data inconsistency. Especially when we are dealing with a huge dataset.
Read more >
Relational Database Normalization Process
Un-normalised data = repeating groups, inconsistent data, delete and insert anomalies. First Normal Form (1NF) = ELIMINATE REPEATING GROUPS (make a separate ...
Read more >
Normalization Vs Denormalization in Database Modeling
Normalization is the technique of dividing the data into multiple tables to reduce data redundancy and inconsistency and to achieve data ...
Read more >
Database normalization description - Office - Microsoft Learn
Inconsistent dependencies can make data difficult to access because the path to find the data may be missing or broken.
Read more >
What is Normalization of Data in Database? - Simplilearn
Data normalization is the process of reorganizing data within a database ... adaptability by removing duplication and inconsistent reliance.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found