question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[FEATURE] When a nested field has mismatched type print the full path to that nested field

See original GitHub issue

Summary

In a complex structure like the following:

{
  "source_machine": {
    "port": 80
  },
  "dest_machine": {
    "port": "http-port"
  }
}

If there was an error with another log where dest_machine.port was an integer this would error and simply state something like: Ignoring field with mismatched type: old=(hard,port,NULLABLE,STRING); new=(hard,port,NULLABLE,INTEGER)

At this point you are left to figure out which structure this port column actually exists in. This is a more simple example but as the schema grows and is more complex, this problem is harder to manually resolve.

Ideally, we can track the path to this using a JSON path or dpath expression. Something like dest_machine.port. This will likely take adding an additional argument to the recursive function merge_schema_entry. Something like a base_path=None and continually build up that base_path string in each recursive iteration so that it can be used in the errors like "{}.{}".format(base_path, new_name) and "{}.{}".format(base_path, old_name)

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
bxparkscommented, Oct 26, 2020

Oh, I understand your problem, you have 2 port fields, so the line number does not help.

0reactions
bxparkscommented, Oct 28, 2020

Pushed v1.2 to PyPI.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Nested field type | Elasticsearch Guide [8.5] | Elastic
The nested type is a specialised version of the object data type that allows arrays of objects to be indexed in a way...
Read more >
Typescript: deep keyof of a nested object - Stack Overflow
My goal is to get all the keys of the given type. The following code work well in this case. But when I...
Read more >
Nesting Schemas — marshmallow 3.19.0 documentation
Use a Nested field to represent the relationship, passing in a nested schema. from marshmallow import Schema, fields class UserSchema(Schema): name = fields...
Read more >
Specify nested and repeated columns in table schemas
This page describes how to define a table schema with nested and repeated columns in BigQuery. For an overview of table schemas, see...
Read more >
Object Fields VS. Nested Field Types in Elasticsearch - Opster
Nested is a special type of object that is indexed as a separate document, and a reference to each of these inner documents...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found