Workaround for "fields have different lengths"

Hi, I understand what is meant by Fields have different lengths, but is there any workaround to avoid this without losing any array elements by coalescing missing values to some default? It’s a critical requirement for us.

For example, form the following input:

{
  "services": [
    {
      "name": "order-api",
      "version": "1"
    },
    {
      "name": "billing-api"
    }
  ]
}

I want to get ["order-api", "billing-api"] for name field, ["1", ""] for version field.

[agrimpelhuber]

I think (hope, guess) I have found a workaround. For me at least it works around the issue of missing fields. Maybe it helps others, too. Let me know how universal that is. The trick for me was to use JSONata instead of JSONPath for the affected fields. Took a little bit of time to get used to it, but I have a few issues sorted out now. Two features in JSONata seem important to me:

• JSONata always returns an array for values, or so I’ve read. Which is tedious when picking the only / single value with [0], but at least there’s a value to work with all the time!
• JSONata supports in-line functions, conditions etc. Yeah!

Here’s a very basic JSONata expression that does it for me with the field “h” (for “hours”)

$.rows.($count(h) > 0 ? ( $string(h) = "null" ? "Null value" : h ) : "Empty Value")

Explanation:

• 2 nested conditions (Condition ? True : False syntax) setting default / replacement values for 2 different “empty field” situations, depending on what the JSON API returns
• The outer one testing for an empty array (remember, values are always arrays)
• The inner one testing for the fact that the array could still be a null value. Not sure whether it should technically be $string(h[0]), but $string(h) works fine for me
• Not sure which of the possible JSON situations (missing / empty /null fields) triggers which condition - you tell me 😃

Hope this helps someone

[marcusolsson]

The return value from JSONPath is already a full array and you completely lose the record alignments in the DataFrame.

Yes, that’s better explanation than mine 😃

It also looks like even if you use the callback to JSONPath it doesn’t appear to actually call it when the field is completely missing.

Thank you for taking the time to look into this 🙏 One of my plans for the next major release is to rewrite it into a backend plugin (in Go) to support alerting. The tricky thing will be to find a JSONPath library for Go. Unfortunately, switching to any other library will likely make it hard to maintain feature parity.

I’ll be somewhat unavailable this week, but I’m planning to spend the next two weeks of my vacation on plugin work, I might take the opportunity to start working on this then.