Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Shiftr : Transforming n level depth array to one level depth

See original GitHub issue

I am new to jolt and don’t know is this question is already answered by someone earlier. I have to transform 2 level or may be more nested level arrays to one level array of objects. Please find input, output and jolt specs below. Input :

{
    "customers": [
        {
            "timestamp": "2016-07-29T00:20:11+0000",
            "attributes": [
                {
                    "id": 3,
                    "name": "Width",
                    "values": [
                        {
                            "id": 12,
                            "name": "REGULAR"
                        },
                        {
                            "id": 13,
                            "name": "WIDE"
                        }
                    ]
                }
            ],
            "dates": [
                {
                    "id": 1234,
                    "name": "Date of Birth",
                    "value": "2000-03-13T15:00:00.000+0000"
                },
                {
                    "id": 321,
                    "name": "Start Date",
                    "value": "2014-03-13T15:00:00.000+0000"
                }
            ]
        },
        {
            "timestamp": "2015-07-29T00:20:11+0000",
            "attributes": [
                {
                    "id": 102,
                    "name": "Men",
                    "values": [
                        {
                            "id": 2413,
                            "name": "Shoes"
                        },
                        {
                            "id": 2412,
                            "name": "Tshirt"
                        }
                    ]
                },
                {
                    "id": 132,
                    "name": "Hidden Collections - US",
                    "values": [
                        {
                            "id": 2078,
                            "name": "Mobile"
                        },
                        {
                            "id": 319314,
                            "name": "Corporate"
                        }
                    ]
                }
            ],
            "dates": [
                {
                    "id": 123,
                    "name": "End Date",
                    "value": "2014-03-13T15:00:00.000+0000"
                }
            ]
        }
    ]
}

Output:

{
  "customers" : [ {
    "attribute" : {
      "3" : [ "REGULAR","WIDE" ]
    },
    "timestamp" : "2016-07-29T00:20:11+0000",
    "dateOfBirth": "2000-03-13T15:00:00.000+0000",
    "startDate" : "2014-03-13T15:00:00.000+0000"
  }, {
    "attribute" : {
      "102" : [ "Shoes","Tshirt" ],
      "132" : [ "Mobile","Corporate" ]
    },
    "timestamp" : "2015-07-29T00:20:11+0000",
    "endDate" : "2014-03-13T15:00:00.000+0000"
  } ]
}

JOLT Specs:

[
  {
    "operation": "shift",
    "spec": {
      "customers": {
        "*": {
          "timestamp": "customers[&1].timestamp",
          "attributes": {
            "*": {
              "values": {
                "*": {
                  "name": "customers[&1].attribute.@(3,id)[]"
                }
              }
            }
          },
          "dates": {
            "*": {
              "id": {
                "1234": {
                  "@(2,value)": "customers[&1].dateOfBirth"
                },
                "321": {
                  "@(2,value)": "customers[&1].startDate"
                },
                "123": {
                  "@(2,value)": "customers[&1].endDate"
                }
              }
            }
          }
        }
      }
    }
  }
]

Please let me know where I am going wrong, any help will be appreciated. Thanks.

Issue Analytics

State:
Created 7 years ago
Comments:6

Top GitHub Comments

1reaction

wisthycommented, Oct 2, 2019

@rajimanu please create a new issue instead of commenting on an old, closed one. If your issue is indeed related to this one, just add a link to it (adding # + issue number)

1reaction

milosimpsoncommented, Aug 14, 2016

Spec

[
  {
    "operation": "shift",
    "spec": {
      "customers": {
        "*": {
          "timestamp": "customers[&1].timestamp",
          "attributes": {
            "*": {
              "values": {
                "*": {
                  "name": "customers[&5].attribute.@(3,id)[]"
                }
              }
            }
          },
          "dates": {
            "*": {
              "id": {
                "1234": {
                  "@(2,value)": "customers[&5].dateOfBirth"
                },
                "321": {
                  "@(2,value)": "customers[&5].startDate"
                },
                "123": {
                  "@(2,value)": "customers[&5].endDate"
                }
              }
            }
          }
        }
      }
    }
  }
]

You had it mostly correct, the key fix customers[&1] --> customers[&5]. The “&” logic means up up tree, grab a value and sub it in. Ex “&5” means look up the tree 6 levels, (starting from 0) 0,1,2,3,4,5 grab the matched LHS value at that spot and sub it in in the expression.

so customers[&5] becomes customers[0] or customers[1] depending on which element of the input customers Array is being processed at that moment.

Top Results From Across the Web

JOLT transformation with two level nested array objects

The trick is using @(<int>,type) at appropriate locations in order to separate the sub-objects and sub-arrays properly such as

Array.prototype.flat() - JavaScript - MDN Web Docs

The flat() method creates a new array with all sub-array elements ... The depth level specifying how deep a nested array structure should...

Removing the depth-degeneracy in optical frequency domain ...

A novel technique using an acousto-optic frequency shifter in optical frequency domain imaging (OFDI) is presented. The frequency shift eliminates the ...

High-Level Synthesis | Vivado Design Suite User Guide - Xilinx

The Xilinx® Vivado® High-Level Synthesis (HLS) tool transforms a C specification ... TIP: Specify the depth argument with an explicit value.

Phased Array Antenna Patterns—Part 3: Sidelobes and ...

Each dot represents the amplitude of one element in this N = 16 array. ... Quantization errors are given a more in-depth treatment...