reference data by url as signal in compiled vega

As was concluded in this issue: https://github.com/vega/vega/issues/2597.

In a VL spec where the data is referenced by values such as:

"data": {
  "name": "SOURCE",
  "values": {
    "nested": [
      {"key": "alpha", "foo": [1, 2], "bar": ["A", "B"]},
      {"key": "beta", "foo": [3, 4, 5], "bar": ["C", "D"]}
    ]
  },
  "format": {"property": "nested"}
}

Is compiled into Vega as such (stripped):

{
  "data": [
    {
      "name": "SOURCE",
      "values": {
        "nested": [
          {"key": "alpha", "foo": [1, 2], "bar": ["A", "B"]},
          {"key": "beta", "foo": [3, 4, 5], "bar": ["C", "D"]}
        ]
      },
      "format": {"property": "nested"}
    },
    {
      "name": "data_0",
      "source": "SOURCE",
      "transform": [
        {"type": "flatten", "fields": ["foo", "bar"], "as": ["foo", "bar"]}
      ]
    }
  ]
}

In a VL spec where the same data is referenced by url such as:

"data": {
  "name": "SOURCE",
  "url": "https://tsws.hkvservices.nl/mangrove.ws/data.ashx?function=dataportal.db.getdata&parameters={database:%27vega%27,key:%27nested_data%27}&contentType=application/json",
  "format": {"property": "nested"}
}

The Vega is compiled as such (stripped):

{
  "data": [
    {
      "name": "SOURCE",
      "url": "https://tsws.hkvservices.nl/mangrove.ws/data.ashx?function=dataportal.db.getdata&parameters={database:%27vega%27,key:%27nested_data%27}&contentType=application/json",
      "format": {"property": "nested", "type": "json"},
      "transform": [
        {"type": "flatten", "fields": ["foo", "bar"], "as": ["foo", "bar"]}
      ]
    }
  ]
}

In the latter the transform is applied in the same data source. This misalignment leads to confusing behaviour when removing datasets from the chart, as was discussed in above mentioned issue.

[domoritz]

That super useful context! Let’s try to create signals for data URLs then. I bumped the priority up.

[mattijn]

For companies that work with multiple people with different skills-set, such as:

• Data comes from person a
• Vega(-Lite/Altair) specifications comes from person b
• Person c combines all together within an app or web application.

Data is improved all the time and Vega(-Lite) specifications mature over time as well. Being able to reference data by URL dynamically is a real pain reliever.

For that reason the applications we develop are roughly 75% referenced by URL (through signal) and 25% referenced by values.

It’s not that we don’t reference by values, but adopting the URL-signal route comes naturally as it is:

• very convenient for person c (an improved json-spec can be updated by merely a copy+paste in the front-end) and
• person b can develop/improve/debug instantaneously in Vega-editor.

Especially with a growing Altair/Vega-Lite community it would be great that these people can directly integrate their visuals within bigger frameworks as is possible with Vega specifications.