Data Package Data Views Specification (Visualizations)

UPDATE August 2017

Draft spec at: http://specs.frictionlessdata.io/views/

UPDATE March 2017

v1.0 DRAFT SPEC AVAILABLE HERE: https://hackmd.io/s/SyTcmXPwl

---

---

OLD

What: a JSON spec for describing views of data like graphs, tables or maps. Probably focus this just on graphs to start with.

This spec is motivated by desire to include views support in to Data Packages. However, we aim to make this generic and reusable outside of Data Packages and we will avoid or flag anything Data Package specific.

This was motivated by experience with ReclineJS where we needed to serialize “view” configurations (e.g. graphs and maps) into JSON. I have been using this informally in data packages for a while - see e.g. https://github.com/datasets/house-prices-us/blob/master/datapackage.json#L157

Proposal

The datapackage.json MAY contain an attribute named views. The value of views MUST be an array where each entry is a data “view” descriptor. Each data view descriptor MUST be a JSON object and MUST have a type key which uniquely defines the type of view (and what other attributes should be present).

Here’s an example:

{
    "id": "...",
    "title": "...",
    "type": "Graph",
    # this is specific to Data Package
    # optional - resource id or name to pull data from. defaults to first resource if not specified
    "resource": 1,
    "state": {
      # this will be specific to each type of graph
      graph-state ...
    }
  }

Planned

Whilst new tools can specify their own “view” information we provide define 2 specific types:

• type: graph
• type: map
• type: vega - follow the vega spec
  • important question

type: graph

Propose: we use Flot or Vega spec - http://trifacta.github.io/vega/

type: map

Propose: we run this off leaflet (??)

Research

• Vega: https://github.com/trifacta/vega/wiki
  • Vega has a lot of relevance. However, it is very oriented to being a JSON version of the “grammar of graphics”. As such it is somewhat painful to use for very simple things out of the box like a line graph or simple map. However, there is no reason that Vega could not be a subtype e.g. type: vega as a graph type in Data Views.
• Plotly have a spec it seems according to their Open Source announcement in Nov 2015: https://plot.ly/javascript/open-source-announcement/ - actual schema is here http://help.plot.ly/json-chart-schema/
  • Quote from announcement re relation to vega-lite: “The plotly.js JSON schema and API is more like MATLAB or Python’s matplotlib than most JavaScript charting libraries available. It focuses on the chart’s physical attributes and attempts to leave the chart data separate (a workflow that scientists and engineers are accustomed to). For chart types that require binning (contour plots, histograms) or min-max decimation (line plots with >100k lines), some precomputation in JavaScript has been unavoidable.”
• Flot “spec”: https://github.com/flot/flot/blob/master/API.md
  • TODO: more analysis and documentation here.
• http://tenxer.github.io/xcharts/docs/#data
• VizJSON from IBM - http://www.ibm.com/developerworks/opensource/library/bd-vizjson/index.html (grammar of graphics oriented)

[domoritz]

I think you guys will like https://github.com/vega/vega-lite/pull/3417. It makes injecting data into a Vega-Lite spec quite easy.

[rufuspollock]

First draft of a views spec is out:

http://specs.frictionlessdata.io/views/

Especially recommend reading: http://specs.frictionlessdata.io/views/#concepts-and-background

Current text is unpolished but underlying model is quite robust and has been actively used and tested for the last 9-12m and is in active use in production in https://datahub.io/