[SIP-4] [Embeddable Charts] Create Superset JS Client

@kristw @conglei @mistercrunch @betodealmeida @hughhhh

cc @john-bodley @michellethomas @graceguo-supercat @timifasubaa

[SIP] [Embeddable Charts] Create Superset Client

This is the first of a few SIPs to progress toward a world where we can embed Superset Charts as React components in other web applications. At a high level this will require @superset/xxx NPM packages with data providers + chart components.

Motivation

This specific SIP is for creating a Superset JS client to handle asynchronous requests for data including authentication.

The motivation for this change is two-fold

1. address technical debt by moving from ajax to fetch
2. encapsulate all request logic into a single JS class, which will enable issuing requests for data from other web applications

Proposed Change

Write a single JS SupersetClient class that

• handles CSRF tokens and verifies authentication
• supports GET/POST/PUT/DELETE methods
• supports customization of headers, etc. if needed

After this we can write corresponding React components to handle fetching specific types of Superset data (including in other apps):

• <DatasourceProvider /> returns available datasources
• <MetadataProvider /> given a datasource id, returns metadata on all of its columns, metrics, etc.
• <QuerydataProvider /> given a query, returns the data for that query; this can be uncoupled from visualizations over time.

We can do this in three phases:

1. Write the client, considering all existing request types
2. Refactor existing ajax requests to leverage new client
3. Write react components

New or Changed Public Interfaces

Example usage of the client:

import SupersetClient from 'xxx';

// Singleton instance of the client
const client = SupersetClient.getInstance({
  host: 'http://0.0.0.0:8088',
  cors: 'enable', // talking to the same domain or not
});

// verify authentication and fetch CSRF
client.init()
  .then(...) // now make some requests
  .catch(...); // in another app, link to Superset for Auth

// later
client.get({ endpoint: 'datasources/' });
client.post({ endpoint: 'explore_json/', postPayload: ... });

Authentication and CORS

In order to support requests made from other web applications, we must

• enable CORS (cross origin request sharing) and
• ensure authentication

The Superset backend already supports enabling CORS for specific endpoints and resources. This is an example config.py:

# CORS Options
ENABLE_CORS = True
CORS_OPTIONS = { 
  "resources": { 
    "*": {
      "origins": "http://localhost:9001", "supports_credentials": True, 
    }
  }
}

By setting the fetch parameter { credentials: 'include' } we can also forward credentials for the current user.

Proof of concept

This is a proof of concept using the SupersetClient code in another web application (PR to come). It demonstrates

• ability to verify user authentication
• ability to fetch and use CSRF token in requests
• support for GET and POST requests for Superset data that ultimately powers a chart.

New dependencies

This should only require the whatwg-fetch polyfill, which has been added / considered in other PRs (eg #5524) and has no license concerns.

Migration Plan and Compatibility

We don’t anticipate that any migration will be necessary or breaking compatibility changes.

Rejected Alternatives

We considered a client that also handles redux state management, but this seems too complex to start and could be added as another layer in the future if it was needed for embeddable charts (we think it won’t be)

Major open questions

A major open question is WHERE this code (and other future @superset/... JS npm packages) should ultimately live. The options are:

1. within the apache/incubator-superset repo under a src/packages/ subfolder, or

2. a separate repository (which?!)

Pros of apache/incubator-superset

• Centralized, all code in one place
• Member management in one place
• Centralized place for all contributions, dependencies, etc.

Cons of apache/incubator-superset

• Combine issues from all components. (could address with per-package tags)
• Possible confusion if/when superset app version dependencies differ from the most-up-to-date packages/* version (ie superset app does not pull from local package source code, rather it pulls from npm for a particular version of that source code)
• Deprecating a package means deleting a code from central repo instead of leaving the git repo alone.
• Combined git log can be very polluted

Overall it seems we should move packages to a different repsoitory. Where should this be? (note: I grabbed the npm @superset namespace for publishing so we should be good there! 👍

[victornoel]

@williaster thanks for your answer, this new endeavour around embeddable charts and chars as plugins is really a great advancement for superset I think 😃

[williaster]

Going to close this as complete: @superset-ui/connection now powers all async requests in the Superset app, and several other modular packages are under active development in @superset-ui to power other ideas described here and captured in other SIPs.

Noting that this SIP was used more as formalizing architecture design and not for any formal voting.