[SIP-74] Superset 2.0 Breaking Changes

[SIP-74] Superset 2.0 Breaking Changes

Status: Voting

The following GitHub project can be used to track the status of individual proposed changes: https://github.com/apache/superset/projects/35

Motivation

Superset has built up a large number of complex behaviors with the aim of maintaining backwards compatibility. As such complexities build up, bugs appear and development becomes more difficult. Superset 2.0 is an opportunity to release a clean API, spartan feature flag options, and far smaller volume of deprecated code.

Proposed Change

Breaking Changes List:

• Remove PUBLIC_ROLE_LIKE_GAMMA
• Turn on SQLLAB_BACKEND_PERSISTENCE by default
• Enable flag for disabling Legacy Datasource Editor.
• Enable D&D w/ hybrid by default
• Remove old import/export code and turn on the VERSIONED_EXPORT flag by default.
• remove SUPERSET_WORKERS
• Remove ROW_LEVEL_SECURITY feature flag (permanently set to true)
• Upgrade minimum supported Python version to 3.8.
• Remove deprecated Celery CLI.
• Remove legacy SIP_15_* logic/flags. This feature has been in flight for a number of years which is sufficient time for institutions to have upgraded.
• remove OLD_API_CHECK_DATASET_OWNERSHIP
• remove legacy ALERTS code and config keys
• Remove ENABLE_REACT_CRUD_VIEWS flag, and sweep up old code
• change the ENABLE_JAVASCRIPT_CONTROLS from a config value to a feature flag
• remove SQLALCHEMY_DOCS_URL and SQLALCHEMY_DISPLAY_TEXT from the config and move to superset_text.js so that it’s in line with the rest of the customizable text options.
• generally remove all code marked as “deprecated” * is deprecated and will be removed in version 2.0.0

We should plan what breaking changes go into this release well in advance to make it as painless as possible.

The blue-sky ideal is that the only breaking changes will be removal of deprecated code and related feature flags. This way, the code we’re shipping will already be mature and production-validated.

Large projects introducing breaking changes bring excessive risk. If we can mature them behind a feature flag first, then we can remove the flag for 2.0 to make it a small, lower-risk change. Changes that cannot be kept behind a feature flag should maintain backwards compatibility so they can go in a minor release instead of a major one.

Projects that do not relate to planned breaking changes should be able to continue development as normal.

Migration Plan and Compatibility

For Long Term Support of Superset 1.x, we should back-port critical fixes to a 1.x.x release. “Critical” here generally means security fixes and high-impact bugs that are able to be back-ported. Porting features, and many of the more complex bugfixes, isn’t likely to be practical considering the fluidity of the Superset codebase. The LTS release should be supported until the next major version, at which point 2.x becomes the LTS release.

To begin the major version release process, we should split off a lts-v1 branch. From that point, master will be considered to be 2.x only. This will give us space to actually make the breaking changes, test, and fix any issues before building the release. Future 1.x changes can be merged to the lts-v1 branch, and we can build 1.x.x releases from there.

We may want to offer some sort of migration assistant that will analyze your Superset configuration and give you hints of how you should prepare for the upgrade.

Rejected Alternatives

Never bumping major versions?

[michael-s-molina]

How long do we plan to support LTS releases? 18, 30 months?

[dpgaspar]

This is great!

Removing old API endpoints, by first migrating them and dropping a deprecation warning on the old code, is a pattern we have been following that helps our users prepare and know in advance what to expect on 2.0. This can probably be replicated for deprecating the old Druid Connector, and on all the CRUD MVC views or even others. Make 2.0 less frighting 😃