[SIP-32] Proposal for moving frontend assets to the base of the repo

[SIP] Proposal for moving frontend assets to the base of the repo

Motivation

Currently frontend assets are embedded in the superset package under superset/assets/, which is symlinked insuperset/assets/static. This is really confusing for newcomers to the project as it’s unclear why code appears to be duplicated. Additionally this is a UNIX symlink so the link isn’t created on Windows machines.

Moreover is seems there’s code (eg, MANIFEST.in, Dockerfile ) that tries to work around the fact that the entire frontend exists in the superset module and most of these files are not needed during runtime. Things like whitelisting files and removing files after builds are workarounds to the actual issue and only add complexity to the problem.

Proposed Change

• Move the frontend code into a top level directory, something like frontend or client
• Configure webpack to output the built/dist files to superset/assets/dist. This can include the package.json (needed for version info) and any other necessary runtime files.
• remove the symlinks.
• update build scripts
• make sure everything works (local builds, docker builds, python package, CI, etc.)
• update documentation.

New or Changed Public Interfaces

[no changes]

New dependencies

[no changes]

Migration Plan and Compatibility

No major/breaking changes should be necessary. The bulk of the work would be around making sure no existing builds/scripts break and documentation is updated. Since most of the changes will only be moving files, rebases should be automatic, though this will probably require a lot of existing branches to be rebased.

Rejected Alternatives

1. Remove the symlink and move everything to superset/static/assets and continue to support the whitelisting in MANIFEST.in and the cleanup in Dockerfile and possibly other places.
2. Continue to support the current file organization and continue to field questions such as: https://github.com/apache/incubator-superset/issues/8804

[DiggidyDave]

I would even go a step farther and explicitly break out top level dirs for 2 distinct, separate projects:

• superset (backend service): all things python go under here, and no python goes above it (unless it is decoupled from superset’s backend service codebase, ie utilities, scripts, etc, for working with the full project)
• superset-frontend: nodejs stuff fully contained under here

…and move all requirements*.txt, and setup.py etc–literally everything that is scoped to the backend–out of the root and into that backend svc root dir.

This would be more explicit, much much cleaner and self-documenting (IMO), and it would allow each project root to have its own well-scoped Dockerfile, .gitignore, etc in the root. It would make it clear where the test infrastructure lives for the different bits of frontend and backend (right now the whole thing is powered by python tox orchestration) and would be more intuitive for folks who are used to one stack or the other.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label #enhancement to this issue, with a confidence of 0.85. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.