[docs] Docs restructure

From the decision in the Notion, the documentation will be restructured into each product.

Goal

Incrementally and smoothly transition to the new structure by preserving the same DX for the community and maintainers (tests should be added along the way).

The new repo structure:

Improvements are not the focus of this initiative. However, they can be listed in this issue and prioritized later.

Plan

To achieve a smooth transition, the work is split into 3 phases.

1. Preparation

Prepare E2E tests, migration tool, update scripts and add UI (each PR should be able to merge without affecting the production code, either set as feature flag or exclude from production build).

• E2E tests for the website via playwright (playwright is already existed in the project). #30128
• Create product pages config and update _app.js to switch between product. #30107
• Migration script to duplicate pages to the new location (under /material/*). #30107
• Update these scripts to compat with the new structure (it must still work with the current structure)
  • docs:api:build #30245
  • ~docs:i18n~ turns out this one needs no change because the translations directory remains the same 😊
  • docs:typescript:formatted #30248
• Update search to redirect to /material/* for material content. #30286
• Refine scripts to generate pages with the decided structure (components & APIs) #30386
• Create similar migration workflow for mui-x repo https://github.com/mui-org/material-ui-x/pull/3515
• UI: Product identifier + versioning. #30283
• UI: Product drawer. (Product drawer is open when clicking on the app icon) #30283
• Deploy to a migration.mui.com to let ahrefs crawl to make sure that all of the links are working as expected. (need to turn enable_redirects on)

2.1 Migrate content

At this phase, the new URLs have to be deployed to production so that algolia can crawl and index. However, before algolia finish the indexing I think we should preserve the old URLs (meaning, people still browsing old URLs and won’t realize that we are migrating the content).

• Run the migration scripts for Core yarn docs:migrate:pages && yarn docs:api. https://github.com/mui-org/material-ui/pull/30757
• Run the migration scripts for MUI-X yarn docs:migrate:pages && yarn docs:api. https://github.com/mui-org/material-ui-x/pull/3730

What’d happen after running the scripts

• data (demos, markdowns) will be moved from docs/src/pages/* to docs/data/*. (Both the old & new URLs are using the same demos, markdowns from the new location so we are always using the single source of truth)
• feature toggle enable_product_scope: true is turned on. This enables several things such as E2E tests for the new URLs, buildApi script, and query markdowns from the new location.
• product identifier & navigation are rendered on new locations but not on old URLs
• All of the links for existing pages remain the same, which means users won’t realize that the duplicated contents exist. (But they can accidentally access via direct URL).
• yarn docs:api will generate API pages for both previous & new URLs

3. New product spaces

Est. mid of Mar.

• Rename material to /material-ui/* https://github.com/mui/material-ui/pull/31200
• #31201
• Set up a new Algolia crawler for indexing new URLs (old URLs are neglected). This new crawler has a different ID than the existing one. (old ID for existing URLs, new ID for new URLs, this way the redirect can be done at any time. https://github.com/mui/material-ui/pull/31178
• Turn on feature toggle enable_mui_base and initialize MUI Base installation page. https://github.com/mui/material-ui/pull/30969
• Refine the search experience for multiple product spaces https://github.com/mui/material-ui/issues/31253
• Move MUI Base content from material to its own space https://github.com/mui/material-ui/issues/31414
• Write a short announcement on the blog. https://github.com/mui/material-ui/pull/32044
• Date pickers docs https://github.com/mui/mui-x/pull/3451
• Remove date pickers from Material-UI https://github.com/mui/material-ui/pull/31810

4. Go live

Est. 4th April 2022.

• Merge these PRs and deploy.
  • Core: https://github.com/mui/material-ui/pull/32048
  • X: https://github.com/mui/mui-x/pull/4313

5. Clean up

• remove material-redirects.spec e2e after redirects period
• delete old pages inside docs/pages/…`
• `remove material-old e2e test
• remove replaceHtmlLinks, replaceUrlLinks
• remove migration script
• remove old config in buildApi.ts
• use new logic in parseMarkdown.js for the API section

cc @mui-org/core @mui-org/x

[michaldudak]

Hyphens are commonly placed in URLs in place of spaces, so even though the name of the library is Material UI, to me it’s perfectly fine to have material-ui in the URL.

[mnajdova]

I feel that https://mui.com/material-ui/react-modal/ would resonate better than https://mui.com/material/react-modal/. Am I the only one?

+1 on this one