[docs] Docs restructure

[siriwatknp]

From the decision in the Notion, the documentation will be restructured into each product.

Goal

Incrementally and smoothly transition to the new structure by preserving the same DX for the community and maintainers (tests should be added along the way).

The new repo structure:

Improvements are not the focus of this initiative. However, they can be listed in this issue and prioritized later.

Plan

To achieve a smooth transition, the work is split into 3 phases.

1. Preparation

Prepare E2E tests, migration tool, update scripts and add UI (each PR should be able to merge without affecting the production code, either set as feature flag or exclude from production build).

• E2E tests for the website via playwright (playwright is already existed in the project). [test] Add E2E website tests #30128
• Create product pages config and update _app.js to switch between product. [docs] Add script to clone pages #30107
• Migration script to duplicate pages to the new location (under /material/*). [docs] Add script to clone pages #30107
• Update these scripts to compat with the new structure (it must still work with the current structure)
  • docs:api:build [core] Update buildApi script to support new structure #30245
  • docs:i18n turns out this one needs no change because the translations directory remains the same 😊
  • docs:typescript:formatted [core] Update docs:typescript:formatted to support new structure #30248
• Update search to redirect to /material/* for material content. [docs] Redirect old urls to /material/* when search #30286
• Refine scripts to generate pages with the decided structure (components & APIs) [docs] Prepare scripts for migrating to new structure #30386
• Create similar migration workflow for mui-x repo [docs] Prepare scripts + E2E tests for migration mui-x#3515
• UI: Product identifier + versioning. [docs] Add products identifier and drawer #30283
• UI: Product drawer. (Product drawer is open when clicking on the app icon) [docs] Add products identifier and drawer #30283
• Deploy to a migration.mui.com to let ahrefs crawl to make sure that all of the links are working as expected. (need to turn enable_redirects on)

2.1 Migrate content

At this phase, the new URLs have to be deployed to production so that algolia can crawl and index. However, before algolia finish the indexing I think we should preserve the old URLs (meaning, people still browsing old URLs and won't realize that we are migrating the content).

• Run the migration scripts for Core yarn docs:migrate:pages && yarn docs:api. [docs] Migrate content to the new location #30757
• Run the migration scripts for MUI-X yarn docs:migrate:pages && yarn docs:api. [docs] Migrate content to the new location mui-x#3730

What'd happen after running the scripts

• data (demos, markdowns) will be moved from docs/src/pages/* to docs/data/*. (Both the old & new URLs are using the same demos, markdowns from the new location so we are always using the single source of truth)
• feature toggle enable_product_scope: true is turned on. This enables several things such as E2E tests for the new URLs, buildApi script, and query markdowns from the new location.
• product identifier & navigation are rendered on new locations but not on old URLs
• All of the links for existing pages remain the same, which means users won’t realize that the duplicated contents exist. (But they can accidentally access via direct URL).
• yarn docs:api will generate API pages for both previous & new URLs

3. New product spaces

Est. mid of Mar.

• Rename material to /material-ui/* [docs] Use material-ui for product name #31200
• [docs] Refining the product identifier & drawer #31201
• Set up a new Algolia crawler for indexing new URLs (old URLs are neglected). This new crawler has a different ID than the existing one. (old ID for existing URLs, new ID for new URLs, this way the redirect can be done at any time. [docs] Use new Algolia app for new structure #31178
• Turn on feature toggle enable_mui_base and initialize MUI Base installation page. [docs] Add Base installation page #30969
• Refine the search experience for multiple product spaces [RFC] Search experience for new product spaces #31253
• Move MUI Base content from material to its own space [docs] Move unstyled components' docs to the Base space #31414
• Write a short announcement on the blog. [blog] Share what's changed about the new docs structure #32044
• Date pickers docs [DatePicker] Initial commit mui-x#3451
• Remove date pickers from Material-UI [docs] Update mui-x on material-ui navigation #31810

4. Go live

Est. 4th April 2022.

• Merge these PRs and deploy.
  • Core: [docs] Redirect to new urls #32048
  • X: [docs] Go live with the new urls! mui-x#4313

5. Clean up

• remove material-redirects.spec e2e after redirects period
• delete old pages inside docs/pages/...`
• `remove material-old e2e test
• remove replaceHtmlLinks, replaceUrlLinks
• remove migration script
• remove old config in buildApi.ts
• use new logic in parseMarkdown.js for the API section

cc @mui-org/core @mui-org/x

[siriwatknp]

@danilo-leal @oliviertassinari @mbrookes Can you take a look at the images for product drawer UI (in the description above), does it make sense to you?

[mbrookes]

The button for opening product content is moved to the bottom-right of the page (similar to https://reactjs.org/docs/getting-started.html)

For ReactJS it seems that it's only a FAB at mobile breakpoint where a fixed Nav is no longer practical. I don't think having it placed bottom right permanently (if that's the intention) is good UX.

Even if positioned more conventionally, I'm not sure it will be immediately obvious with just the icon how to navigate between products. Perhaps the docs need a home page?

[siriwatknp]

Perhaps the docs need a home page?

How would that help?

Note: The mobile device is only ~3% from google analytics.

[mbrookes]

How would that help?

By having a top level page that allows for navigation between products, rather than depending on an obscure icon.

Note: The mobile device is only ~3% from google analytics.`

Then why would you chose to optimise for it by using a mobile-first design pattern on desktop? (bottom-right FAB as the primary navigation mechanism for moving between products)