Adopt new end-to-end test plan

[LayZeeDK]

Summary

Adopt the new cost-efficient end-to-end test plan for @nxworker/workspace. This replaces legacy correctness specs (excluding performance/stress specs migrating to benchmark suite) with modular scenario functions executed via a single orchestrator spec.

End-to-End Test Plan (Cost-Efficient Suite) for @nxworker/workspace

Objective

Replace traditional correctness-focused e2e tests with a lean benchmark suite that: (1) validates critical user flows (publish → install → generate → move-file operations) and (2) produces stable performance metrics across a 6-OS CI matrix (triggered on merge to main and manual workflow_dispatch). Optimize for low runtime, determinism, and actionable performance deltas while minimizing environment variability and unnecessary file I/O.

Core Principles

1. Critical Path Coverage: Only test flows proving the plugin works when consumed (publish → install → generators).
2. Deterministic: Pin versions, fixed workspace names, seeded ordering; avoid "latest" installs.
3. Minimal Fixtures: Most scenarios use 2-3 library projects only; include an application project only for app→lib move scenarios.
4. Cross-Project Moves: Include app → lib and lib → lib scenarios to exercise differing source roots.
5. Options Coverage: At least one scenario per optional generator flag.
6. Fast Failure: Abort suite if local publish or install sanity checks fail.
7. Isolation: Each scenario cleans its temp workspace; no hidden state coupling.
8. Explicit Imports & Paths: Avoid barrels except package entrypoint; keeps assertions simple.
9. Live Output: Use child process spawning with stdio: 'inherit' for Nx/registry commands to surface progress; keep custom logs minimal.
10. Reliable Cleanup: Registry and temp dirs torn down even on failure.

Scenario Catalogue

Domain	ID	Purpose	Notes
Local Registry	REG-START	Start Verdaccio and confirm package availability	Always run
Publish Flow	PUBLISH	Local publish of plugin (dry + actual)	Always run
Install Flow	INSTALL	Create new workspace (2 libs), install plugin, simple import check	Always run
Basic Generator	MOVE-SMALL	Move single file lib→lib (workspace with 2 libs) default options	Always run
App to Lib Move	APP-TO-LIB	Move a file from application project to library (path adjustments)	Extended coverage
Explicit Directory	MOVE-PROJECT-DIR	Move with projectDirectory specified	Option coverage
Derive Directory	MOVE-DERIVE-DIR	Move with deriveProjectDirectory=true (mutually exclusive)	Option coverage
Skip Export	MOVE-SKIP-EXPORT	Move exported file with skipExport flag and assert index unchanged	Option coverage
Skip Format	MOVE-SKIP-FORMAT	Move file with skipFormat=true (assert unformatted state minimal)	Option coverage
Allow Unicode	MOVE-UNICODE	Move file with Unicode characters in path when allowUnicode=true	Option coverage
Remove Empty Project	MOVE-REMOVE-EMPTY	Move last source files to new project triggering removal	Option coverage
Path Aliases	PATH-ALIASES	Workspace with 3 libs; moves across multiple tsconfig path aliases	Import rewriting correctness
Export Updates	EXPORTS	Move exported file and verify index updated (normal path)	Export rewriting correctness
Repeat / Idempotence	REPEAT-MOVE	Re-run MOVE-PROJECT-DIR ensuring no duplicate exports	Idempotence
Graph Reaction	GRAPH-REACTION	Force project graph rebuild after moves	Graph update correctness
Scale Sanity	SCALE-LIBS	Generate many small libs (≥10) then one lib→lib move (no app)	Structure sanity under scale
Smoke Sentinel	SMOKE-SENTINEL	Combined publish+install+single move (app→lib; workspace with 1 app + 1 lib)	Composite path coverage

(Deliberately excluding performance measurement details; handled in separate issue. CI remains unchanged and continues using existing e2e target/matrix.)

Reliability & Flake Control

• Pin all toolchain versions.
• Keep scenarios minimal to reduce variability.
• Use retry only for initial scaffold network fetch (one retry max).
• Fail fast on registry or publish issues to avoid cascading noise.

Implementation Steps (Ordered)

1. Implement harness utilities (registry control, workspace creation helper supporting 2-3 libs and optional app).
2. Add scenario modules implementing each ID above.
3. Create orchestrator spec invoking scenarios directly (single Jest test file).
4. Reuse Verdaccio server across scenarios with global setup/teardown.
5. Migrate existing legacy e2e spec content into discrete scenario functions (exclude performance/stress test spec files; those are migrating to e2e benchmarks)
6. Validate all generator option scenarios locally.
7. Remove legacy flows after two stable runs.

Orchestrator Spec Rationale

Using a single orchestrator spec:

• Reuses expensive setup (Verdaccio, workspace scaffold) once, reducing runtime and flakiness.
• Centralizes ordering, inclusion filtering, and cleanup in one place—simplifies maintenance.
• Enables selective execution (fast vs full) via env flags instead of per-file naming patterns.
• Minimizes CI noise: consolidated output through stdio: 'inherit'.
• Preserves modularity: each scenario is an isolated run() function with shared helpers.
• Eliminates duplicate registry/workspace lifecycle code.
• Adds/removes scenarios with a small diff (catalogue + orchestrator import) improving review clarity.

Scenario Design Guidelines

• Keep scenario code < ~100 lines; refactor helpers aggressively.
• Single domain focus per scenario.
• Avoid retries unless transient scaffold failure.
• Use direct Nx commands with stdio: 'inherit' to stream progress; avoid nested package managers beyond necessity.
• Pre-check Verdaccio port before start.
• Minimize logging noise.

Author Checklist (Non-Measurement)

• Assert only critical invariants (presence of install, correct version, successful move). Avoid broad file tree diffs; however, perform targeted import path assertions in scenarios requiring correctness of rewritten imports (PATH-ALIASES, EXPORTS, REPEAT-MOVE, APP-TO-LIB). Deep diff only when functionally required for these import updates.
• Keep fixtures small; batch only when it reduces setup duplication.
• Reuse initialized resources within a scenario; tear down once.
• Avoid adding timing/memory collection in this suite.
• No timers beyond what is required for readiness polling.
• Keep file I/O minimal; no metric JSON output.
• Disable verbose logging by default; rely on stdio: 'inherit' streaming for visibility.
• Seed any pseudo-random ordering to ensure reproducibility.
• Keep temp workspace path short (helps Windows path limits).
• Use the custom uniqueId() helper; do not import lodash uniqueId. Rationale: lodash uniqueId is only a process-local incremental counter and not globally unique, risking collisions across parallel processes/CI runners.

Risk & Mitigation

Risk	Impact	Mitigation
Flaky network during scaffold	False regressions	Pin versions + single retry + fallback smoke skip with note
Port conflicts	Setup failures	Pre-check & incremental port search (limit 3 tries)
CI load variation	Metric noise	Compare against per-OS baseline; require consecutive breaches before failure
Memory metrics variance	False alerts	Use broad thresholds; rely mainly on duration for gating
Large scale scenario time drift	Nightly noise	Isolate extended suite; exclude from merge gating

Acceptance Criteria

• All defined e2e scenarios listed in the Scenario Catalogue pass across all configured OS/CPU platforms (Windows, macOS, Ubuntu arm64/x64) for two consecutive runs.
• Legacy tests considered retired once this universal pass condition is met.

Open Questions

1. Validation window length?
2. Failure snapshot granularity?
3. Registry port fallback strategy?