feat: reorganize services and specs for src cleanup (#165)

## Summary
- collapse AWS service modules under `quilt_mcp.services` and remove the
empty `quilt_mcp.aws` namespace
- refresh specs for issue #164, documenting inventory, design, and
checklists for the new directory layout
- align with latest main branch changes (new coverage tooling,
reorganized tests) and ensure `make test` passes

## Testing
- make test

Author: drernie

CHANGELOG.md

@@ -22,6 +22,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Testing**: Updated make coverage target to run comprehensive multi-suite analysis with threshold validation
 - **CI Performance**: PR feedback time reduced to <5 minutes with single Python version testing
 - **Release Safety**: Production releases now require successful main branch validation via workflow dependencies
+- **Source Layout**: Collapsed legacy `quilt_mcp.aws` namespace into `quilt_mcp.services`, updated tests/docs, and added full spec package describing the src cleanup (Issue #164)
 
 ## [0.6.6] - 2025-09-18
 

spec/164-cleanup-src/01-requirements.md

@@ -0,0 +1,32 @@
+<!-- markdownlint-disable MD013 -->
+# Requirements - Cleanup src Layout
+
+**GitHub Issue**: #164 "cleanup src"
+**Problem Statement**: The `src/` directory contains numerous single-file subfolders that add unnecessary nesting. We need a clearer layout without gratuitous behavioral changes and with all references updated accordingly.
+
+## User Stories
+
+1. **As a maintainer**, I want related modules grouped in meaningful packages so that I can navigate the codebase quickly.
+2. **As a contributor**, I want module paths to reflect actual project structure so that refactoring and new features can be implemented with less guesswork.
+3. **As a release engineer**, I want configuration and documentation references to stay accurate after the reorganization so that builds and published artifacts remain reliable.
+
+## Acceptance Criteria
+
+1. Identify single-file directories under `src/` that can be collapsed without losing organizational clarity.
+2. Consolidate modules into a rational structure that maintains behavior (imports, entry points, CLI) with minimal churn.
+3. Update all code, configuration, and top-level documentation references impacted by the new structure (excluding historical specs).
+4. Ensure automated tests, linting, and packaging workflows continue to pass after the reorganization.
+5. Communicate the new layout to developers via updated operational docs where necessary (e.g., `CLAUDE.md`).
+
+## Success Metrics
+
+1. Directory depth under `src/` is reduced for previously single-file packages while preserving semantic grouping.
+2. No runtime regressions or import errors occur when running `make test` and `make lint`.
+3. Contributors can locate modules using names that match their function without redundant folder indirection.
+4. Documentation that references affected paths is updated on the same branch with no broken links.
+
+## Open Questions
+
+1. Are there historical reasons certain modules live in their own package that we must preserve (e.g., for dynamic imports)?
+2. Do any third-party integrations depend on the existing package paths outside this repository (e.g., published API packages)?
+3. Are there additional operational docs beyond `CLAUDE.md` and top-level README that reference the current layout?

spec/164-cleanup-src/02-analysis.md

@@ -0,0 +1,46 @@
+<!-- markdownlint-disable MD013 -->
+# Analysis - Cleanup src Layout
+
+**Reference**: Requirements in `spec/164-cleanup-src/01-requirements.md`
+
+## Current Architecture
+
+1. The `src/` tree exposes the `quilt_mcp` package along with `deploy/` assets and the CLI entry point `main.py`.
+2. Within `quilt_mcp`, major domains already exist: `tools/`, `search/`, `visualization/`, `optimization/`, `aws/`, `telemetry/`, `validators/`, and `services/`.
+3. Several subpackages (e.g., `search/backends`, `visualization/generators`) are richly populated and use deeper hierarchies, while others (`services/`, `validators/`, `aws/`) are shallow wrappers around one or two modules.
+4. Some directories (such as `config/`, `operations/`, `utilities/`) appear to be stubs or legacy placeholders with no tracked `.py` files, though compiled bytecode artifacts remain from previous executions.
+5. Tests (see `tests/`) and runtime modules import functionality directly from current module paths (e.g., `quilt_mcp.tools.package_ops`, `quilt_mcp.services.quilt_service`).
+
+## Implementation Idioms & Conventions
+
+1. Public APIs are exposed via module-level functions and classes; consumers import from the package path (no dynamic discovery).
+2. `__init__.py` files typically re-export symbols (especially in `tools` and `visualization`) to simplify imports.
+3. Behavior-driven tests patch nested module paths extensively (e.g., `@patch("quilt_mcp.services.quilt_service.quilt3")`), coupling tests to the current directory layout.
+4. Configuration for packaging and entry points (e.g., `pyproject.toml`, `setup.cfg`, `Makefile`) relies on module paths inside `src/`.
+
+## System Constraints & Limitations
+
+1. Reorganization must preserve import paths consumed by published packages or external automation; breaking changes would ripple through tests and possibly deployed clients.
+2. Build tooling expects packages under `src/` following the standard `src-layout`; moving files requires updating `pyproject` metadata if package names change.
+3. Without runtime guards, collapsing packages risks circular imports if formerly separated modules now reside in a single file/scope.
+4. Tests assume module paths for patching; failing to realign them will cause import errors at test runtime.
+
+## Technical Debt & Refactoring Opportunities
+
+1. Single-module subpackages (e.g., `services`, `telemetry`, `aws`) add unnecessary nesting and cognitive overhead when the submodule names already convey domain context.
+2. Empty placeholder packages (`config`, `operations`, `utilities`) indicate either dead code or missing implementations that should be removed or populated.
+3. Some modules (e.g., `version_sync.py`, `utils.py`) serve as catch-alls; reorganizing could clarify their relationships to other domains.
+4. Lack of documentation about rationale for the current layout makes it difficult to distinguish intentional boundaries from historical baggage.
+
+## Gaps Between Current State & Requirements
+
+1. No current mapping documents which directories are candidates for consolidation or removal.
+2. There is no automated safeguard ensuring docs/configs reference the correct module paths after refactors.
+3. Tests cover behavior but may not assert on module locations; restructuring risks silent breakage if certain modules are loaded dynamically.
+
+## Architectural Challenges
+
+1. Maintaining backward-compatible import paths may require alias modules (`__init__` exports) during transition, which must be planned carefully to avoid duplication.
+2. Consolidating packages may affect relative imports inside modules; re-evaluating import strategy is necessary to prevent circular dependencies.
+3. Updating top-level documentation (e.g., `CLAUDE.md`) is mandatory per guidelines, yet we must avoid modifying historical specs.
+4. Ensuring minimal churn means batching logical moves and keeping diffs focused, but file moves inherently create large diffs and require precise test coverage.

spec/164-cleanup-src/03-specifications.md

@@ -0,0 +1,37 @@
+<!-- markdownlint-disable MD013 -->
+# Specifications - Cleanup src Layout
+
+**Reference**: Analysis in `spec/164-cleanup-src/02-analysis.md`
+
+## Desired End State
+
+1. Consolidated `src/` structure where single-module directories are either promoted into parent packages or merged into cohesive modules without losing semantic clarity.
+2. All Python imports, entry points, configuration references, and documentation are updated to match the new layout.
+3. Tests, linting, and packaging workflows run green without requiring changes to behavioral expectations.
+4. Historical specifications remain untouched; new documentation (e.g., `CLAUDE.md`) captures rationale for directory moves.
+
+## Scope & Boundaries
+
+- **In Scope**: Python modules under `src/quilt_mcp`, associated tests, CLI entry points, build configuration, top-level operational docs.
+- **Out of Scope**: Historical spec documents under `spec/`, release history, or changes to public API semantics.
+
+## Engineering Constraints
+
+1. Maintain import compatibility by either preserving module names or providing backward-compatible alias exports where possible.
+2. Keep diffs focused on file moves and necessary path updates; avoid opportunistic refactors unrelated to directory cleanup.
+3. Ensure refactoring steps follow TDD: introduce failing tests capturing expected behavior before moving code.
+4. Preserve package namespace integrity required by packaging metadata (`pyproject.toml`, `setup.cfg`).
+
+## Success Criteria
+
+1. Macroscopically, `src/quilt_mcp` exhibits a clear, intention-revealing hierarchy with minimal single-file directories.
+2. `make test` and `make lint` complete successfully post-change without flaky behavior.
+3. Tests remain behavior-driven; no direct assertions on file locations unless required to guard critical imports.
+4. Documentation explicitly notes the new module organization to aid future contributors.
+
+## Risks & Mitigations
+
+1. **Risk**: Breaking external imports. **Mitigation**: Provide compatibility shims or communicate intentional renames within the package where unavoidable.
+2. **Risk**: Missing updates in configuration or docs. **Mitigation**: Search/replace guided by checklist; run lint/tests to detect stale paths.
+3. **Risk**: Large, hard-to-review diffs. **Mitigation**: Execute moves in small, well-tested episodes guided by the workflow checklist.
+4. **Risk**: Circular imports introduced by moves. **Mitigation**: Design target structure carefully and adjust import direction to maintain acyclic dependencies.

spec/164-cleanup-src/04-phases.md

@@ -0,0 +1,23 @@
+<!-- markdownlint-disable MD013 -->
+# Phases - Cleanup src Layout
+
+**Reference**: Specifications in `spec/164-cleanup-src/03-specifications.md`
+
+## Phase 1 - Inventory & Safety Net
+
+- Audit `src/quilt_mcp` to catalog single-file directories and intended targets for consolidation.
+- Add behavior-driven tests (or expand existing ones) that lock in import expectations and key tool discovery flows before restructuring.
+- Document findings and planned moves without touching production code yet.
+
+## Phase 2 - Restructure Core Packages
+
+- Collapse identified single-file directories (e.g., `services`, `aws`, `telemetry`) into clearer module layouts while preserving behavior.
+- Adjust imports, configuration, and docs to align with the new structure.
+- Provide compatibility layers or re-export patterns where necessary to avoid breaking consumer imports.
+- Update operational documentation (`CLAUDE.md`) with the new layout insights.
+
+## Phase 3 - Cleanup & Validation
+
+- Remove obsolete placeholder packages (`config`, `operations`, `utilities`) or repurpose them with clear intent.
+- Run full lint/test/coverage suite to confirm stability.
+- Finalize checklists, ensure compatibility notes are recorded, and prepare for integration.

spec/164-cleanup-src/05-phase1-design.md

@@ -0,0 +1,21 @@
+<!-- markdownlint-disable MD013 -->
+# Phase 1 Design - Inventory & Safety Net
+
+## Objectives
+
+1. Catalog single-file directories and justify whether they should be merged or retained.
+2. Strengthen behavior-driven tests that exercise module discovery/import flows to prevent regressions during restructuring.
+3. Produce an inventory artifact that informs later phases without touching production code yet.
+
+## Key Activities
+
+- Traverse `src/quilt_mcp` and document the current tree, identifying empty or placeholder packages.
+- Review existing tests (e.g., `tests/unit/test_utils.py`, `tests/e2e/test_unified_search.py`) to confirm coverage of import-heavy workflows.
+- Add or enhance tests that assert the MCP server exposes expected tool registrations via public APIs (not file paths).
+- Record decisions in the phase checklist so reorganizations in later phases are backed by explicit rationale.
+
+## Deliverables
+
+- Updated tests capturing the expected behavior of tool discovery/imports (failing first per TDD).
+- Phase 1 checklist entries populated with inventory findings and test status.
+- No production code changes yet; only tests and documentation produced in this phase.

spec/164-cleanup-src/06-phase1-episodes.md

@@ -0,0 +1,17 @@
+<!-- markdownlint-disable MD013 -->
+# Phase 1 Episodes - Inventory & Safety Net
+
+## Episode 1 - Capture Current Layout
+
+- Script or document the existing `quilt_mcp` directory tree with annotations on single-file directories.
+- Record legacy/empty packages, noting if they are safe to remove or need further investigation.
+
+## Episode 2 - Strengthen Import Behavior Tests
+
+- Add behavior-driven tests that exercise tool discovery/import routines to fail if expected modules disappear.
+- Use factories/fixtures to interact with the public API (e.g., `quilt_mcp.utils` discovery functions) rather than importing private modules directly.
+
+## Episode 3 - Catalogue External References
+
+- Search configuration (`pyproject.toml`, `setup.cfg`, `Makefile`) and docs (`CLAUDE.md`, README) for sensitive import paths.
+- Document findings and potential impact areas in the phase checklist.

spec/164-cleanup-src/07-phase1-checklist.md

@@ -0,0 +1,19 @@
+<!-- markdownlint-disable MD013 -->
+# Phase 1 Checklist - Inventory & Safety Net
+
+ - [x] Inventory single-file directories in `src/quilt_mcp` and document decisions
+- [x] Confirm existing tests cover critical import paths
+- [x] Add new behavior-driven tests for tool discovery/imports (failing first)
+- [ ] Update checklist with catalog of external references (configs/docs)
+- [x] Run `make test-unit` to validate safety net additions
+
+## Inventory Notes
+
+- Collapsible candidates: `services/` (single functional module), `aws/` (two concrete modules plus `__init__`), `telemetry/` (multi-module but may expose single entry point), `validators/` (three validator modules sharing purpose).
+- Empty legacy packages: `config/`, `operations/`, `operations/quilt3/`, and `utilities/` (plus `utilities/aws/`) contain no tracked `.py` files; safe removal requires confirming no external imports rely on them.
+- Rich subpackages to keep structured: `search/` hierarchy, `visualization/` (analyzers/generators/layouts/utils), and `optimization/` already house multiple cohesive modules.
+
+## Test Notes
+
+- Confirmed existing unit/integration tests exercise imports for service-layer functionality.
+- Added failing test (`TestQuiltServiceAuthentication.test_services_package_exports_core_classes`) to require `quilt_mcp.services` to expose service classes directly.

spec/164-cleanup-src/08-phase2-design.md

@@ -0,0 +1,28 @@
+<!-- markdownlint-disable MD013 -->
+# Phase 2 Design - Restructure Core Packages
+
+## Objectives
+
+1. Collapse targeted single-module directories into a clearer hierarchy while preserving public behavior.
+2. Provide compatibility exports or adapters so existing imports remain functional.
+3. Update code, configs, and documentation to reflect the new layout.
+
+## Target Structure (Tentative)
+
+- Flatten service implementations under `quilt_mcp/services/` so core classes (`QuiltService`, `AthenaQueryService`, permission discovery primitives) share a package.
+- Delete the legacy `quilt_mcp/aws/` namespace entirely after migrating modules to `services/`.
+- Evaluate `telemetry/` and `validators/` to determine whether consolidating into single modules improves clarity or whether subpackages remain justified.
+- Remove empty packages (`config`, `operations`, `utilities`) after ensuring no references depend on them.
+
+## Key Activities
+
+- Move/rename files using tooling that preserves git history where possible.
+- Adjust import statements across the codebase (tests, runtime, configs) to reference the new module locations.
+- Introduce compatibility imports (e.g., re-export moved symbols from legacy module paths) when necessary to keep changes non-breaking.
+- Keep changes behavior-focused; defer stylistic refactors unless required to maintain functionality.
+
+## Deliverables
+
+- Updated module layout matching the agreed structure.
+- Passing tests and lint checks with updated imports.
+- Documentation updates (especially `CLAUDE.md`) describing the new organization and compatibility notes.

spec/164-cleanup-src/09-phase2-episodes.md

@@ -0,0 +1,22 @@
+<!-- markdownlint-disable MD013 -->
+# Phase 2 Episodes - Restructure Core Packages
+
+## Episode 1 - Services Module Consolidation
+
+- Co-locate `QuiltService`, `AthenaQueryService`, and permission discovery primitives under the `quilt_mcp.services` package.
+- Update tests and runtime references to import from the new structure.
+
+## Episode 2 - Remove Legacy AWS Namespace
+
+- Delete the `quilt_mcp/aws` package after migrating modules.
+- Ensure all imports reference `quilt_mcp.services` equivalents.
+
+## Episode 3 - Telemetry & Validators Review
+
+- Decide whether telemetry and validator packages remain multi-module or should expose simplified entry points.
+- Apply renames/moves as needed, adding compatibility exports.
+
+## Episode 4 - Documentation & Config Updates
+
+- Update configuration files and `CLAUDE.md` with the new structure.
+- Verify no stale paths remain via targeted search.