[DAPS-1513] Rustlike JS

[AronPerez]

Ticket

• 1513

Description

This PR implements a Rust-compatible design pattern system for the DataFed repository management, preparing the codebase for eventual Rust migration while maintaining JavaScript functionality.

Key changes:

1. Created Repository Type System - Implemented enum-like types (globus, metadata_only) to support different repository behaviors
2. Replaced Inheritance with Composition - Transformed OOP patterns to use trait-like operations and factory patterns
3. Added Result Type Error Handling - Implemented Rust-style Result types for explicit error propagation
4. Created Modular Repository Implementation - Separated concerns into:
   - types.js - Type definitions and data structures
   - factory.js - Type-based repository creation
   - operations.js - Trait-like repository operations
   - validation.js - Pure validation functions
   - globus.js / metadata.js - Type-specific implementations
5. Added Rust Book References - Comprehensive documentation linking to relevant Rust patterns

How Has This Been Tested?

• Docker Compose with ArangoDB 3.12.4
• Node.js Foxx microservices
• Local development environment

Unit Tests - Created repository_types.test.js covering:

• Type creation and validation
• Factory pattern functionality
• Operation implementations
• Error handling scenarios

Integration Tests - Created test-foxx-api.sh:

• API endpoint connectivity
• Repository CRUD operations
• Type-specific behaviors
• Error response validation

Artifacts

API Response Examples:

  // Version endpoint working at /api/1
  {
    "release_year": 2025,
    "api_major": 1,
    "api_minor": 1,
    "component_major": 1
  }

  // Repository with type field
  {
    "_id": "repo/metadata_only_repo",
    "type": "metadata_only",
    "title": "Metadata Only Repository",
    "capacity": 1000000
  }

Test Execution Output:

  === Foxx API Test Suite ===
  Testing: Version endpoint... PASSED
  Testing: Foxx service health... PASSED
  Testing: List repositories endpoint... PASSED
  Tests passed: 3

Summary by Sourcery

Introduce a Rust-compatible repository type system in JavaScript by modularizing repository logic into enum-like types, trait-like operations, and composition-based factories, update the Foxx API to use the new patterns while preserving legacy compatibility, and add supporting tests, examples, and documentation to facilitate a future Rust migration.

New Features:

• Define a RepositoryType enum and a Rust-style Result type for explicit error handling
• Implement factory-based repository creation and trait-like operations in types.js, factory.js, operations.js, validation.js, globus.js, and metadata.js
• Add a new Foxx API router (repo_router_new.js) that uses the repository type system for CRUD and allocation endpoints
• Provide an ArangoDB migration script to backfill the type field on existing repositories
• Include usage examples (example.js) and a detailed README documenting the Rust-compatible design patterns

Enhancements:

• Refactor the legacy Repo class to delegate to the new repository operations internally for backward compatibility

Documentation:

• Add a comprehensive README in the repository module illustrating the design patterns and migration benefits
• Include example code demonstrating repository creation, trait-like operations, and error handling

Tests:

• Add unit tests in repository_types.test.js covering types, factory, validation, and operations
• Add integration tests for the Foxx API endpoints to verify repository CRUD and allocation behaviors

Chores:

• Modularize repository code into separate files for better maintainability and future Rust migration

[sourcery-ai]

Reviewer's Guide

Implements a Rust-compatible repository type system with enum-like types, explicit Result-based error handling, and trait-like operations; refactors the Foxx API and legacy Repo class to leverage these modular patterns; and adds comprehensive tests, examples, documentation, and a migration script.

Class diagram for the new repository type system

classDiagram
    class RepositoryOps {
        +validate(repository)
        +createAllocation(repository, allocationParams)
        +deleteAllocation(repository, subjectId)
        +supportsDataOperations(repository)
        +getCapacityInfo(repository)
        +save(repository)
        +update(repository, updates)
        +find(repoId)
        +list(filter)
        +checkPermission(repository, userId, permission)
    }
    class Result {
        +ok(value)
        +err(error)
    }
    class RepositoryType {
        <<enum>>
        GLOBUS
        METADATA_ONLY
    }
    class ExecutionMethod {
        <<enum>>
        TASK
        DIRECT
    }
    class Repository {
        type
        data
    }
    class GlobusConfig {
        endpoint
        path
        pub_key
        address
        exp_path
        domain
    }
    class RepositoryData {
        _key
        _id
        type
        title
        desc
        capacity
        admins
    }
    RepositoryData <|-- GlobusConfig : composition
    RepositoryType <.. Repository : type
    RepositoryData <.. Repository : data
    Result <.. RepositoryOps
    ExecutionMethod <.. Result

Loading

File-Level Changes

Change	Details	Files
Introduce repository type definitions and Result pattern for explicit error handling	Define RepositoryType and ExecutionMethod enums Implement Result.ok/err constructors Add createRepositoryData, createGlobusConfig, createRepository functions Introduce createAllocationResult for allocation responses	core/database/foxx/api/repository/types.js
Implement factory pattern and trait-like dynamic dispatch for repository operations	Implement createRepositoryByType with exhaustive type-based switch Add getRepositoryImplementation and executeRepositoryOperation for dynamic dispatch Define RepositoryOps methods (save, update, find, list, checkPermission) returning Result types	core/database/foxx/api/repository/factory.js core/database/foxx/api/repository/operations.js
Extract modular validation and type-specific implementations	Add pure validation functions in validation.js Implement metadata-only trait operations in metadata.js Implement Globus-specific trait operations in globus.js	core/database/foxx/api/repository/validation.js core/database/foxx/api/repository/metadata.js core/database/foxx/api/repository/globus.js
Refactor Foxx API to leverage new system and maintain backward compatibility	Update legacy Repo class to use RepositoryOps and Result in repo.js Introduce repo_router_new.js with factory-based endpoints for list, view, create, update, and allocations	core/database/foxx/api/repo.js core/database/foxx/api/repo_router_new.js
Add testing, documentation, examples, and migration support	Add unit tests covering types, factory, validation, and operations Provide example usage in example.js Include README.md documenting design patterns and migration benefits Add migration script to populate type field on existing repositories	core/database/foxx/tests/repository_types.test.js core/database/foxx/api/repository/example.js core/database/foxx/api/repository/README.md core/database/foxx/migrations/add_repository_type.js

Possibly linked issues

• [Arch] Design Pattern Translation: Convert Repository Factory to Rust-Compatible Pattern #1513: The PR implements a Rust-compatible repository design using enums, traits, composition, and Result types, directly addressing the issue's architectural goals for Rust migration.
• [Arch] Design Pattern Translation: Convert Repository Factory to Rust-Compatible Pattern #1513: The PR directly implements the repository type system, including enum, validation, and modular design as specified.
• [Foxx, Core] Repository Type System: Implement Type Enum and Validation #1514: The PR implements the core repository type system, new API endpoints, and allocation logic, directly fulfilling the issue's requirements for type support and response format.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[AronPerez]

Note

Context: The constructor now:

• Uses RepositoryOps.find() to get the repository (line 86)
• Stores the result in this.#repository if found (line 90)

---

The old pattern let repo = g_db._document(this.#repo_id) would have directly fetched the document from the database. The new pattern already has this data stored in this.#repository from the
constructor, making it more efficient and consistent with the new repository type system.

The #repository private field contains a repository object that follows the new Rust-inspired pattern, which has a structure like:

  {
    type: "globus" | "metadata_only",
    data: { /* actual repository data */ }
  }

[Copilot]

Pull Request Overview

This PR implements a Rust-compatible design pattern system for the DataFed repository management, preparing the codebase for eventual Rust migration while maintaining JavaScript functionality. The implementation replaces inheritance-based patterns with composition and introduces explicit error handling through Result types.

• Creates modular repository type system with enum-like types (globus, metadata_only) and trait-like operations
• Implements factory patterns for type-based repository creation and Result types for explicit error propagation
• Maintains backward compatibility by refactoring the legacy Repo class to delegate to new repository operations internally

Reviewed Changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tests/repository_types.test.js	Comprehensive unit tests covering type system functionality and operations
migrations/add_repository_type.js	Database migration script to backfill type field on existing repositories
api/repository/validation.js	Pure validation functions using Result pattern for error handling
api/repository/types.js	Core type definitions including RepositoryType enum and Result type
api/repository/operations.js	Trait-like repository operations with standardized interfaces
api/repository/metadata.js	Metadata-only repository implementation with synchronous operations
api/repository/globus.js	Globus repository implementation with asynchronous task-based operations
api/repository/factory.js	Factory pattern for type-based repository creation and dynamic dispatch
api/repository/example.js	Usage examples demonstrating Rust-compatible patterns
api/repository/README.md	Comprehensive documentation of design patterns and migration benefits
api/repo_router_new.js	New Foxx API router using repository type system
api/repo.js	Legacy Repo class refactored to use new repository operations internally

[AronPerez]

Stacked diff

[JoshuaSBrown]

This is not an ssh key, we are using Curve25519, which is part of elliptic-curve cryptography that is used by ZeroMQ.

[JoshuaSBrown]

It would be better to identify which fields are found to be missing.

[AronPerez]

Updated 🫡

[JoshuaSBrown]

Shouldn't these two files be included in this PR, since they are included here? Sorry for the confusion, I know I'm creating work for you Aaron, just want to make sure I'm looking at things correctly.

[AronPerez]

It's a placeholder for the changes in https://github.com/ORNL/DataFed/pull/1543/files. core/database/foxx/api/repository/globus.js and core/database/foxx/api/repository/metadata.js are in there

[AronPerez]

All review items already addressed in code. Documentation simplified per feedback.

LGTM

[JoshuaSBrown]

Please address these two items.

message: "Missing required repository fields",

@param {string} config.pub_key - Public SSH key for authentication

[JoshuaSBrown]

This is not an ssh key, see below.

[AronPerez]

Updated https://github.com/ORNL/DataFed/pull/1523/files#diff-8321b8ffb15edeb055813863d9f5ee0155112ec11e2ca706e6ea2a01fb10a455R33