A CLI for evaluating FOLIO modules against technical council criteria.

This tool provides a modular, extensible framework for automatically evaluating FOLIO modules against acceptance criteria. It currently supports Java modules with a pluggable architecture for adding support for additional programming languages.

Note: Dependency analysis includes all transitive dependencies for Maven and npm projects.

⚠️ WARNING: This tool executes build commands (Maven, Gradle, npm) on cloned repositories. Malicious build files (pom.xml, build.gradle, package.json) can execute arbitrary code with your user's permissions.

For local development, use the provided devcontainer configuration in .devcontainer/. The container provides isolation with all required tools (Node.js, Java 21, Maven, Gradle).

Use any devcontainer-compatible tool (VS Code, JetBrains IDEs, GitHub Codespaces, devcontainer CLI, or Docker directly).

Note: Running this tool in containers as part of CI/CD pipelines is appropriate and expected, as CI systems already provide isolation.

Note: Only install locally if you understand the security risks. The tool will execute build commands on untrusted repositories.

# Install dependencies
yarn install

# Build the project
yarn build

# Install globally (optional)
yarn global add .

1. In this tc-module-eval project, click Actions and then (in the left sidebar) FOLIO Module Evaluation. Or load that page directly.

2. Click Run Workflow. In the popup,

   • Enter the GitHub URL of the module to evaluate.

   • Click Run Workflow.

3. Wait for the workflow run to complete, 1-2 minutes, then click into it.

4. Under Artifacts, click to download the reports (as a zip file) for review.

# Evaluate a repository
folio-eval evaluate https://github.com/folio-org/mod-search

# Evaluate specific criteria only
folio-eval evaluate <repo-url> --criteria S001,S002,B005

# Custom output directory or format
folio-eval evaluate <repo-url> --output ./my-reports --json-only

# List supported languages
folio-eval list-languages

# Show framework information
folio-eval info

1. Clone the target repository
2. Detect appropriate language evaluator
3. Load criteria definitions for the detected language
4. Run evaluation against criteria (with optional filtering)
5. Generate reports (HTML + JSON)
6. Clean up temporary files (unless --no-cleanup is specified)

Criteria are defined in TypeScript in src/criteria-definitions.ts with explicit identifiers and metadata:

Each criterion includes:

• ID: Unique identifier (e.g., A001, S001, B001, F001)
• Code: Same as ID for consistency
• Description: Human-readable description of the requirement
• Section: Category (Administrative, Shared, Backend, Frontend)
• Category: Subcategory for organization

• A###: Administrative criteria (e.g., A001 - License compliance)
• S###: Shared/Common criteria applicable to all modules (e.g., S001-S014)
• B###: Backend-specific criteria for Java modules (e.g., B001-B016)
• F###: Frontend-specific criteria for UI modules (e.g., F001-F007)

Use the --criteria option to filter which criteria are evaluated (e.g., --criteria S001,S002,B005)

• Visual dashboard with color-coded results
• Statistics summary (pass/fail/manual counts)
• Detailed criterion-by-criterion breakdown
• Responsive design for various screen sizes

• Machine-readable format for integration
• Complete evaluation data including evidence
• Suitable for further processing or analysis

• Node.js 18.x or later
• Yarn 1.22.x or later
• Maven 3.x or later (required for evaluating Java modules)

# Clone and install dependencies
git clone <repository-url>
cd module_eval
yarn install

# Development mode (TypeScript, no build required)
yarn dev evaluate <repo-url>
yarn dev info

# Production mode (requires build)
yarn build
yarn start evaluate <repo-url>

# Testing
yarn test
yarn test --watch

# Clean and rebuild
yarn clean && yarn build

The framework includes configuration files that can be updated manually:

• config/license-categories.json - License category mappings (A=Compatible, B=Conditional, X=Prohibited)
• config/license-variations.json - License name variations and normalizations
• config/special-exceptions.json - Special case handling for specific dependencies

These files follow the Apache Software Foundation license categorization policy.

Apache-2.0 License