Gold Digger is a Rust-based MySQL/MariaDB query tool that exports results to structured data files (CSV, JSON, TSV). Designed for headless operation and automation workflows, it provides CLI-first configuration with environment variable fallbacks.

• CLI-first design with environment variable fallbacks and comprehensive command-line interface
• Multiple output formats: CSV (RFC 4180), JSON with pretty-printing, TSV
• Safe type handling: Graceful NULL and type conversion without panics, with intelligent JSON type inference
• Secure TLS support: Built-in rustls implementation with detailed error handling and comprehensive TLS testing framework
• Comprehensive error handling: Structured exit codes, intelligent error categorization, and actionable error messages
• Shell completion: Support for Bash, Zsh, Fish, and PowerShell with easy generation
• Configuration debugging: JSON config dump with automatic credential redaction
• Query flexibility: Support for inline queries or external query files
• Verbose logging: Multi-level verbose output with security-aware credential redaction
• Empty result handling: Configurable behavior for queries returning no data
• Cross-platform: Linux, macOS, and Windows support with consistent behavior

The name "Gold Digger" refers to the tool's ability to extract valuable data from databases - just as gold miners dig through earth to find precious metal, this tool digs through database tables to extract valuable information and insights. It's designed to help you discover the "golden" data hidden within your database systems.

Download pre-built binaries from the GitHub Releases page, which include:

• Cross-platform binaries for Linux (x86_64, ARM64), macOS (Intel, Apple Silicon), and Windows
• Automated installers for easy setup
• Signed artifacts with Cosign for supply chain security
• Complete SBOMs (Software Bill of Materials) for security auditing

# Shell installer (Linux/macOS)
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/EvilBit-Labs/gold_digger/releases/latest/download/gold_digger-installer.sh | sh

# PowerShell installer (Windows)
powershell -c "irm https://github.com/EvilBit-Labs/gold_digger/releases/latest/download/gold_digger-installer.ps1 | iex"

# Homebrew (macOS/Linux)
brew install EvilBit-Labs/tap/gold-digger

# MSI installer (Windows)
# Download from releases page: gold_digger-x86_64-pc-windows-msvc.msi
# Note: The MSI installer does not include license dialogs. The MIT license is available in the LICENSE file and project documentation.

To build and install Gold Digger from source:

git clone [email protected]:EvilBit-Labs/gold_digger.git
cd gold_digger
cargo install --path .

# Standard build
cargo build --release

# Minimal build (fewer features)
cargo build --release --no-default-features --features "json csv"

Gold Digger includes built-in secure database connections with rustls implementation:

• Pure Rust TLS: Consistent cross-platform behavior using rustls with platform certificate store integration
• Platform Certificate Store: Automatic integration with system certificate stores (Windows/macOS/Linux)
• Flexible Security Controls: Four distinct TLS validation modes via mutually exclusive CLI flags
• Enhanced Error Messages: Intelligent error classification with specific CLI flag suggestions
• Security Warnings: Prominent warnings for insecure TLS modes with clear guidance

Gold Digger supports CLI-first configuration with environment variable fallbacks. CLI flags take precedence over environment variables.

# Basic usage with CLI flags
gold_digger --db-url "mysql://user:pass@localhost:3306/mydb" \
  --query "SELECT id, name FROM users LIMIT 10" \
  --output /tmp/results.json

# Pretty-print JSON output
gold_digger --db-url "mysql://user:pass@localhost:3306/mydb" \
  --query "SELECT id, name FROM users LIMIT 10" \
  --output /tmp/results.json --pretty

# Use query file instead of inline query
gold_digger --db-url "mysql://user:pass@localhost:3306/mydb" \
  --query-file query.sql --output /tmp/results.csv

# Force output format regardless of file extension
gold_digger --db-url "mysql://user:pass@localhost:3306/mydb" \
  --query "SELECT id, name FROM users LIMIT 10" \
  --output /tmp/results --format csv

# Verbose logging
gold_digger -v --db-url "mysql://user:pass@localhost:3306/mydb" \
  --query "SELECT COUNT(*) as total FROM users" --output stats.json

# Exit successfully on empty result sets
gold_digger --allow-empty --db-url "mysql://user:pass@localhost:3306/mydb" \
  --query "SELECT * FROM users WHERE id = 999999" --output empty.json

# Generate shell completions
gold_digger completion bash > ~/.bash_completion.d/gold_digger
gold_digger completion zsh > ~/.zsh/completions/_gold_digger
gold_digger completion fish > ~/.config/fish/completions/gold_digger.fish
gold_digger completion powershell > $PROFILE

# Debug configuration (credentials redacted)
gold_digger --db-url "mysql://user:pass@localhost:3306/mydb" \
  --query "SELECT 1" --output test.json --dump-config

# TLS with custom CA certificate
gold_digger --db-url "mysql://user:[email protected]:3306/mydb" \
  --tls-ca-file /path/to/internal-ca.pem \
  --query "SELECT id FROM users LIMIT 5" --output results.json

# Skip hostname verification for development
gold_digger --db-url "mysql://user:[email protected]:3306/mydb" \
  --insecure-skip-hostname-verify \
  --query "SELECT COUNT(*) FROM logs" --output count.json

# Accept invalid certificates for testing (DANGEROUS)
gold_digger --db-url "mysql://user:[email protected]:3306/mydb" \
  --allow-invalid-certificate \
  --query "SELECT * FROM test_data" --output test.csv

Note: TLS flags are mutually exclusive - use only one at a time.

Supported shells for completion generation:

• bash - Bash shell completion
• zsh - Zsh shell completion
• fish - Fish shell completion
• powershell - PowerShell completion

When CLI flags are not provided, Gold Digger falls back to environment variables:

• DATABASE_URL: MySQL/MariaDB connection URL in standard format: mysql://username:password@host:port/database
• DATABASE_QUERY: SQL query to execute
• OUTPUT_FILE: Path to output file. Extension determines format:
  • .csv → CSV output with RFC 4180 formatting
  • .json → JSON output with {"data": [...]} structure
  • .txt or any other extension → TSV (tab-separated values)

# Linux/macOS
OUTPUT_FILE=/tmp/results.json \
DATABASE_URL="mysql://user:pass@localhost:3306/mydb" \
DATABASE_QUERY="SELECT id, name FROM users LIMIT 10" \
gold_digger

# Windows PowerShell
$env:OUTPUT_FILE="C:\temp\results.json"
$env:DATABASE_URL="mysql://user:pass@localhost:3306/mydb"
$env:DATABASE_QUERY="SELECT id, name FROM users LIMIT 10"
gold_digger

# Using justfile for development
just run /tmp/out.json "mysql://user:pass@host:3306/db" "SELECT 1 as test"

Gold Digger uses structured exit codes for better automation and error handling:

• 0: Success with results (or empty with --allow-empty)
• 1: Success but no rows returned (use --allow-empty to get exit code 0)
• 2: Configuration error (missing/invalid parameters, mutually exclusive flags, TLS configuration issues)
• 3: Database connection/authentication failure (access denied, connection refused, TLS handshake failures)
• 4: Query execution failure (SQL syntax errors, type conversion errors, database-level errors)
• 5: File I/O operation failure (cannot read query file, cannot write output file, permission errors)

The exit code mapping includes intelligent error detection based on error message patterns, providing consistent behavior across different failure scenarios.

Gold Digger includes comprehensive test suites to ensure reliability and correctness across multiple database systems and configurations:

Run the standard test suite (no external dependencies):

# Run all unit tests
cargo test

# Run tests with nextest (faster parallel execution)
cargo nextest run

# Run tests excluding Docker-dependent tests
just test-no-docker

Gold Digger features a comprehensive integration testing framework that validates functionality against real MySQL and MariaDB databases using testcontainers for automated container management:

# Run integration tests (requires Docker)
cargo test --features integration_tests

# Run all tests including integration tests
cargo test --features integration_tests

# Run specific integration test categories
cargo test --features integration_tests --test tls_variants_test
cargo test --features integration_tests --test integration_tests
cargo test --features integration_tests --test tls_integration

# Using justfile commands
just test-integration  # Run only integration tests
just test-all         # Run all tests including integration tests

The integration testing framework provides:

• Multi-Database Support: Both MySQL (8.0+) and MariaDB (10.11+) testing
• TLS/Non-TLS Testing: Secure and standard connection validation with ephemeral certificates
• Container Management: Automated MySQL/MariaDB container lifecycle with health checks
• Test Data Management: Comprehensive schema and seed data covering all data types
• Format Validation: CSV (RFC4180), JSON, and TSV format compliance testing
• Error Scenario Testing: Connection failures, SQL errors, and file I/O validation
• Performance Testing: Large dataset handling and memory usage validation
• Security Testing: Credential protection and TLS certificate validation

tests/
├── integration/
│   ├── mod.rs              # Common test utilities and setup functions
│   ├── common.rs           # Shared CLI execution and output parsing utilities
│   └── containers.rs       # MySQL/MariaDB container management with health checks
├── fixtures/
│   ├── schema.sql          # Comprehensive test database schema
│   ├── seed_data.sql       # Test data covering all data types and edge cases
│   └── tls/                # TLS certificates for secure connection testing
├── test_support/           # Shared testing utilities
│   ├── cli.rs              # CLI execution helpers
│   ├── containers.rs       # Container management utilities
│   ├── fixtures.rs         # Test data and schema utilities
│   └── parsing.rs          # Output parsing and validation
├── tls_variants_test.rs    # TLS and non-TLS database variant testing
├── tls_integration.rs      # TLS connection and certificate validation
├── integration_tests.rs    # Main integration test entry point
└── database_seeding_test.rs # Database schema and data seeding tests

The framework supports comprehensive TLS testing with both secure and standard connection modes:

// TLS-enabled database testing
let tls_db = TestDatabaseTls::mysql();
let container = DatabaseContainer::new_tls(tls_db)?;

// Non-TLS database testing
let plain_db = TestDatabasePlain::mysql();
let container = DatabaseContainer::new_plain(plain_db)?;

// Connection validation
let tls_validation = container.validate_tls_connection()?;
let plain_validation = container.validate_plain_connection()?;

Integration tests use testcontainers to provide isolated database environments:

• MySQL: Versions 8.0, 8.1 with both TLS and non-TLS configurations
• MariaDB: Version 10.11+ with SSL certificate management
• TLS Configuration: Ephemeral certificate generation with secure defaults
• Test Data: Comprehensive schema covering all MySQL data types
• Fixtures: Pre-seeded test data including edge cases and Unicode content

Integration Tests:

• Docker installed and running
• MySQL and MariaDB testcontainers modules (automatically pulled)
• integration_tests feature enabled
• Tests are marked with #[ignore] by default for CI efficiency
• Sufficient disk space (~500MB for Docker images)

Unit Tests:

• No external dependencies required
• Run in CI environments
• Cover configuration, error handling, and format validation

1. Unit Tests: Fast tests without external dependencies
2. TLS Variant Tests: TLS and non-TLS database configuration testing
3. TLS Integration Tests: Certificate validation and secure connections
4. Database Integration Tests: Real database query execution and data validation
5. Container Management Tests: Database container lifecycle and health checks
6. Data Seeding Tests: Schema creation and test data population
7. CLI Integration Tests: Command-line interface and configuration testing
8. Performance Tests: Large dataset handling and memory usage validation
9. Cross-Platform Tests: Consistent behavior across Linux, macOS, and Windows

# Generate coverage report
cargo llvm-cov --html

# Generate coverage for CI
cargo llvm-cov --lcov --output-path lcov.info

# Coverage with integration tests
cargo llvm-cov --html -- --include-ignored

The integration testing framework is designed for CI environments with:

• GitHub Actions: Docker service enabled with appropriate timeouts
• Resource Management: Tests designed for shared CI resources with configurable timeouts
• Container Cleanup: Automatic cleanup prevents resource leaks
• Retry Logic: Adaptive backoff for container startup in CI environments
• Cross-Platform Support: Linux and macOS runners with Docker availability detection

The integration testing framework is actively under development. Current implementation includes:

• ✅ Core Infrastructure: MySQL/MariaDB container management with testcontainers
• ✅ TLS Support: TLS and non-TLS database variants with certificate management
• ✅ Container Management: Health checks, resource cleanup, and CI compatibility
• ✅ Test Data: Comprehensive schema and seed data for all MySQL data types
• 🚧 Advanced Testing: Data type validation, output format testing, and performance tests are planned

See the Integration Testing and TLS Variants documentation for detailed information.

Gold Digger maintains high security and quality standards for all releases:

• Signed Artifacts: All release binaries are cryptographically signed using GitHub attestation
• Supply Chain Security: Automated security scanning of all dependencies
• Software Bill of Materials (SBOM): Complete dependency information in CycloneDX format included with each release
• Cross-Platform Distribution: 6 target platforms (ARM64 & x86_64 for macOS/Linux/Windows) via cargo-dist

• Cross-Platform Testing: All releases tested on Linux, macOS, and Windows
• Code Coverage: Comprehensive test coverage tracked and maintained
• Static Analysis: Automated security analysis with CodeQL
• Zero-Warning Policy: All code passes strict linting standards

Gold Digger is authored by @unclesp1d3r

We welcome your feedback and suggestions for Gold Digger! If you have any ideas for new features, encounter any bugs or issues, or have any other comments, please reach out to us by creating an issue on our GitHub repository.

If you're interested in contributing to Gold Digger, we encourage you to submit a pull request. Please see our CONTRIBUTING.md for more information on how to get started.

Our team is committed to providing a welcoming and inclusive environment for all contributors. Please adhere to our CODE_OF_CONDUCT.md when contributing to the project.

Thank you for your interest in Gold Digger, and we look forward to hearing from you!