Skip to content

A secure, scalable Python-based system for managing GitHub utility scripts with virtual environment isolation.

Notifications You must be signed in to change notification settings

spatnaik17/github-utils

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GitHub Utilities

A secure, scalable Python-based system for managing GitHub utility scripts with virtual environment isolation.

Overview

This system provides a centralized way to run GitHub utility scripts with:

  • Virtual Environment Isolation: All scripts run in an isolated Python environment
  • Secure Execution: Environment variable validation and secure script loading
  • Extensible Architecture: Easy to add new utilities
  • Configuration Management: JSON-based configuration system

Features

  • 🔒 Secure Virtual Environment: Automatic setup and management
  • 🚀 Easy Execution: Single command to run any utility
  • 📊 Repository Analyzer: Comprehensive analysis of all your repositories
  • 🔄 Fork Updater: Bulk update all forked repositories with upstream changes
  • Multi-Threaded Fork Updater: High-performance concurrent repository syncing
  • 🍴 Forks Lister: List all forked repositories with parent information
  • 🔱 Repository Forker: Fork repositories using GitHub API without local downloads
  • ⚙️ Configurable: JSON configuration for customization
  • 🎯 Error Handling: Robust error handling and validation

Quick Start

Interactive Mode (Recommended for First-Time Users)

Simply run the script without arguments to enter Interactive Mode:

./run_github_utils.sh

Interactive Mode features:

  • 🎯 Guided Experience: Step-by-step menus for all utilities
  • 🔐 Credential Setup: Prompts for GitHub token and username if not set
  • 📝 Parameter Help: Interactive prompts with defaults and explanations
  • Safe Defaults: Recommended settings for all operations
  • 🔄 Continuous Operation: Run multiple utilities without restarting

See INTERACTIVE_MODE.md for detailed guide.

Prerequisites

  • Python 3.8 or higher
  • Git (for fork updater functionality)
  • GitHub personal access token

Authentication

For Git operations (clone/fetch/push), prefer one of:

  • GitHub CLI (HTTPS):

    • Install gh and authenticate once: gh auth login
    • Configure Git to use gh-managed credentials: gh auth setup-git
    • Keep remotes as clean HTTPS URLs (no tokens in URLs). This is the recommended method.
  • SSH:

    • Set up an SSH key with GitHub and ensure ssh -T [email protected] succeeds
    • Use --use-ssh on fork updaters to use SSH remotes

Note: Environment variable GITHUB_TOKEN is still required for API calls (listing repos, analysis, etc.). It is no longer embedded in Git remote URLs.

Setup

Linux/macOS/WSL

  1. Set Environment Variables:

    export GITHUB_TOKEN="your_github_token_here"
    export GITHUB_USERNAME="your_username_here"
  2. Create GitHub Token:

  3. Run Setup:

    ./run_github_utils.sh setup

Windows

Option 1: Command Prompt (Batch)

  1. Run the installer:

    install_windows.bat
  2. Set Environment Variables:

    set GITHUB_TOKEN=your_github_token_here
    set GITHUB_USERNAME=your_username_here
  3. Use the utilities:

    run_github_utils.bat list
    run_github_utils.bat fork_downloader --repos-dir "C:\GitHub\Forked"

Option 2: PowerShell

  1. Run the installer:

    .\install_windows.ps1
  2. Set Environment Variables:

    $env:GITHUB_TOKEN = "your_github_token_here"
    $env:GITHUB_USERNAME = "your_username_here"
  3. Use the utilities:

    .\run_github_utils.ps1 list
    .\run_github_utils.ps1 fork_downloader -repos-dir "C:\GitHub\Forked"

Setting Permanent Environment Variables (Windows):

  1. Search for "Environment Variables" in Windows Start menu
  2. Click "Edit the system environment variables"
  3. Click "Environment Variables..." button
  4. Add GITHUB_TOKEN and GITHUB_USERNAME as user variables

Basic Usage

Linux/macOS/WSL

# List available utilities
./run_github_utils.sh list

# Run repository analyzer
./run_github_utils.sh analyzer

# Run fork updater (HTTPS via GitHub CLI or credential helper)
./run_github_utils.sh fork_updater

# Run fork updater using SSH remotes
./run_github_utils.sh fork_updater --use-ssh

# Run fork updater with custom directory
./run_github_utils.sh fork_updater --repos-dir ~/my_forks

# Generate HTML repository report
./run_github_utils.sh html_reporter

# Generate HTML report with custom output directory
./run_github_utils.sh html_reporter --output-dir ~/reports

# Search for popular repositories by topic
./run_github_utils.sh topic_searcher --topic python

# Search with custom parameters
./run_github_utils.sh topic_searcher --topic machine-learning --limit 20 --sort-by forks

# Search for top contributors by topic
./run_github_utils.sh contributor_searcher --topic python

# Search with custom contributor parameters
./run_github_utils.sh contributor_searcher --topic react --limit 15 --min-repos 5

# Analyze organization/user repositories
./run_github_utils.sh org_analyzer microsoft

# Analyze user repositories with custom output directory
./run_github_utils.sh org_analyzer torvalds --output-dir ~/reports

# Download missing forked repositories
./run_github_utils.sh fork_downloader

# Download with SSH and custom directory
./run_github_utils.sh fork_downloader --repos-dir ~/forks --use-ssh

# Clone all missing repositories
./run_github_utils.sh repo_cloner

# Clone only owned repositories with SSH
./run_github_utils.sh repo_cloner --repo-type owned --use-ssh --repos-dir ~/projects

# Analyze starred repositories and followed contributors
./run_github_utils.sh stars_followers

# Generate stars & followers report with custom output directory
./run_github_utils.sh stars_followers --output-dir ~/reports

# List all repositories from users you follow
./run_github_utils.sh following_repos

# Filter repositories by criteria
./run_github_utils.sh following_repos --repo-type owned --min-stars 10

# Find recent Python repositories from followed users
./run_github_utils.sh following_repos --language Python --days-since-update 30

# List all forked repositories
./run_github_utils.sh list_forks

# List forks with HTML report instead of Markdown
./run_github_utils.sh list_forks --html

# Fork repositories using GitHub API
./run_github_utils.sh fork_repositories owner/repo
./run_github_utils.sh fork_repositories forks.json --dry-run

# Sync forks online (no cloning)
./run_github_utils.sh online_fork_sync
./run_github_utils.sh online_fork_sync --workers 8 --no-confirm
./run_github_utils.sh online_fork_sync --branch main --output-dir ~/reports

# Show help
./run_github_utils.sh help

Windows (Command Prompt)

REM List available utilities
run_github_utils.bat list

REM Run repository analyzer
run_github_utils.bat analyzer

REM Run fork updater
run_github_utils.bat fork_updater

REM Run fork updater with custom directory
run_github_utils.bat fork_updater --repos-dir "C:\Projects\my_forks"

REM Generate HTML repository report
run_github_utils.bat html_reporter

REM Generate HTML report with custom output directory
run_github_utils.bat html_reporter --output-dir "C:\Reports"

REM Search for popular repositories by topic
run_github_utils.bat topic_searcher --topic python

REM Search with custom parameters
run_github_utils.bat topic_searcher --topic machine-learning --limit 20 --sort-by forks

REM Search for top contributors by topic
run_github_utils.bat contributor_searcher --topic python

REM Analyze organization/user repositories
run_github_utils.bat org_analyzer --owner microsoft

REM Download missing forked repositories
run_github_utils.bat fork_downloader --repos-dir "C:\GitHub\Forked"

REM Download with SSH and custom directory
run_github_utils.bat fork_downloader --repos-dir "C:\Forks" --use-ssh

REM Clone all missing repositories
run_github_utils.bat repo_cloner

REM Clone only owned repositories with SSH
run_github_utils.bat repo_cloner --repo-type owned --use-ssh --repos-dir "C:\Projects"

REM Analyze starred repositories and followed contributors
run_github_utils.bat stars_followers

REM Generate stars & followers report with custom output directory
run_github_utils.bat stars_followers --output-dir "C:\Reports"

REM List all repositories from users you follow
run_github_utils.bat following_repos

REM Filter repositories by criteria
run_github_utils.bat following_repos --repo-type owned --min-stars 10

REM Find recent Python repositories from followed users
run_github_utils.bat following_repos --language Python --days-since-update 30

REM List all forked repositories
run_github_utils.bat list_forks

REM List forks with HTML report instead of Markdown
run_github_utils.bat list_forks --html

REM Fork repositories using GitHub API
run_github_utils.bat fork_repositories owner/repo
run_github_utils.bat fork_repositories forks.json --dry-run

REM Show help
run_github_utils.bat help

Windows (PowerShell)

# List available utilities
.\run_github_utils.ps1 list

# Run repository analyzer
.\run_github_utils.ps1 analyzer

# Run fork updater
.\run_github_utils.ps1 fork_updater

# Run fork updater with custom directory
.\run_github_utils.ps1 fork_updater -repos-dir "C:\Projects\my_forks"

# Generate HTML repository report
.\run_github_utils.ps1 html_reporter

# Generate HTML report with custom output directory
.\run_github_utils.ps1 html_reporter -output-dir "C:\Reports"

# Search for popular repositories by topic
.\run_github_utils.ps1 topic_searcher -topic python

# Search with custom parameters
.\run_github_utils.ps1 topic_searcher -topic machine-learning -limit 20 -sort-by forks

# Search for top contributors by topic
.\run_github_utils.ps1 contributor_searcher -topic python

# Analyze organization/user repositories
.\run_github_utils.ps1 org_analyzer -owner microsoft

# Download missing forked repositories
.\run_github_utils.ps1 fork_downloader -repos-dir "C:\GitHub\Forked"

# Download with SSH and custom directory
.\run_github_utils.ps1 fork_downloader -repos-dir "C:\Forks" -use-ssh

# Clone all missing repositories
.\run_github_utils.ps1 repo_cloner

# Clone only owned repositories with SSH
.\run_github_utils.ps1 repo_cloner -repo-type owned -use-ssh -repos-dir "C:\Projects"

# Analyze starred repositories and followed contributors
.\run_github_utils.ps1 stars_followers

# Generate stars & followers report with custom output directory
.\run_github_utils.ps1 stars_followers -output-dir "C:\Reports"

# List all repositories from users you follow
.\run_github_utils.ps1 following_repos

# Filter repositories by criteria
.\run_github_utils.ps1 following_repos -repo-type owned -min-stars 10

# Find recent Python repositories from followed users
.\run_github_utils.ps1 following_repos -language Python -days-since-update 30

# List all forked repositories
.\run_github_utils.ps1 list_forks

# List forks with HTML report instead of Markdown
.\run_github_utils.ps1 list_forks -html

# Fork repositories using GitHub API
.\run_github_utils.ps1 fork_repositories owner/repo
.\run_github_utils.ps1 fork_repositories forks.json -dry-run

# Show help
.\run_github_utils.ps1 help

Available Utilities

1. Repository Analyzer

  • Purpose: Analyze all your GitHub repositories
  • Output: Console report + JSON file with detailed data
  • Features:
    • Repository statistics (stars, forks, issues)
    • Language distribution
    • Repository status (private, archived, forked)
    • Export to timestamped JSON file

2. Fork Updater

  • Purpose: Sync all forked repositories with their upstream sources
  • Features:
    • Automatic upstream remote setup
    • Detects default branch (master/main)
    • Stashes local changes before merging
    • Bulk processing with progress reporting

3. Multi-Threaded Fork Updater ⚡

  • Purpose: High-performance concurrent syncing of all forked repositories
  • Features:
    • Multi-threaded processing: Process multiple repositories simultaneously
    • Configurable concurrency: 1-10 worker threads (default: 5)
    • Structured logging: Color-coded output with timestamps and thread IDs
    • Progress tracking: Real-time progress with success/error statistics
    • Rate limiting: Respects GitHub API limits
    • Thread-safe operations: Safe concurrent git operations
    • Verbose mode: Debug logging for troubleshooting
    • Performance metrics: Reports repositories per minute
    • Workflow scope handling: Graceful handling of workflow file restrictions

Performance: Typically 3-5x faster than sequential processing for large repository sets.

Usage Examples:

# Basic usage (5 workers)
./run_github_utils.sh fork_updater_multithread

# High performance (8 workers)
./run_github_utils.sh fork_updater_multithread --workers 8

# Use SSH remotes
./run_github_utils.sh fork_updater_multithread --use-ssh

# Verbose debugging
./run_github_utils.sh fork_updater_multithread --verbose

# Custom directory with specific worker count
./run_github_utils.sh fork_updater_multithread --repos-dir ~/my-forks --workers 6

# Disable colors (for logging to files)
./run_github_utils.sh fork_updater_multithread --no-color

4. HTML Repository Reporter

  • Purpose: Generate a comprehensive HTML report of all repositories
  • Output: Interactive HTML file + JSON data backup
  • Features:
    • Beautiful responsive web interface
    • Categorized view (owned vs forked repositories)
    • Repository cards with detailed information
    • User profile summary with statistics
    • Programming language distribution
    • Repository topics and README snippets
    • Parent repository information for forks
    • Repository badges (private, fork, archived)
    • Star ratings and activity metrics
    • Mobile-friendly responsive design

4. Topic Repository Searcher

  • Purpose: Search and discover popular repositories by topic
  • Output: Beautiful HTML report + JSON data backup
  • Features:
    • Search GitHub repositories by any topic
    • Sort by stars, forks, or last updated
    • Customizable result limits (default: top 10)
    • Comprehensive repository details including:
      • Repository ranking and statistics
      • Owner information with avatars
      • Programming language and topics
      • README previews
      • Homepage and clone links
      • License and metadata information
    • Professional gradient design with animations
    • Repository cards with hover effects
    • Statistics dashboard with totals
    • Mobile-responsive layout

5. Topic Contributor Searcher

  • Purpose: Discover and analyze top contributors by topic
  • Output: Beautiful HTML report + JSON data backup
  • Features:
    • Search GitHub contributors by any topic
    • Analyze contributor portfolios and contributions
    • Customizable result limits (default: top 10)
    • Minimum repository filter for quality contributors
    • Comprehensive contributor profiles including:
      • Contributor ranking and avatar
      • Bio, company, location, and contact info
      • Portfolio statistics (repositories, stars, forks, followers)
      • Programming languages used
      • Top repository showcase
      • Repository preview with links
      • Contribution scoring system
    • Professional gradient design with vibrant colors
    • Contributor cards with detailed metrics
    • Statistics dashboard with combined totals
    • Mobile-responsive layout

6. Organization/User Repository Analyzer ⭐

  • Purpose: Analyze all repositories for any GitHub organization or user, highlighting most starred and forked
  • Output: Beautiful HTML report + JSON data backup + console summary
  • Features:
    • Universal Analysis: Works with any GitHub organization or user (not just your own)
    • Repository Ranking: Automatic ranking by stars, forks, and watchers
    • Visual Highlights: Star and fork indicators in console output
    • Comprehensive Statistics: Total stars, forks, watchers across all repositories
    • Top Repository Lists: Dedicated sections for most starred and most forked repositories
    • Language Distribution: Analysis of programming languages used
    • Entity Information: Profile details, bio, location, company information
    • Repository Details: Full metadata including topics, licenses, creation dates
    • Fork Analysis: Parent repository information for forked repositories
    • Interactive HTML Report: Professional gradient design with responsive layout
    • Repository Cards: Detailed cards with all statistics and metadata
    • Search Functionality: Easy discovery of repositories within large organizations
    • Export Capabilities: JSON data export for further analysis

Perfect for:

  • Competitive analysis of organizations
  • Developer portfolio review
  • Open source project discovery
  • Repository benchmarking
  • Due diligence research

7. GitHub Fork Downloader 📥

  • Purpose: Download all forked repositories that are missing locally to a specified directory
  • Output: Cloned repositories + JSON report + console progress
  • Features:
    • Smart Detection: Scans local directory and compares with GitHub forks
    • Missing Repository Identification: Lists all forks not present locally
    • Batch Cloning: Downloads all missing repositories in sequence
    • Multiple Clone Methods: Supports both HTTPS and SSH cloning
    • Upstream Remote Setup: Automatically configures upstream remotes for forks
    • Progress Tracking: Real-time download progress with success/failure tracking
    • Dry Run Mode: Preview what would be downloaded without actually cloning
    • Detailed Reporting: JSON export with clone results and repository metadata
    • Error Handling: Robust error handling with timeout protection
    • Directory Management: Automatic directory creation and organization
    • Fork Analysis: Parent repository information and relationship tracking
    • Resume Capability: Skip already downloaded repositories on subsequent runs

Perfect for:

  • Setting up development environments
  • Backing up forked repositories
  • Syncing work across multiple machines
  • Repository collection management
  • Ensuring complete fork coverage

8. GitHub Repository Cloner 🏠

  • Purpose: List all your repositories, check which exist locally, and clone missing ones
  • Output: Cloned repositories + JSON report + console progress
  • Features:
    • Complete Repository Coverage: Lists ALL your repositories (owned + forked)
    • Intelligent Detection: Scans local directory and compares with GitHub repositories
    • Flexible Filtering: Clone owned only, forked only, or all repositories
    • Missing Repository Identification: Lists all repos not present locally
    • Batch Cloning: Downloads all missing repositories efficiently
    • Multiple Clone Methods: Supports both HTTPS and SSH cloning
    • Upstream Remote Setup: Automatically configures upstream remotes for forks
    • Progress Tracking: Real-time cloning progress with success/failure tracking
    • Dry Run Mode: Preview what would be cloned without actually cloning
    • Detailed Reporting: JSON export with clone results and repository metadata
    • Repository Type Filtering: Choose between owned, forked, or all repositories
    • Error Handling: Robust error handling with timeout protection
    • Resume Capability: Skip already cloned repositories on subsequent runs

Difference from Fork Downloader:

  • Fork Downloader: Only downloads forked repositories
  • Repository Cloner: Downloads ALL your repositories (owned + forked) with filtering options

Perfect for:

  • Complete local backup of all your repositories
  • Setting up development environments from scratch
  • Ensuring you have all your work locally
  • Selective cloning by repository type
  • Comprehensive repository synchronization

9. Online Fork Sync 🌐

  • Purpose: Sync your forked repositories with their upstream parents using GitHub’s API, without cloning locally
  • Output: JSON report with per-repository results and summary
  • Features:
    • Server-side merge via merge-upstream endpoint
    • Multi-threaded execution (1–10 workers)
    • Dry-run preview and confirmation prompts
    • Robust error handling and conflict detection
    • Conservatively rate-limited API calls

Usage Examples:

# Preview only
./run_github_utils.sh online_fork_sync --dry-run

# Run with 8 workers, skip confirmation
./run_github_utils.sh online_fork_sync --workers 8 --no-confirm

# Target a specific branch and custom output
./run_github_utils.sh online_fork_sync --branch main --output-dir ~/reports

10. Repository Visibility Updater 🔒

  • Purpose: Update repository visibility (private/public) for owned or forked repositories
  • Output: Console progress + JSON report
  • Features:
    • Bulk Visibility Updates: Change multiple repositories at once
    • Repository Type Filtering: Update owned, forked, or all repositories
    • Selective Updates: Target specific repositories by name
    • Dry Run Mode: Preview changes without actually updating
    • Safety Checks: Confirmation prompts for irreversible changes
    • Progress Tracking: Real-time update progress with success/failure tracking
    • Detailed Reporting: JSON export with update results and repository metadata
    • Error Handling: Robust error handling with rollback capabilities
    • Batch Processing: Efficient processing of large repository sets
    • Permission Validation: Ensures you have the necessary permissions

Perfect for:

  • Managing repository privacy across your portfolio
  • Bulk privacy updates for organizational compliance
  • Securing sensitive repositories
  • Making projects publicly available
  • Repository housekeeping and organization

10. Stars & Followers Analyzer ⭐

  • Purpose: List all starred repositories and followed contributors with detailed information
  • Output: Beautiful HTML report + JSON data backup + console listing
  • Features:
    • Complete Starred Repository Listing: All your starred repositories in organized format
    • Complete Followed Users Listing: All users you follow with their details
    • Individual Repository Details: Name, description, stars, language, topics, and owner info
    • Individual User Profiles: Bio, company, location, follower count, and top repositories
    • Sorted Listings: Alphabetically sorted for easy browsing
    • Interactive HTML Report: Professional gradient design with responsive layout
    • Repository Cards: Detailed cards with metadata, topics, and owner information
    • User Profiles: Complete user profiles with bio, company, location, and top repositories
    • Repository Insights: Fork relationships, license information, and activity metrics
    • Export Capabilities: JSON data export for further analysis
    • Mobile-Friendly: Responsive design works on all devices
    • Console Preview: Shows first 10 repositories and users in terminal output

Perfect for:

  • Getting a complete list of all your starred repositories
  • Reviewing all the developers and organizations you follow
  • Creating a comprehensive inventory of your GitHub interests
  • Browsing your starred repositories in a beautiful interface
  • Exporting your GitHub stars and follows for backup or analysis
  • Discovering repositories you starred but forgot about

11. Following Repositories Analyzer 📚

  • Purpose: List all repositories from users and organizations you follow with detailed information and filtering
  • Output: Beautiful HTML report + JSON data backup + console listing with filtering
  • Features:
    • Complete Repository Listing: All repositories from every user/organization you follow
    • Advanced Filtering: Filter by repository type, stars, language, and recent activity
    • User Organization: Groups repositories by their owners with user profiles
    • Individual Repository Details: Full metadata including description, stats, topics, and licenses
    • User Profiles: Complete information about each followed user/organization
    • Activity Filtering: Show only repositories updated within specified timeframe
    • Language Filtering: Focus on repositories in specific programming languages
    • Star Filtering: Filter by minimum star count to find popular repositories
    • Fork Handling: Choose to include or exclude forked repositories
    • Interactive HTML Report: Professional responsive design with user sections
    • Repository Cards: Detailed cards with all metadata and statistics
    • Export Capabilities: JSON data export for further analysis
    • Mobile-Friendly: Responsive design works on all devices
    • Console Preview: Shows summary and top repositories in terminal output

Perfect for:

  • Discovering new repositories from developers you follow
  • Keeping track of what your followed users are working on
  • Finding popular repositories in specific programming languages
  • Monitoring recent activity from followed users and organizations
  • Creating a comprehensive view of your GitHub network's projects
  • Research and discovery of trending repositories in your areas of interest

12. Forked Repositories Lister 🍴

  • Purpose: List all forked repositories with their parent information
  • Output: Markdown report (default) or HTML report + JSON data backup + console listing
  • Features:
    • Complete Forks Listing: All your forked repositories with parent details
    • Parent Repository Information: Full details about each parent repository
    • Dual Output Formats: Markdown (default) or HTML report generation
    • Repository Details: Names, descriptions, URLs, languages, topics, and licenses
    • Clone Information: Both HTTPS and SSH clone URLs for parent and fork
    • Repository Statistics: Stars, forks, watchers for both parent and fork
    • Ownership Information: Owner details with avatars and profile links
    • Status Badges: Private and archived repository indicators
    • Professional HTML Design: Gradient design with responsive layout (optional)
    • Markdown Format: Clean, readable text format for easy reference (default)
    • Export Capabilities: JSON data export for further analysis
    • Console Preview: Shows summary and list of forks in terminal output

Perfect for:

  • Getting a complete inventory of all your forked repositories
  • Quick reference to parent repository information
  • Finding clone URLs for parent repositories
  • Tracking which repositories you've forked
  • Creating documentation of your forked projects
  • Comparing your fork with the parent repository details

13. Repository Forker 🔱

  • Purpose: Fork GitHub repositories using the GitHub API without downloading them locally
  • Output: JSON report with fork results + console progress
  • Features:
    • Single Repository Forking: Fork individual repositories by URL or owner/repo format
    • Batch Forking: Fork multiple repositories from JSON or text files
    • Smart Parsing: Automatically detects and parses various input formats
    • JSON File Support: Process JSON files from tools like list_forks to fork parent repositories
    • Text File Support: Process plain text files with repository names/URLs (one per line)
    • Duplicate Detection: Automatically removes duplicate repositories from the list
    • Already Forked Check: Skips repositories you've already forked
    • Dry Run Mode: Preview what would be forked without actually forking
    • Detailed Logging: Real-time progress with success/failure tracking
    • Error Handling: Robust error handling with detailed error messages
    • JSON Export: Complete results exported to JSON for analysis
    • No Local Downloads: Uses GitHub API only, no local cloning required
    • Multiple Input Formats: Accepts owner/repo, URLs, JSON files, or text files
    • Workflow Integration: Perfect companion to list_forks tool

Perfect for:

  • Forking parent repositories from your existing forks
  • Batch forking multiple repositories for contribution
  • Creating a curated collection of repository forks
  • Forking repositories for study or reference
  • Setting up development environments with specific forks
  • Contributing to multiple open source projects
  • Creating backups through forks

File Structure

GitHub_Utilities/
├── github_master.py           # Master Python script
├── run_github_utils.sh        # Shell script for Linux/macOS/WSL
├── run_github_utils.bat       # Batch script for Windows Command Prompt
├── run_github_utils.ps1       # PowerShell script for Windows
├── install_windows.bat        # Windows installation script (Command Prompt)
├── install_windows.ps1        # Windows installation script (PowerShell)
├── requirements.txt           # Python dependencies
├── config.json               # Configuration file
├── .gitattributes            # Git line ending configuration
├── README.md                 # This file
├── repo_analyzer/
│   └── github_repository_analyzer.py
├── fork_updater/
│   └── github_forks_updater.py
├── fork_updater_multithread/
│   └── github_forks_updater_mt.py
├── html_reporter/
│   └── github_html_reporter.py
├── topic_searcher/
│   └── github_topic_searcher.py
├── contributor_searcher/
│   └── github_contributor_searcher.py
├── org_analyzer/
│   └── github_org_analyzer.py
├── fork_downloader/
│   └── github_fork_downloader.py
├── repo_cloner/
│   └── github_repo_cloner.py
├── visibility_updater/
│   └── github_visibility_updater.py
├── stars_followers/
│   └── github_stars_followers.py
├── following_repos/
│   └── github_following_repos.py
├── list_forks/
│   └── github_list_forks.py
├── fork_repositories/
│   └── github_fork_repositories.py
└── tests/
    ├── test_framework.py
    ├── test_repo_analyzer.py
    ├── test_fork_updater.py
    ├── test_html_reporter.py
    ├── test_topic_searcher.py
    ├── test_contributor_searcher.py
    ├── test_org_analyzer.py
    ├── test_fork_downloader.py
    ├── test_repo_cloner.py
    ├── test_visibility_updater.py
    ├── test_stars_followers.py
    ├── test_following_repos.py
    ├── test_list_forks.py
    ├── test_fork_repositories.py
    └── run_all_tests.py

Configuration

Edit config.json to customize:

{
  "github_token_env": "GITHUB_TOKEN",
  "github_username_env": "GITHUB_USERNAME",
  "default_output_dir": "./output",
  "default_repos_dir": "./repos",
  "settings": {
    "enable_colored_output": true,
    "verbose_logging": false,
    "auto_create_directories": true,
    "max_concurrent_operations": 5
  }
}

Security Features

  • Environment Variable Validation: Ensures required tokens are set
  • Virtual Environment Isolation: Scripts run in isolated Python environment
  • No Hardcoded Credentials: All authentication via environment variables
  • Safe Script Loading: Dynamic module loading with error handling

Adding New Utilities

  1. Create your utility script in a subdirectory
  2. Add configuration to config.json:
    "your_utility": {
      "name": "Your Utility Name",
      "description": "Description of what it does",
      "script_path": "your_dir/your_script.py",
      "function": "main_function_name"
    }
  3. Add handler method to GitHubUtilityManager class in github_master.py

Troubleshooting

Common Issues

  1. Missing Environment Variables:

    export GITHUB_TOKEN="your_token"
    export GITHUB_USERNAME="your_username"
  2. Permission Denied:

    chmod +x run_github_utils.sh
  3. Python/Git Not Found:

    • Install Python 3.7+
    • Install Git for fork functionality
  4. Clean Virtual Environment:

    ./run_github_utils.sh clean
    ./run_github_utils.sh setup

Getting Help

# Show main help
./run_github_utils.sh help

# Show utility help
./run_github_utils.sh --help

Examples

Analyze Repositories

./run_github_utils.sh analyzer

Output: Console report + github_repos_YYYYMMDD_HHMMSS.json

Update All Forks

./run_github_utils.sh fork_updater --repos-dir ~/Projects/forks

Generate HTML Repository Report

./run_github_utils.sh html_reporter

Output: Interactive HTML file + JSON backup in ./output/

Generate HTML Report with Custom Directory

./run_github_utils.sh html_reporter --output-dir ~/reports

Search Popular Repositories by Topic

./run_github_utils.sh topic_searcher --topic python

Output: Interactive HTML report in ./output/ with top 10 Python repositories

Advanced Topic Search

# Search for top 20 machine learning repositories sorted by forks
./run_github_utils.sh topic_searcher --topic machine-learning --limit 20 --sort-by forks

# Search for React repositories with custom output directory
./run_github_utils.sh topic_searcher --topic react --output-dir ~/topic_reports

Search Top Contributors by Topic

./run_github_utils.sh contributor_searcher --topic python

Output: Interactive HTML report in ./output/ with top 10 Python contributors

Advanced Contributor Search

# Search for top 15 machine learning contributors with minimum 5 repositories
./run_github_utils.sh contributor_searcher --topic machine-learning --limit 15 --min-repos 5

# Search for React contributors with custom output directory
./run_github_utils.sh contributor_searcher --topic react --output-dir ~/contributor_reports

Analyze Organization/User Repositories

./run_github_utils.sh org_analyzer microsoft

Output: Interactive HTML report + JSON backup + console summary with highlighted top repositories

Advanced Organization Analysis

# Analyze a specific user with custom output directory
./run_github_utils.sh org_analyzer torvalds --output-dir ~/analysis

# Analyze popular organizations
./run_github_utils.sh org_analyzer google
./run_github_utils.sh org_analyzer facebook
./run_github_utils.sh org_analyzer apache

# Analyze individual developers
./run_github_utils.sh org_analyzer gaearon
./run_github_utils.sh org_analyzer tj

Download Missing Forked Repositories

./run_github_utils.sh fork_downloader

Output: Cloned repositories in ./repos/ + JSON report in ./output/

Advanced Fork Download

# Download to custom directory using SSH
./run_github_utils.sh fork_downloader --repos-dir ~/my-forks --use-ssh

# Dry run to see what would be downloaded
./run_github_utils.sh fork_downloader --dry-run

# Download with custom output directory for reports
./run_github_utils.sh fork_downloader --repos-dir ~/forks --output-dir ~/reports

# Use SSH for cloning (requires SSH key setup)
./run_github_utils.sh fork_downloader --use-ssh

Clone All Missing Repositories

./run_github_utils.sh repo_cloner

Output: Cloned repositories in ./repos/ + JSON report in ./output/

Advanced Repository Cloning

# Clone only owned repositories (not forks)
./run_github_utils.sh repo_cloner --repo-type owned --repos-dir ~/my-projects

# Clone only forked repositories
./run_github_utils.sh repo_cloner --repo-type forked --repos-dir ~/forks

# Clone all repositories using SSH
./run_github_utils.sh repo_cloner --use-ssh --repos-dir ~/all-repos

# Dry run to see what would be cloned
./run_github_utils.sh repo_cloner --dry-run

# Clone owned repos with custom output directory for reports
./run_github_utils.sh repo_cloner --repo-type owned --repos-dir ~/projects --output-dir ~/reports

Update Repository Visibility

./run_github_utils.sh visibility_updater --repo-type owned --private --dry-run

Output: Console preview + JSON report in ./output/

Advanced Visibility Updates

# Make all owned repositories private
./run_github_utils.sh visibility_updater --repo-type owned --private

# Make all forked repositories public
./run_github_utils.sh visibility_updater --repo-type forked --public

# Update specific repositories
./run_github_utils.sh visibility_updater --repo-names "repo1,repo2,repo3" --private

# Dry run to preview changes
./run_github_utils.sh visibility_updater --repo-type all --public --dry-run

# Update with custom output directory for reports
./run_github_utils.sh visibility_updater --repo-type owned --private --output-dir ~/reports

List Forked Repositories

./run_github_utils.sh list_forks

Output: Markdown report + JSON data in ./output/ + console listing

Advanced Fork Listing

# Generate HTML report instead of Markdown
./run_github_utils.sh list_forks --html

# List forks with custom output directory
./run_github_utils.sh list_forks --output-dir ~/reports

# Generate HTML report with custom output directory
./run_github_utils.sh list_forks --html --output-dir ~/fork_reports

Fork Repositories

./run_github_utils.sh fork_repositories owner/repo

Output: JSON report with fork results in ./output/ + console progress

Advanced Repository Forking

# Fork a single repository by URL
./run_github_utils.sh fork_repositories https://github.com/torvalds/linux

# Fork multiple repositories from a JSON file (e.g., from list_forks)
./run_github_utils.sh fork_repositories output/github_forks_list_20241004.json

# Fork from a text file with repository names
./run_github_utils.sh fork_repositories repos.txt

# Dry run to preview what would be forked
./run_github_utils.sh fork_repositories repos.json --dry-run

# Fork with custom output directory
./run_github_utils.sh fork_repositories owner/repo --output-dir ~/reports

# Complete workflow: List forks, then fork their parents
./run_github_utils.sh list_forks
./run_github_utils.sh fork_repositories output/github_forks_list_*.json --dry-run
./run_github_utils.sh fork_repositories output/github_forks_list_*.json

License

This project is open source. Feel free to modify and extend for your needs.

About

A secure, scalable Python-based system for managing GitHub utility scripts with virtual environment isolation.

Resources

Stars

Watchers

Forks