A modern, high-performance file hashing utility with parallel processing, multiple algorithms, and beautiful progress bars.
FileHasher generates cryptographic hashes for files in directories, allowing you to:
- Compare directory contents across different machines or backups
- Find duplicate files within or across directories
- Detect file changes and corruption
- Synchronize directories with automated scripts
- Parallel processing with multiple worker threads
- Individual progress bars for each worker
- Multi-core optimization for modern CPUs
- Real-time progress with Rich terminal UI
- MD5 (default, backward compatible)
- SHA1, SHA256, SHA512
- Blake2B, Blake2S
- Algorithm tracking in hash files
- Mismatch warnings when switching algorithms
- Performance comparison of all algorithms
- Throughput measurements (MB/s)
- Algorithm recommendations based on your hardware
- Rich progress bars with colors and animations
- Verbose output showing current files being processed
- Configuration file support (
.filehasher.ini) - Comprehensive help and examples
pip install filehasherRequirements:
- Python 3.6+
- tqdm (for fallback progress bars)
- rich (for enhanced UI)
- Generate hashes for current directory:
filehasher --generate- Compare directories:
# Generate hashes for source
filehasher --generate .source-hashes
# Generate hashes for destination
filehasher --generate .dest-hashes
# Compare and generate sync script
filehasher --compare .dest-hashes .source-hashes- Edit and run the generated script:
# Edit filehasher_script.sh to customize actions
vim filehasher_script.sh
# Run the synchronization script
bash filehasher_script.sh# Use 4 workers with individual progress bars
filehasher --generate --parallel --workers 4
# Verbose output showing current files
filehasher --generate --parallel --verbose --algorithm sha256Visual Output:
Generating sha256 hashes...
Worker 1 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% (153/153) current_file.jpg
Worker 2 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% (153/153) another_file.txt
Worker 3 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% (153/153) processing_file.dat
Worker 4 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% (152/152) final_file.py
# Use specific algorithm
filehasher --generate --algorithm sha256
# Benchmark all algorithms on your hardware
filehasher --benchmark
# Benchmark with custom test file
filehasher --benchmark --benchmark-file large_video.mp4Benchmark Results:
Benchmark Results:
--------------------------------------------------------------------------------
Algorithm Avg Time (s) Min Time (s) Max Time (s) Throughput (MB/s)
--------------------------------------------------------------------------------
sha256 0.0045 0.0044 0.0047 2199.27
sha1 0.0047 0.0047 0.0048 2106.49
sha512 0.0075 0.0075 0.0076 1329.89
blake2b 0.0146 0.0146 0.0147 682.83
md5 0.0160 0.0153 0.0163 626.82
blake2s 0.0247 0.0246 0.0248 405.60
Create .filehasher.ini in your project directory:
[filehasher]
default_algorithm = sha256
benchmark_iterations = 5
quiet = falseUsage: filehasher [OPTIONS] [HASHFILE]
File Hasher - Generate and compare file hashes with multiple algorithms.
Options:
-g, --generate Generate hashes (remove hashfile if exists)
-a, --append Append hashes to hashfile
-u, --update Update hashfile (clean old entries and append new)
-A, --algorithm Hash algorithm (default: md5)
-b, --benchmark Benchmark all supported hash algorithms
-P, --parallel Process files in parallel using multiple workers
--workers WORKERS Number of parallel workers (default: CPU count)
-v, --verbose Show detailed progress including filenames
-q, --quiet Suppress progress output
-c, --compare Compare hashes from hashfiles
-h, --help Show this help message
Supported algorithms: md5, sha1, sha256, sha512, blake2b, blake2s
- Setup source directory:
cd /path/to/source
filehasher --generate --algorithm sha256 .source-hashes- Setup destination directory:
cd /path/to/destination
filehasher --generate --algorithm sha256 .dest-hashes- Compare and generate sync script:
filehasher --compare .dest-hashes .source-hashes- Review and execute:
# Check what will be done
cat filehasher_script.sh
# Edit if needed
vim filehasher_script.sh
# Execute synchronization
bash filehasher_script.sh# Generate/update hashes
filehasher --update
# Find duplicates within directory
filehasher --compare .hashes .hashesThe generated script will show:
- Files with identical content (duplicates)
- Commands to remove unwanted duplicates
| Algorithm | Security | Speed | Use Case |
|---|---|---|---|
| MD5 | 🚀 Fastest | Legacy compatibility | |
| SHA1 | ⚡ Fast | Quick integrity checks | |
| SHA256 | ✅ Strong | ⚡ Fast | General purpose, recommended |
| SHA512 | ✅ Very Strong | 🐌 Slower | High-security requirements |
| Blake2B | ✅ Very Strong | ⚡ Fast | Modern, high-performance |
| Blake2S | ✅ Strong | ⚡ Fast | 32-bit systems, embedded |
When switching algorithms, FileHasher warns you:
⚠️ WARNING: Algorithm mismatch detected!
Existing hash file uses: md5
Requested algorithm: sha256
This may cause inconsistent hash comparisons.
Continue anyway? (y/N):
- Multi-core CPUs: Near-linear speedup with worker count
- Large directories: Significantly faster processing
- Progress visibility: Individual worker progress tracking
- Resource efficient: Optimal load balancing
Default settings can be customized in .filehasher.ini:
[filehasher]
default_algorithm = sha256
benchmark_iterations = 3
quiet = falseFileHasher is open source and welcomes contributions!
MIT License - see LICENSE file for details.
FileHasher - Modern file integrity and synchronization made beautiful.