The Uni-Dock-Benchmarks repository provides a comprehensive collection of datasets for benchmarking the Uni-Dock docking system's performance and accuracy. The datasets include prepared structures and input files for both Uni-Dock V1 and V2 for benchmarks.
Benchmark data within the repository is categorized into two primary sections:
molecular_dockingvirtual_screening
Under the molecular_docking directory, you will find several well-known benchmark datasets:
Astex: Hartshorn, M. J., Verdonk, M. L., Chessari, G., Brewerton, S. C., Mooij, W. T., Mortenson, P. N., & Murray, C. W. (2007). Diverse, high-quality test set for the validation of protein− ligand docking performance. Journal of medicinal chemistry, 50(4), 726-741.CASF2016: Su, M., Yang, Q., Du, Y., Feng, G., Liu, Z., Li, Y., & Wang, R. (2018). Comparative assessment of scoring functions: the CASF-2016 update. Journal of chemical information and modeling, 59(2), 895-913.PoseBuster: Buttenschoen, M., Morris, G. M., & Deane, C. M. (2024). PoseBusters: AI-based docking methods fail to generate physically valid poses or generalise to novel sequences. Chemical Science.
We performed the following preparation steps for the proteins and ligands in the datasets.
- After obtaining the protein structures from the RCSB database based on the PDB code, we retained the crystal waters that affect the binding mode and completed missing protein side chains and lost hydrogen atoms.
- For ligands, we searched the RCSB database for the isomer SMILES corresponding to the PDB code and determined the correct protonation state according to the receptor pocket environment. Then, we generated 3D conformations for each ligand.
After excluding systems for covalent ligand bindings, problematic binding mechanisms and those with large natural products or polypeptide ligands, 69 systems from Astex, 271 systems from CASF-2016 and 396 systems from PoseBuster were used as benchmarks.
The correctness of protein side chain structure and hydrogen bond networks have crucial impact on ligand docking, and hence the structure preparation for both protein and ligand determines the difficultness of producing correct ligand docking poses. We use our internal tools to prepare the initial structures of receptor and ligands so that we can obtain better docking results. In addition, we also integrated the open-sourced version of structure preparation algorithms for Uni-Dock V2 into the unified protocol in the Uni-Dock V2 github repository.
We prepare the receptor structure in two versions, protein with co-crystallized water version and protein only version, to test the overall effect of the presence of water on ligand docking experiments.
The directory structure for each dataset is as follows:
<DataSetName>
├── <PDB_ID>
│ ├── <PDB_ID>_ligand.sdf # Ligand co-crystal structure processed in SDF format
│ ├── <PDB_ID>_protein_water_cleaned.pdb # Prepared receptor structure with protein and crystallized water in PDB format
│ ├── <PDB_ID>_protein_cleaned.pdb # Prepared receptor structure with only protein in PDB format
│ ├── ligand_prepared.sdf # Reprepared ligand 3D conformation used in docking test in SDF format
│ ├── unidock1_protein # Folder for input files of Uni-Dock V1, with protein only in the receptor structure
│ │ ├── ligand_prepared_torsion_tree.sdf # Prepared ligand structure with torsion tree information used in Uni-Dock V1 input in SDF format
│ │ └── receptor.pdbqt # Prepared receptor structure used in Uni-Dock V1 input in PDBQT format
│ ├── unidock1_protein_water # Folder for input files of Uni-Dock V1, with protein and water in the receptor structure
│ │ ├── ligand_prepared_torsion_tree.sdf # Prepared ligand structure with torsion tree information used in Uni-Dock V1 input in SDF format
│ │ └── receptor.pdbqt # Prepared receptor structure used in Uni-Dock V1 input in PDBQT format
│ ├── unidock2_protein # Folder for input files of Uni-Dock V2, with protein only in the receptor structure
│ │ ├── <PDB_ID>_unidock2.json # Integrated JSON input file for Uni-Dock V2 docking engine
│ │ └── receptor_parameterized.dms # Prepared and parameterized receptor structure in DMS format
│ └── unidock2_protein_water # Folder for input files of Uni-Dock V2, with protein and water in the receptor structure
│ ├── <PDB_ID>_unidock2.json # Integrated JSON input file for Uni-Dock V2 docking engine
│ └── receptor_parameterized.dms # Prepared and parameterized receptor structure in DMS format
└── pdb_center.csv # CSV file recording the protein pocket center with respect to the <PDB_ID> for each system
Under the virtual_screening directory, you will find several meticulously selected benchmark datasets:
D4: Lyu, J., Wang, S., Balius, T. E., Singh, I., Levit, A., Moroz, Y. S., ... & Irwin, J. J. (2019). Ultra-large library docking for discovering new chemotypes. Nature, 566(7743), 224-229.GBA: Tran-Nguyen, V. K., Jacquemard, C., & Rognan, D. (2020). LIT-PCBA: an unbiased data set for machine learning and virtual screening. Journal of chemical information and modeling, 60(9), 4263-4273.NSP3: Schuller, M., Correy, G. J., Gahbauer, S., Fearon, D., Wu, T., Díaz, R. E., ... & Ahel, I. (2021). Fragment binding to the Nsp3 macrodomain of SARS-CoV-2 identified through crystallographic screening and computational docking. Science advances, 7(16), eabf8711.PPARG: Tran-Nguyen, V. K., Jacquemard, C., & Rognan, D. (2020). LIT-PCBA: an unbiased data set for machine learning and virtual screening. Journal of chemical information and modeling, 60(9), 4263-4273.sigma2: Alon, A., Lyu, J., Braz, J. M., Tummino, T. A., Craik, V., O’Meara, M. J., ... & Kruse, A. C. (2021). Structures of the σ2 receptor enable docking for bioactive ligand discovery. Nature, 600(7890), 759-764.
The following table summarizes the statistics of the datasets:
| Dataset | PDB ID | N_Actives | N_Inactives | N_Total |
|---|---|---|---|---|
| D4 | 5WIU | 226 | 598 | 824 |
| GBA | 5LVX | 286 | 458,205 | 458,491 |
| NSP3 | 5RS7 | 65 | 3,515 | 3,580 |
| PPARG | 5Y2T | 29 | 7,292 | 7,321 |
| sigma2 | 7M94 | 228 | 596 | 824 |
The directory structure for each dataset is as follows:
<DataSetName>
├── docking_grid.json # JSON file recording the protein pocket center and the box sizes
├── <PDB_ID>_receptor.pdb # Original unprocessed receptor structure in PDB format
├── <PDB_ID>_protein_cleaned.pdb # Prepared receptor structure with only protein in PDB format
├── actives_cleaned.sdf # Preprocessed and cleaned active molecules in SDF format
├── actives.sdf # Active molecules in SDF format
├── inactives_cleaned.sdf # Preprocessed and cleaned inactive molecules in SDF format
├── inactives.sdf # Inactive molecules in SDF format
├── unidock1_protein # Folder for input files of Uni-Dock V1, with protein only in the receptor structure
│ ├── actives_prepared_torsion_tree.sdf # Prepared active molecule structure with torsion tree information used in Uni-Dock V1 input in SDF format
│ ├── inactives_prepared_torsion_tree.sdf # Prepared inactive molecule structure with torsion tree information used in Uni-Dock V1 input in SDF format
│ └── receptor.pdbqt # Prepared receptor structure used in Uni-Dock V1 input in PDBQT format
└── unidock2_protein # Folder for input files of Uni-Dock V2, with protein only in the receptor structure
├── actives_unidock2.json # Integrated JSON input file of active molecules for Uni-Dock V2 docking engine
├── inactives_unidock2.json # Integrated JSON input file of inactive molecules for Uni-Dock V2 docking engine
└── receptor_parameterized.dms # Prepared and parameterized receptor structure in DMS format
Important Note
Due to the substantial number of inactive molecules in the GBA dataset, the directory contains several large files that exceed GitHub's size limits. These files have been moved to cloud storage. To obtain the complete GBA directory, please run the following command in your terminal:
./getGBA.shTwo scripts are provided to run benchmark tests on Uni-Dock executable binaries:
The main entry script for running a single benchmark test. It supports both Uni-Dock V1 and V2, and can run either molecular docking or virtual screening benchmarks.
Molecular Docking:
# Uni-Dock V2 with receptor without water
python scripts/run_test.py --version 2 --bin ud2 --type molecular_docking --nowater --device 1 --savedir my_res --seed 121
# Uni-Dock V2 with receptor containing water (default)
python scripts/run_test.py --version 2 --bin ud2 --type molecular_docking --device 1 --savedir my_res --seed 121
# Uni-Dock V1 with receptor containing water
python scripts/run_test.py --version 1 --bin ud1 --type molecular_docking --device 1 --savedir my_res --seed 121Virtual Screening:
python scripts/run_test.py --version 2 --bin ud2 --type virtual_screening --device 0 --savedir res_vs --seed 122--savedir <DIR>(required) - Output directory for the results--bin <PATH>(required) - Path to the Uni-Dock executable binary--version <1|2>(required) - Uni-Dock version (1 or 2)NOTE: Specifying
--version 2automatically loads the filescripts/ud2.yamlas default configuration.--type <molecular_docking|virtual_screening>(required) - Type of benchmark test--device <ID>(optional, default: 0) - GPU device ID--seed <INTEGER>(optional, default: 123) - Random seed for reproducibility--nowater(optional) - Use receptor without water (only for molecular_docking). Default: uses water-containing receptor--rootdir <PATH>(optional) - Root directory of the data (Uni-Dock-Benchmarks dir). If not provided, the script should be run from the Uni-Dock-Benchmarks directory
The script automatically submits 3 benchmark tasks, each using different GPU devices and seeds, which you can then average for more reliable results. All tasks run in the background (nohup) and will not block.
Check detailed help information by running the script.
# Basic usage with default values
bash scripts/submit_udbench.sh res2_0.4.4.1_dock 0 1 2
# Specify all parameters
bash scripts/submit_udbench.sh res2_0.4.4.1_dock 0 1 2 --bin ud2_0.4.4.1 --version 2 --type molecular_docking
# Use --nowater parameter
bash scripts/submit_udbench.sh res2_0.4.4.1_dock 0 1 2 --bin ud2_0.4.4.1 --version 2 --type molecular_docking --nowater
# Virtual screening task
bash scripts/submit_udbench.sh res_vs 0 1 2 --bin ud2_v0.2 --version 2 --type virtual_screeningAfter submission, you can manage tasks using the recorded PID files.