The repository follows the directory structure below:

Treble-Counterfactual-VLMs/
├── experiments/
├── lavis/
├── llava/
├── POPE/
└── README.md

Before running the project, please download the required dependencies:

• LAVIS: Clone from LAVIS and place it in the lavis/ directory.
• LLaVA: Clone from LLaVA and place it in the llava/ directory.
• POPE: Clone from POPE and place it in the POPE/ directory.
• Download the COCO train2014 and val2014 datasets, and place them in the following directories:

experiments/data/
            ├── train2014/
            └── val2014/

• experiments/test/ contains the test files for LLaVA and InstructBlip.
• experiments/eval/ contains the GPT evaluation code for assessing the MMHal-Bench performance.
• experiments/Treble_Counterfactual_utils/ contains the utils for implementing treble counterfactual test-time intervention