This repository contains the official pytorch implementation of the CVPR2025 paper: "Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding".

We will update more detailed result (including dataset, training, verification) in the future

• 2025.02.27: Build project page
• 2025.03.03:Add code and .sh file
• 2025.03.04:Add Dataset
• 2025.03.04:Add detailed process description

conda env create --file Octopus.yaml

or

conda create Octopus python=3.10
conda activate Octopus
git clone https://github.com/LijunZhang01/Octopus
cd Octopus
pip install -r requirements.txt

• Please download and extract the MSCOCO 2014 dataset from this link to the ''dataset'' folder
• For AMBER evaluation, see this link,and then move to the ''data'' folder

The directory structure is as follows:

+ data
  -train.json
  -image(AMBER dataset image)
  -query
  -relation.json
  -safe_words.txt
  -annotations.json
  -metrics.txt
+ dataset
  -images(COCO dataset image)

About model Pre-trained checkpoints

• LLaVA-1.5: Download LLaVA-1.5 merged 7B
• InstructBLIP: Download InstructBLIP

• AMBER:

bash eval_bench/scripts/train_token_amber.sh

• Need to specify "json_path", "data_path","checkpoint_path","model_path"

• ps: Replace all "/home/zlj/Octopus" paths with your own directory paths

• AMBER:

bash eval_bench/scripts/eval_token_amber.sh

• Need to specify "json_path", "data_path","checkpoint_path","model_path"

• ps: Replace all "/home/zlj/Octopus" paths with your own directory paths

This codebase borrows from most notably VCD, AvisC, and LLaVA. Many thanks to the authors for generously sharing their codes!

If you find this repository helpful for your project, please consider citing our work :

@misc{suo2025octopusalleviatinghallucinationdynamic,
      title={Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding}, 
      author={Wei Suo and Lijun Zhang and Mengyang Sun and Lin Yuanbo Wu and Peng Wang and Yanning Zhang},
      year={2025},
      eprint={2503.00361},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2503.00361}, 
}