Finegrained CLIPScore to detect object hallucination
This repository contains the source code for the following research paper: Vision-Encoders (Already) Know What They See: Mitigating Object Hallucination via Simple Fine-Grained CLIPScore
pip install -r requirements.txt
python -m spacy download en_core_web_smFor vanila CLIPScore
python eval_ohd-caps.py --score_type clipFor F-CLIPScore
python eval_ohd-caps.py --score_type fclipamcf. Further constrained version of F-CLIPScore is implemented in scorers.py
@misc{oh2025visionencodersalreadyknowsee,
title={Vision-Encoders (Already) Know What They See: Mitigating Object Hallucination via Simple Fine-Grained CLIPScore},
author={Hongseok Oh and Wonseok Hwang},
year={2025},
eprint={2502.20034},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2502.20034},
}