Code Repository for Challenging the Evaluator: LLM Sycophancy Under User Rebuttal
images/: Contains plots and figures used in the report.llm_refutation_judge/: Contains results of the LLM refutation judge.llm_refutations/: Contains pairs of disagreeing LLM responses.llm_responses/: Contains first pass LLM responses to multiple choice questions.mcq_data/: Includes sampled multiple choice questions from different datasets.prompt_templates/: Contains some prompt templates used for generating LLM responses in different stages.refutation_responses/: Contains LLM responses to the refutation prompts.src/: Contains the code used to generate the report.statistics_results/: Contains statistics and results derived fromrefutation_responses/.all_mcqs.json: Contains all the multiple choice questions used in the project.requirements.txt: Contains the required packages for the project.README.md: This file.