-
Notifications
You must be signed in to change notification settings - Fork 4.8k
Closed
Description
I'd like to use vicuna_bench. After python gen_api_answer.py --bench-name vicuna_bench --model gpt-3.5-turbo succeeds, python gen_judgment.py --bench-name vicuna_bench --model-list gpt-3.5-turbo failed with
88%|█████████████████████████████████████████████████████████████████████████████████████████▎ | 70/80 [17:27<02:29, 14.97s/it]
Traceback (most recent call last):
File ".../FastChat/fastchat/llm_judge/gen_judgment.py", line 309, in <module>
play_a_match_func(match, output_file=output_file)
File ".../FastChat/fastchat/llm_judge/common.py", line 203, in play_a_match_single
score, user_prompt, judgment = run_judge_single(
File ".../FastChat/fastchat/llm_judge/common.py", line 141, in run_judge_single
kwargs["ref_answer_2"] = ref_answer["choices"][0]["turns"][1]
IndexError: list index out of range
Metadata
Metadata
Assignees
Labels
No labels