Hyperparameter of the number of distractor documents and the ratio of golden documents in training RAFT

[yifan1130]

Hi, I just wonder what the number of distractor documents and the ratio of golden documents in training RAFT are in producing the results in the paper. I saw the default ratio in the script the ratio is 1.0, which means no distractor documents are used. I wonder whether the default number in the scripts represents the hyperparameter used to produce the results in the paper?

[kaiwen129]

Hi, the optimal value for the P% hyperparameter may vary from dataset to dataset. To answer your question, the paper used various values of P%, not just the default value of 1.0 in the script.