Skip to content

Could I arise one PR to fix MRR metric statics method ???  #119

@Yazooliu

Description

@Yazooliu

I saw
https://github.com/PaddlePaddle/RocketQA/blob/main/research/DuReader-Retrieval-Baseline/metric/evaluation.py

line 135 is wrong.
百度MRR代码问题截图

for MRR 定义:
MRR定义

计算分母时,分母表示的是问题对应的推理答案长度。

而您的代码在计算时分母是ground_truth的长度。实际应用中ground_truth的长度和推理结果的长度不一定相同,所以您的代码有问题,请求提交PR修改,请批准。

祝好
Yazooliu

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions