Skip to content

案例符号回归 Transformer4SR在Nvidia A100精度结果与官方差距大 #1131

@yuchunliu-metax

Description

@yuchunliu-metax

bug 描述 bug description

简述:
案例transformer4sr 在A100平台验证srsd-feynman_easy模式精度指标与官方差距大
1)A100 精度结果 Hit rate: 0/30 ,与官网精度指标8/30差距较大
2)公式预测结果与官方差距大
A100:

The ground truth is: 25*x1 + x2*log(x1)
The prediction is: -C*x1 + 1/log(x2)

官方:

The ground truth is: 25*x1 + x2*log(x1)
The prediction is: -C*x1 + x2*log(x1)

软件版本:paddle:2.6.2 , PaddleScience:commit ab417c8
测试案例:PaddleScience/examples/transformer4sr
测试环境:NVIDIA A100-PCIE-40GB

问题复现:
测试命令:
1) 训练:python transformer4sr.py
2) 评估:python transformer4sr.py mode=eval EVAL.pretrained_model_path=./best_model.pdparams

日志:
eval.log
train.log
best_model.zip

精度验证:
1、A100 精度结果:

=> Mean ZSS distance: 0.677 +- 0.230
=> Hit rate: 0/30
target C*x1*x2 pred: C/(x1 + x2 + x3**2)**2
target C*x1/x2**2 pred: C*x1*log(sin(1/x2)**2)
target C*x1*x2 pred: C*x1*sin(x2)**2
target C*x1*x2 pred: C*x1*sin(x2)**2
target C*x1*x2**2 pred: 1/(x1**2*cos(exp(-C*x2)))
target C*x1*x2*sin(x3) pred: C*x2**2*x3*log(x1)
target C*x1*x2*x3*sin(x4) pred: C*x1*x2*log(x3)
target C*x1/x2 pred: C*x1*log(cos(x1))
target C*sin(x1)/sin(x2) pred: x2 + exp(-C*x2 + x1)
target C*x1*x3/(x1*x2 + x3) pred: sin(C*x2**2)**2/x1**2
The ground truth is: 25*x1 + x2*log(x1)
The prediction is: -C*x1 + 1/log(x2)

2、官方 精度结果:

Mean ZSS distance(srsd-feynman_easy): 0.658 +- 0.390
Hit rate(srsd-feynman_easy): 8/30

Image

其他补充信息 Additional Supplementary Information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions