Skip to content

Conversation

@From00
Copy link
Collaborator

@From00 From00 commented Mar 1, 2024

PR types

Performance optimization

PR changes

Models

Description

自动并行Llama模型支持SwiGLU,以优化性能和显存。
Llama1-13B-PP2-SD4-GBS128-ACC32配置,单机A100提速0.9%,相关数据:
image

关联PR: #7967

遗留TODO: 补齐框架SwiGLU算子切分推导规则(测试配置未开mp,走兜底规则)

@paddle-bot
Copy link

paddle-bot bot commented Mar 1, 2024

Thanks for your contribution!

@codecov
Copy link

codecov bot commented Mar 1, 2024

Codecov Report

Attention: Patch coverage is 40.00000% with 12 lines in your changes are missing coverage. Please review.

Project coverage is 56.42%. Comparing base (dff7dcc) to head (1b113be).

Files Patch % Lines
paddlenlp/transformers/llama/modeling_auto.py 40.00% 6 Missing ⚠️
...ddlenlp/transformers/llama/modeling_auto_static.py 40.00% 6 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #8038   +/-   ##
========================================
  Coverage    56.42%   56.42%           
========================================
  Files          589      589           
  Lines        90252    90266   +14     
========================================
+ Hits         50921    50929    +8     
- Misses       39331    39337    +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@zhiqiu zhiqiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wawltor wawltor merged commit a574900 into PaddlePaddle:develop Mar 2, 2024
heavyrain-lzy added a commit to heavyrain-lzy/PaddleNLP that referenced this pull request Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants