Skip to content

Commit 05ae377

Browse files
idontkonwheridontkonwher
authored
[METAX] Support llama for MX C550 (#9186)
* [METAX] Support llama for MX C550 * fix typo, add paddlenlp branch, add greedy strategy for demo * add MXMACA software stack acquisition path
1 parent c9d5673 commit 05ae377

File tree

1 file changed

+156
-0
lines changed

1 file changed

+156
-0
lines changed

‎llm/metax/llama/README.md‎

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
## 🚣‍♂️ 使用PaddleNLP在MX C550下跑通llama2-13b模型 🚣
2+
3+
PaddleNLP在曦云®C550（[了解沐曦](https://www.metax-tech.com/)）上对llama2-13B模型进行了深度适配和优化，该套件实现了曦云C550和GPU的训推入口完全统一，达到了『无缝切换』的效果。
4+
曦云 C500 系列 GPU 是沐曦基于自主知识产权 GPU IP 打造的旗舰系列产品，具有强大的多精度混合算力，64GB 大容量高带宽内存，以及先进的多卡互联 MetaLink 技术。它搭载MXMACA®软件栈，全面兼容主流 GPU 生态，应用迁移零成本，
5+
可方便快捷地支撑智算、通用计算和数据处理等应用场景。
6+
7+
## 🚀 快速开始 🚀
8+
9+
### （0）在开始之前，您需要有一台插有曦云C550机器，对此机器的系统要求如下：
10+
11+
| 芯片类型 | vbios版本 | MXMACA版本 |
12+
| -------- | --------- | --------------- |
13+
| 曦云C550 | ≥ 1.13 | ≥ 2.23.0.1018 |
14+
15+
**注：如果需要验证您的机器是否插有曦云C550 GPU，只需系统环境下输入以下命令，看是否有输出：**
16+
17+
```
18+
mx-smi
19+
20+
#输出如下
21+
mx-smi version: 2.1.6
22+
23+
=================== MetaX System Management Interface Log ===================
24+
Timestamp : Mon Sep 23 06:24:52 2024
25+
26+
Attached GPUs : 8
27+
+---------------------------------------------------------------------------------+
28+
| MX-SMI 2.1.6 Kernel Mode Driver Version: 2.5.014 |
29+
| MACA Version: 2.23.0.1018 BIOS Version: 1.13.4.0 |
30+
|------------------------------------+---------------------+----------------------+
31+
| GPU NAME | Bus-id | GPU-Util |
32+
| Temp Power | Memory-Usage | |
33+
|====================================+=====================+======================|
34+
| 0 MXC550 | 0000:2a:00.0 | 0% |
35+
| 31C 44W | 810/65536 MiB | |
36+
+------------------------------------+---------------------+----------------------+
37+
| 1 MXC550 | 0000:3a:00.0 | 0% |
38+
| 31C 46W | 810/65536 MiB | |
39+
+------------------------------------+---------------------+----------------------+
40+
| 2 MXC550 | 0000:4c:00.0 | 0% |
41+
| 31C 47W | 810/65536 MiB | |
42+
+------------------------------------+---------------------+----------------------+
43+
| 3 MXC550 | 0000:5c:00.0 | 0% |
44+
| 31C 46W | 810/65536 MiB | |
45+
+------------------------------------+---------------------+----------------------+
46+
| 4 MXC550 | 0000:aa:00.0 | 0% |
47+
| 30C 46W | 810/65536 MiB | |
48+
+------------------------------------+---------------------+----------------------+
49+
| 5 MXC550 | 0000:ba:00.0 | 0% |
50+
| 31C 47W | 810/65536 MiB | |
51+
+------------------------------------+---------------------+----------------------+
52+
| 6 MXC550 | 0000:ca:00.0 | 0% |
53+
| 30C 46W | 810/65536 MiB | |
54+
+------------------------------------+---------------------+----------------------+
55+
| 7 MXC550 | 0000:da:00.0 | 0% |
56+
| 30C 47W | 810/65536 MiB | |
57+
+------------------------------------+---------------------+----------------------+
58+
59+
+---------------------------------------------------------------------------------+
60+
| Process: |
61+
| GPU PID Process Name GPU Memory |
62+
| Usage(MiB) |
63+
|=================================================================================|
64+
| no process found |
65+
+---------------------------------------------------------------------------------+
66+
```
67+
68+
### （1）环境准备：(这将花费您5~55min时间)
69+
70+
1. 使用容器构建运行环境（可选）
71+
72+
```
73+
# 您可以使用 --device=/dev/dri/card0 指定仅GPU 0在容器内可见（其它卡同理），--device=/dev/dri 表示所有GPU可见
74+
docker run -it --rm --device=/dev/dri
75+
--device=/dev/mxcd --group-add video -network=host --uts=host --ipc=host --privileged=true --shm-size 128g registry.baidubce.com/paddlepaddle/paddle:2.6.1-gpu-cuda11.7-cudnn8.4-trt8.4
76+
```
77+
78+
2. 安装MXMACA软件栈
79+
80+
> 您可以联系 [email protected] 以获取MXMACA安装包及技术支持， 已授权用户可以访问[沐曦软件中心](https://sw-download.metax-tech.com/login)获取相关安装包。
81+
>
82+
83+
```
84+
# 假设您已下载并解压好MXMACA驱动
85+
sudo bash /path/to/maca_package/mxmaca-sdk-install.sh
86+
```
87+
88+
3. 安装PaddlePaddle
89+
90+
①如果您已经通过Metax获取了 PaddlePaddle安装包，您可以直接进行安装：
91+
92+
`pip install paddlepaddle_gpu-2.6.0+mc*.whl`
93+
94+
②您也可以通过源码自行编译PaddlePaddle安装包，请确保您已经正确安装MXMACA软件栈。编译过程使用了基于MXMACA的cu-bridge编译工具，您可以访问[文档](https://gitee.com/p4ul/cu-bridge/tree/master/docs/02_User_Manual)获取更多信息。
95+
96+
```
97+
98+
# 1. 访问 PaddlePaddle github仓库clone代码并切换至mxmaca分支.
99+
git clone https://github.com/PaddlePaddle/Paddle.git
100+
git checkout release-mxmaca/2.6
101+
# 2. 拉取第三方依赖
102+
git submodule update --init
103+
# 3. 配置环境变量
104+
export MACA_PATH=/real/maca/install/path
105+
export CUDA_PATH=/real/cuda/install/path
106+
export CUCC_PATH=${MACA_PATH}/tools/cu-bridge
107+
export PATH=${CUDA_PATH}/bin:${CUCC_PATH}/bin:${CUCC_PATH}/tools:${MACA_PATH}/bin:$PATH
108+
export LD_LIBRARY_PATH=${MACA_PATH}/lib:${MACA_PATH}/mxgpu_llvm/lib:${LD_LIBRARY_PATH}
109+
# 4. 检查配置是否正确
110+
cucc --version
111+
# 5. 执行编译
112+
makdir -p build && cd build
113+
cmake_maca .. -DPY_VERSION=3.8 -DWITH_GPU=ON -DWITH_DISTRIBUTE=ON -DWITH_NCCL=ON
114+
make_maca -j64
115+
# 6. 等待编译完成后安装whl包
116+
pip install python/dist/paddlepaddle_gpu*.whl
117+
```
118+
119+
4. 克隆PaddleNLP仓库代码，并安装依赖
120+
121+
```
122+
# PaddleNLP是基于paddlepaddle『飞桨』的自然语言处理和大语言模型(LLM)开发库，存放了基于『飞桨』框架实现的各种大模型，llama2-13B模型也包含其中。为了便于您更好地使用PaddleNLP，您需要clone整个仓库。
123+
git clone https://github.com/PaddlePaddle/PaddleNLP.git
124+
cd PaddleNLP
125+
git checkout origin/release/3.0-beta1
126+
python -m pip install -r requirements.txt
127+
python -m pip install -e .
128+
```
129+
130+
### （2）推理：(这将花费您5~10min时间)
131+
132+
1. 尝试运行推理demo
133+
134+
```
135+
cd llm/predict
136+
python predictor.py --model_name_or_path meta-llama/Llama-2-13b-chat --dtype bfloat16 --output_file "infer.json" --batch_size 1 --decode_strategy "greedy_search"
137+
```
138+
139+
成功运行后，可以查看到推理结果的生成，样例如下：
140+
141+
```
142+
***********Source**********
143+
解释一下温故而知新
144+
***********Target**********
145+
146+
***********Output**********
147+
"温故而知新" (wēn gù er zhī xīn) is a Chinese idiom that means "to know the old and appreciate the new." It is often used to describe the idea that one can gain a deeper understanding and appreciation of something by studying its history and traditions, and then applying that knowledge to new situations and challenges.
148+
149+
The word "温" (wēn) in this idiom means "old" or "ancient," and "故" (gù) means "former" or "past." The word "知" (zhī) means "to know" or "to understand," and "新" (xīn) means "new."
150+
151+
This idiom is often used in the context of education, where it is believed that students should be taught the traditional methods and theories of a subject before being introduced to new and innovative ideas. By understanding the history and foundations of a subject, students can better appreciate and apply the new ideas and techniques that they are learning.
152+
153+
In addition to education, "温故而知新" can also be applied to other areas of life, such as business, where it is important to understand the traditions and practices of the industry before introducing new products or services. By understanding the past and the foundations of a particular field, one can gain a deeper appreciation of the present and make more informed decisions about the future.
154+
```
155+
156+
2. 您也可以尝试参考 [文档](../../../legacy/examples/benchmark/wiki_lambada/README.md) 中的说明使用 wikitext 数据集验证推理精度。

0 commit comments

Comments
 (0)