中文羊驼大模型二期 v3.1

[ymcui]

本次更新推出长上下文版聊天/指令模型Chinese-Alpaca-2-7B-16K和Chinese-Alpaca-2-13B-16K，可直接应用于各类长文本下游任务。

🚀 Chinese-Alpaca-2-7B-16K、Chinese-Alpaca-2-13B-16K

• 与Chinese-LLaMA-2-16K类似，通过位置插值方法，在增量数据上进行训练，使模型获得长文本处理能力
• 长上下文版模型（Chinese-LLaMA-2-16K、Chinese-Alpaca-2-16K）支持16K，且均可通过NTK方法进一步扩展
• 在长文本评测LongBench（中文+代码任务）上，16K版模型相比标准版均有显著的性能提升，其中Chinese-Alpaca-2-7B-16K和Chinese-Alpaca-2-13B-16K分别获得28.3和29.5的平均分，在同类模型中具有一定优势
• Chinese-Alpaca-2-7B-16K、Chinese-Alpaca-2-13B-16K已加入到⚔️ 模型竞技场，欢迎体验评测

温馨提示

• transformers 4.31.0以上版本可直接加载16K模型；llama.cpp下则需要在推理时添加--rope-scale 4参数
• 使用长上下文版模型之前，务必参阅推理与部署表格，查看相关方法是否支持
• 建议：主要处理长文本任务时，使用16K版模型；其他场景建议使用标准版模型

其他更新

• 修复了训练脚本中gradient_checkpointing相关问题（Fix bug w.r.t. using gradient_checkpoint without tuning embed_tokens and Fix typo of template in langchain_qa.py #175）
• privateGPT：添加了对16K模型的支持（privateGPT: 16k long-context model support #195）
• LangChain, TGW, API：添加了对16K模型的支持（LangChain, TGW, API: Add support for 16k-context model #196）
• FAQ：添加了问题9、10、11（FAQ: Update FAQ 9 and 10 #197）
• LongBench：添加了推理脚本（Add LongBench script #198、Update Longbench script #217）

For English release note, please refer to Discussion.

---

This discussion was created from the release 中文羊驼大模型二期 v3.1.

[ymcui]

Release Note v3.1

This update introduces the long-context chat/command models Chinese-Alpaca-2-7B-16K and Chinese-Alpaca-2-13B-16K, which can be directly applied to various long-text downstream tasks.

🚀 Chinese-Alpaca-2-7B-16K & Chinese-Alpaca-2-13B-16K

• Similar to Chinese-LLaMA-2-16K, it uses the Position Interpolation Method to train on incremental data, enabling the model to handle long texts.
• The long-context models (Chinese-LLaMA-2-16K, Chinese-Alpaca-2-16K) support 16K and can be further expanded using the NTK method.
• On the LongBench evaluation (Chinese + code tasks), the 16K models show significant performance improvements compared to the standard versions. Specifically, Chinese-Alpaca-2-7B-16K and Chinese-Alpaca-2-13B-16K scored an average of 28.3 and 29.5 respectively, showing certain advantages among similar models.
• Chinese-Alpaca-2-7B-16K & Chinese-Alpaca-2-13B-16K have been added to the ⚔️ Model Arena. You are welcome to try and evaluate them.

Friendly Reminder

• Transformers version 4.31.0 and above can directly load the 16K models; for llama.cpp, you need to add the --rope-scale 4 parameter during inference.
• Before using the long-context models, please refer to the Inference and Deployment table to check if the relevant methods are supported.
• Recommendation: Use the 16K models primarily for long-text tasks; for other scenarios, it's recommended to use the standard models.

Other Updates

• Fixed gradient_checkpointing related issues in the training script (Fix bug w.r.t. using gradient_checkpoint without tuning embed_tokens and Fix typo of template in langchain_qa.py #175).
• privateGPT: Added support for the 16K models (privateGPT: 16k long-context model support #195).
• LangChain, TGW, API: Added support for the 16K models (LangChain, TGW, API: Add support for 16k-context model #196).
• FAQ: Added questions 9, 10, 11 (FAQ: Update FAQ 9 and 10 #197).
• LongBench: Added inference scripts (Add LongBench script #198, Update Longbench script #217).