Skip to content

Commit d1c43cd

Browse files
DesmonDayDesmonDay
committed
Merge branch 'develop' of https://github.com/PaddlePaddle/PaddleNLP into fix_sth
2 parents 696f305 + 0d05544 commit d1c43cd

File tree

184 files changed

+9695
-334
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

184 files changed

+9695
-334
lines changed

‎.copyright.hook‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ RE_SHEBANG = re.compile(r"^[ \t\v]*#[ \t]?\!")
7171
def _check_copyright(path):
7272
head=[]
7373
try:
74-
with open(path) as f:
74+
with open(path, encoding="utf-8") as f:
7575
head = [next(f) for x in range(4)]
7676
except StopIteration:
7777
pass

‎LICENSE‎

Lines changed: 0 additions & 203 deletions
This file was deleted.

‎docs/model_zoo/index.rst‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,8 @@ PaddleNLP的Transformer预训练模型包含从 `huggingface.co`_ 直接转换
8484
ChineseBert <transformers/ChineseBert/contents>
8585
ConvBert <transformers/ConvBert/contents>
8686
CTRL <transformers/CTRL/contents>
87+
Deberta <transformers/Deberta/contents>
88+
DebertaV2 <transformers/DebertaV2/contents>
8789
DistilBert <transformers/DistilBert/contents>
8890
ELECTRA <transformers/ELECTRA/contents>
8991
ERNIE <transformers/ERNIE/contents>
@@ -145,6 +147,10 @@ Transformer预训练模型适用任务汇总
145147
+--------------------+-------------------------+----------------------+--------------------+-----------------+-----------------+
146148
|CTRL_ | ✅ | ❌ | ❌ | ❌ | ❌ |
147149
+--------------------+-------------------------+----------------------+--------------------+-----------------+-----------------+
150+
|Deberta_ | ✅ | ✅ | ✅ | ❌ | ✅ |
151+
+--------------------+-------------------------+----------------------+--------------------+-----------------+-----------------+
152+
|DebertaV2_ | ✅ | ✅ | ✅ | ❌ | ✅ |
153+
+--------------------+-------------------------+----------------------+--------------------+-----------------+-----------------+
148154
|DistilBert_ | ✅ | ✅ | ✅ | ❌ | ❌ |
149155
+--------------------+-------------------------+----------------------+--------------------+-----------------+-----------------+
150156
|ELECTRA_ | ✅ | ✅ | ✅ | ❌ | ✅ |
@@ -220,6 +226,8 @@ Transformer预训练模型适用任务汇总
220226
.. _ChineseBert: https://arxiv.org/abs/2106.16038
221227
.. _ConvBert: https://arxiv.org/abs/2008.02496
222228
.. _CTRL: https://arxiv.org/abs/1909.05858
229+
.. _DeBERTa: https://arxiv.org/abs/2006.03654
230+
.. _DebertaV2: https://arxiv.org/abs/2111.09543
223231
.. _DistilBert: https://arxiv.org/abs/1910.01108
224232
.. _ELECTRA: https://arxiv.org/abs/2003.10555
225233
.. _ERNIE: https://arxiv.org/abs/1904.09223

‎docs/model_zoo/transformers/Deberta/contents.rst‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
2+
3+
------------------------------------
4+
Deberta模型汇总
5+
------------------------------------
6+
7+
8+
9+
下表汇总介绍了目前PaddleNLP支持的Deberta模型对应预训练权重。
10+
11+
+----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+
12+
| Pretrained Weight | Language | Details of the model |
13+
+==================================================================================+==============+==================================================================================+
14+
|``microsoft/deberta-base`` | English | 12-layer, 768-hidden, |
15+
| | | 12-heads, 100M parameters. |
16+
| | | It outperforms BERT and RoBERTa on majority of NLU tasks with 80GB training data.|
17+
+----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+

‎docs/model_zoo/transformers/DebertaV2/contents.rst‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
2+
3+
------------------------------------
4+
DebertaV2模型汇总
5+
------------------------------------
6+
7+
8+
9+
下表汇总介绍了目前PaddleNLP支持的DebertaV2模型对应预训练权重。
10+
11+
+----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+
12+
| Pretrained Weight | Language | Details of the model |
13+
+==================================================================================+==============+==================================================================================+
14+
|``deepset/deberta-v3-large-squad2`` | English | 24-layer, 1024-hidden, |
15+
| | | 16-heads, 304M parameters. |
16+
| | | The deberta-v3-large model fine-tuned using the SQuAD2.0 dataset. |
17+
+----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+
18+
|``microsoft/deberta-v2-xlarge`` | English | 24-layer, 1536-hidden, |
19+
| | | 24-heads, 900M parameters. |
20+
| | | The deberta-v2 model. |
21+
+----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+
22+
|``microsoft/deberta-v3-base`` | English | 12-layer, 768-hidden, |
23+
| | | 12-heads, 86M parameters. |
24+
| | | The deberta-v2 model. |
25+
+----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+
26+
|``microsoft/deberta-v3-large`` | English | 24-layer, 1024-hidden, |
27+
| | | 16-heads, 304M parameters. |
28+
| | | The deberta-v2 model. |
29+
+----------------------------------------------------------------------------------+--------------+----------------------------------------------------------------------------------+
30+

‎docs/source/paddlenlp.transformers.deberta.modeling.rst‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
modeling
2+
===========================================
3+
4+
.. automodule:: paddlenlp.transformers.deberta.modeling
5+
:members:
6+
:no-undoc-members:
7+
:show-inheritance:

‎docs/source/paddlenlp.transformers.deberta.rst‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
deberta
2+
========================================
3+
4+
.. automodule:: paddlenlp.transformers.deberta
5+
:members:
6+
:no-undoc-members:
7+
:show-inheritance:
8+
9+
10+
.. toctree::
11+
:maxdepth: 4
12+
13+
paddlenlp.transformers.deberta.modeling
14+
paddlenlp.transformers.deberta.tokenizer

‎docs/source/paddlenlp.transformers.deberta.tokenizer.rst‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
tokenizer
2+
=================================================
3+
4+
.. automodule:: paddlenlp.transformers.deberta.tokenizer
5+
:members:
6+
:no-undoc-members:
7+
:show-inheritance:

‎docs/source/paddlenlp.transformers.deberta_v2.modeling.rst‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
modeling
2+
===========================================
3+
4+
.. automodule:: paddlenlp.transformers.deberta_v2.modeling
5+
:members:
6+
:no-undoc-members:
7+
:show-inheritance:

0 commit comments

Comments
 (0)