Skip to content

Commit 9989d60

Browse files
authored
[FastDeploy] Add ernie-m fastdeploy example (PaddlePaddle#4891)
* Add ernie-m doc * Add inference scripts * Add README * update readme * remove predictor * Add fastdeploy test * update requirements * Add extra index url * remove extra * Add seq cls infer * add extra-index-url * Add comment * remove install fastdeploy * Update option * Update doc
1 parent ce18df0 commit 9989d60

File tree

12 files changed

+327
-529
lines changed

12 files changed

+327
-529
lines changed

model_zoo/ernie-3.0/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1584,7 +1584,7 @@ AutoTokenizer.from_pretrained("ernie-3.0-medium-zh", use_fast=True)
15841584

15851585
</div>
15861586

1587-
目前ERNIE 3.0模型已提供基于FastDeploy的部署示例,支持在多款硬件(CPU、GPU、昆仑芯、华为昇腾以及Graphcore IPU)以及推理引擎后端进行部署。具体的适配的硬件以及推理引擎请参考:[FastDeploy 部署指南](./deploy/README.md)
1587+
目前 ERNIE 3.0 模型已提供基于 FastDeploy 的部署示例,支持在多款硬件(CPU、GPU、昆仑芯、华为昇腾以及 Graphcore IPU)以及推理引擎后端进行部署。具体的适配的硬件以及推理引擎请参考:[FastDeploy 部署指南](./deploy/README.md)
15881588

15891589
<a name="Python部署"></a>
15901590

model_zoo/ernie-m/README.md

+27-4
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,8 @@
1010
* [单机多卡](#单机多卡)
1111
* [预测评估](#预测评估)
1212
* [部署](#部署)
13-
* [Python部署](#Python部署)
13+
* [FastDeploy 部署](#FastDeploy部署)
14+
* [Python 部署](#Python部署)
1415
* [服务化部署](#服务化部署)
1516
* [参考论文](#参考论文)
1617

@@ -198,15 +199,37 @@ python run_classifier.py \
198199
| VECO Large | 88.9 | 82.4 | 86.0 | 84.7 | 85.3 | 86.2 | **85.8** | 80.1 | 83.0 | 77.2 | 80.9 | 82.8 | 75.3 | **83.1** | **83.0** | 83.0 |
199200
| **ERNIE-M Large** | **89.5** | **86.5** | **86.9** | **86.1** | **86.0** | **86.8** | 84.1 | **83.8** | **84.1** | **84.5** | **82.1** | 83.5 | 81.1 | 79.4 | 77.9 | **84.2** |
200201

202+
<a name="部署"></a>
203+
201204
## 部署
202205

203-
### Python部署
206+
我们基于 FastDeploy 为 ERNIE-M 提供了多种部署方案,可以满足不同场景下的部署需求,请根据实际情况进行选择。
207+
208+
<a name="FastDeploy部署"></a>
209+
210+
### FastDeploy 部署
211+
212+
⚡️[FastDeploy](https://github.com/PaddlePaddle/FastDeploy)是一款全场景、易用灵活、极致高效的AI推理部署工具,为开发者提供多硬件、多推理引擎后端的部署能力。开发者只需调用一行代码即可随意切换硬件、推理引擎后端。
213+
214+
<div align="center">
215+
216+
<img src="https://user-images.githubusercontent.com/54695910/213087724-7175953a-0e07-4af8-a4a1-5304163da2e0.png" >
217+
218+
</div>
219+
220+
目前 ERNIE-M 模型已提供基于 FastDeploy 的部署示例,支持在多款硬件(CPU、GPU、昆仑芯、华为昇腾以及 Graphcore IPU)以及推理引擎后端进行部署。
221+
222+
<a name="Python部署"></a>
223+
224+
#### Python 部署
225+
226+
Python 部署请参考:[Python 部署指南](./deploy/python/README.md)
204227

205-
Python部署请参考:[Python 部署指南](./deploy/predictor/README.md)
228+
<a name="服务化部署"></a>
206229

207230
### 服务化部署
208231

209-
* [PaddleNLp SimpleServing 服务化部署指南](./deploy/simple_serving/README.md)
232+
* [PaddleNLP SimpleServing 服务化部署指南](./deploy/simple_serving/README.md)
210233

211234

212235
## 参考论文

model_zoo/ernie-m/deploy/predictor/README.md

-91
This file was deleted.

0 commit comments

Comments
 (0)