|
1 |
| -English | [简体中文](README_CN.md) |
2 |
| - |
3 |
| -# PaddleClas Model Deployment |
4 |
| - |
5 |
| -## Model Description |
6 |
| - |
7 |
| -- [PaddleClas Release/2.4](https://github.com/PaddlePaddle/PaddleClas/tree/release/2.4) |
8 |
| - |
9 |
| -Now FastDeploy supports the deployment of the following models |
10 |
| - |
11 |
| -- [PP-LCNet Models](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/PP-LCNet.md) |
12 |
| -- [PP-LCNetV2 Models](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/PP-LCNetV2.md) |
13 |
| -- [EfficientNet Models](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/EfficientNet_and_ResNeXt101_wsl.md) |
14 |
| -- [GhostNet Models](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Mobile.md) |
15 |
| -- [MobileNet Models(including v1,v2,v3)](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Mobile.md) |
16 |
| -- [ShuffleNet Models](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Mobile.md) |
17 |
| -- [SqueezeNet Models](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Others.md) |
18 |
| -- [Inception Models](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/Inception.md) |
19 |
| -- [PP-HGNet Models](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/PP-HGNet.md) |
20 |
| -- [ResNet Models(including vd series)](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/ResNet_and_vd.md) |
21 |
| - |
22 |
| -## Prepare PaddleClas Deployment Model |
23 |
| - |
24 |
| -For PaddleClas model export, refer to [Model Export](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/inference_deployment/export_model.md#2-%E5%88%86%E7%B1%BB%E6%A8%A1%E5%9E%8B%E5%AF%BC%E5%87%BA). |
25 |
| - |
26 |
| -Attention:The model exported by PaddleClas contains two files, including `inference.pdmodel` and `inference.pdiparams`. However, it is necessary to prepare the generic [inference_cls.yaml](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/deploy/configs/inference_cls.yaml) file provided by PaddleClas to meet the requirements of deployment. FastDeploy will obtain from the yaml file the preprocessing information required during inference. FastDeploy will get the preprocessing information needed by the model from the yaml file. Developers can directly download this file. But they need to modify the configuration parameters in the yaml file based on personalized needs. Refer to the configuration information in the infer section of the PaddleClas model training [config.](https://github.com/PaddlePaddle/PaddleClas/tree/release/2.4/ppcls/configs/ImageNet) |
27 |
| - |
28 |
| - |
29 |
| -## Download Pre-trained Model |
30 |
| - |
31 |
| -For developers' testing, some models exported by PaddleClas (including the inference_cls.yaml file) are provided below. Developers can download them directly. |
32 |
| - |
33 |
| -| Model | Parameter File Size |Input Shape | Top1 | Top5 | |
34 |
| -|:---------------------------------------------------------------- |:----- |:----- | :----- | :----- | |
35 |
| -| [PPLCNet_x1_0](https://bj.bcebos.com/paddlehub/fastdeploy/PPLCNet_x1_0_infer.tgz) | 12MB | 224x224 |71.32% | 90.03% | |
36 |
| -| [PPLCNetV2_base](https://bj.bcebos.com/paddlehub/fastdeploy/PPLCNetV2_base_infer.tgz) | 26MB | 224x224 |77.04% | 93.27% | |
37 |
| -| [EfficientNetB7](https://bj.bcebos.com/paddlehub/fastdeploy/EfficientNetB7_infer.tgz) | 255MB | 600x600 | 84.3% | 96.9% | |
38 |
| -| [EfficientNetB0](https://bj.bcebos.com/paddlehub/fastdeploy/EfficientNetB0_infer.tgz)| 19MB | 224x224 | 77.38% | 93.31% | |
39 |
| -| [EfficientNetB0_small](https://bj.bcebos.com/paddlehub/fastdeploy/EfficientNetB0_small_infer.tgz)| 18MB | 224x224 | 75.8% | 92.58% | |
40 |
| -| [GhostNet_x1_3](https://bj.bcebos.com/paddlehub/fastdeploy/GhostNet_x1_3_infer.tgz) | 27MB | 224x224 | 75.79% | 92.54% | |
41 |
| -| [GhostNet_x1_3_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/GhostNet_x1_3_ssld_infer.tgz) | 29MB | 224x224 | 79.3% | 94.49% | |
42 |
| -| [GhostNet_x0_5](https://bj.bcebos.com/paddlehub/fastdeploy/GhostNet_x0_5_infer.tgz) | 10MB | 224x224 | 66.8% | 86.9% | |
43 |
| -| [MobileNetV1_x0_25](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV1_x0_25_infer.tgz) | 1.9MB | 224x224 | 51.4% | 75.5% | |
44 |
| -| [MobileNetV1_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV1_ssld_infer.tgz) | 17MB | 224x224 | 77.9% | 93.9% | |
45 |
| -| [MobileNetV2_x0_25](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV2_x0_25_infer.tgz) | 5.9MB | 224x224 | 53.2% | 76.5% | |
46 |
| -| [MobileNetV2](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV2_infer.tgz) | 13MB | 224x224 | 72.15% | 90.65% | |
47 |
| -| [MobileNetV2_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV2_ssld_infer.tgz) | 14MB | 224x224 | 76.74% | 93.39% | |
48 |
| -| [MobileNetV3_small_x1_0](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV3_small_x1_0_infer.tgz) | 11MB | 224x224 | 68.24% | 88.06% | |
49 |
| -| [MobileNetV3_small_x0_35_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV3_small_x0_35_ssld_infer.tgz) | 6.4MB | 224x224 | 55.55% | 77.71% | |
50 |
| -| [MobileNetV3_large_x1_0_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/MobileNetV3_large_x1_0_ssld_infer.tgz) | 22MB | 224x224 | 78.96% | 94.48% | |
51 |
| -| [ShuffleNetV2_x0_25](https://bj.bcebos.com/paddlehub/fastdeploy/ShuffleNetV2_x0_25_infer.tgz) | 2.4MB | 224x224 | 49.9% | 73.79% | |
52 |
| -| [ShuffleNetV2_x2_0](https://bj.bcebos.com/paddlehub/fastdeploy/ShuffleNetV2_x2_0_infer.tgz) | 29MB | 224x224 | 73.15% | 91.2% | |
53 |
| -| [SqueezeNet1_1](https://bj.bcebos.com/paddlehub/fastdeploy/SqueezeNet1_1_infer.tgz) | 4.8MB | 224x224 | 60.1% | 81.9% | |
54 |
| -| [InceptionV3](https://bj.bcebos.com/paddlehub/fastdeploy/InceptionV3_infer.tgz) | 92MB | 299x299 | 79.14% | 94.59% | |
55 |
| -| [PPHGNet_tiny_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/PPHGNet_tiny_ssld_infer.tgz) | 57MB | 224x224 | 81.95% | 96.12% | |
56 |
| -| [PPHGNet_small](https://bj.bcebos.com/paddlehub/fastdeploy/PPHGNet_small_infer.tgz) | 87MB | 224x224 | 81.51% | 95.82% | |
57 |
| -| [PPHGNet_base_ssld](https://bj.bcebos.com/paddlehub/fastdeploy/PPHGNet_base_ssld_infer.tgz) | 274MB | 224x224 | 85.0% | 97.35% | |
58 |
| -| [ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz) | 98MB | 224x224 | 79.12% | 94.44% | |
59 |
| -| [ResNet50](https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_infer.tgz) | 91MB | 224x224 | 76.5% | 93% | |
60 |
| -| [ResNeXt50_32x4d](https://bj.bcebos.com/paddlehub/fastdeploy/ResNeXt50_32x4d_infer.tgz) | 89MB | 224x224 | 77.75% | 93.82% | |
61 |
| -| [DenseNet121](https://bj.bcebos.com/paddlehub/fastdeploy/DenseNet121_infer.tgz) | 29MB | 224x224 | 75.66% | 92.58% | |
62 |
| -| [PULC_person_exists](https://bj.bcebos.com/paddlehub/fastdeploy/person_exists_infer.tgz) | 6MB | 224x224 | | | |
63 |
| -| [ViT_large_patch16_224](https://bj.bcebos.com/paddlehub/fastdeploy/ViT_large_patch16_224_infer.tgz) | 1.1GB | 224x224 | 83.23% | 96.50%| |
64 |
| - |
65 |
| -## Detailed Deployment Documents |
66 |
| - |
67 |
| -- [Python Deployment](python) |
68 |
| -- [C++ Deployment](cpp) |
69 |
| -- [Serving Deployment](serving) |
| 1 | +# PaddleClas高性能全场景模型部署方案—FastDeploy |
| 2 | + |
| 3 | +## 目录 |
| 4 | +- [FastDeploy介绍](#FastDeploy介绍) |
| 5 | +- [图像分类模型部署](#图像分类模型部署) |
| 6 | +- [常见问题](#常见问题) |
| 7 | + |
| 8 | +## 1. FastDeploy介绍 |
| 9 | +<div id="FastDeploy介绍"></div> |
| 10 | + |
| 11 | +**[⚡️FastDeploy](https://github.com/PaddlePaddle/FastDeploy)**是一款**全场景**、**易用灵活**、**极致高效**的AI推理部署工具,支持**云边端**部署。使用FastDeploy可以简单高效的在X86 CPU、NVIDIA GPU、飞腾CPU、ARM CPU、Intel GPU、Graphcore IPU、昆仑、昇腾、瑞芯微、晶晨、算能等10+款硬件上对PaddleClas模型进行快速部署,并且支持Paddle Inference、Paddle Lite、TensorRT、OpenVINO、ONNXRuntime、RKNPU2、SOPHGO等多种推理后端。 |
| 12 | + |
| 13 | +<div align="center"> |
| 14 | + |
| 15 | +<img src="https://user-images.githubusercontent.com/31974251/224941235-d5ea4ed0-7626-4c62-8bbd-8e4fad1e72ad.png" > |
| 16 | + |
| 17 | +</div> |
| 18 | + |
| 19 | +## 2. 图像分类模型部署 |
| 20 | +<div id="图像分类模型部署"></div> |
| 21 | + |
| 22 | +### 2.1 硬件支持列表 |
| 23 | + |
| 24 | +|硬件类型|该硬件是否支持|使用指南|Python|C++| |
| 25 | +|:---:|:---:|:---:|:---:|:---:| |
| 26 | +|X86 CPU|✅|[链接](./cpu-gpu)|✅|✅| |
| 27 | +|NVIDIA GPU|✅|[链接](./cpu-gpu)|✅|✅| |
| 28 | +|飞腾CPU|✅|[链接](./cpu-gpu)|✅|✅| |
| 29 | +|ARM CPU|✅|[链接](./cpu-gpu)|✅|✅| |
| 30 | +|Intel GPU(集成显卡)|✅|[链接](./cpu-gpu)|✅|✅| |
| 31 | +|Intel GPU(独立显卡)|✅|[链接](./cpu-gpu)|✅|✅| |
| 32 | +|Graphcore IPU|✅|[链接](./graphcore)|✅|✅| |
| 33 | +|昆仑|✅|[链接](./kunlunxin)|✅|✅| |
| 34 | +|昇腾|✅|[链接](./ascend)|✅|✅| |
| 35 | +|瑞芯微|✅|[链接](./rockchip)|✅|✅| |
| 36 | +|晶晨|✅|[链接](./amlogic)|--|✅|✅| |
| 37 | +|算能|✅|[链接](./sophgo)|✅|✅| |
| 38 | + |
| 39 | + |
| 40 | +### 2.2. 详细使用文档 |
| 41 | +- X86 CPU |
| 42 | + - [部署模型准备](./cpu-gpu) |
| 43 | + - [Python部署示例](./cpu-gpu/python/) |
| 44 | + - [C++部署示例](./cpu-gpu/cpp/) |
| 45 | +- NVIDIA GPU |
| 46 | + - [部署模型准备](./cpu-gpu) |
| 47 | + - [Python部署示例](./cpu-gpu/python/) |
| 48 | + - [C++部署示例](./cpu-gpu/cpp/) |
| 49 | +- 飞腾CPU |
| 50 | + - [部署模型准备](./cpu-gpu) |
| 51 | + - [Python部署示例](./cpu-gpu/python/) |
| 52 | + - [C++部署示例](./cpu-gpu/cpp/) |
| 53 | +- ARM CPU |
| 54 | + - [部署模型准备](./cpu-gpu) |
| 55 | + - [Python部署示例](./cpu-gpu/python/) |
| 56 | + - [C++部署示例](./cpu-gpu/cpp/) |
| 57 | +- Intel GPU |
| 58 | + - [部署模型准备](./cpu-gpu) |
| 59 | + - [Python部署示例](./cpu-gpu/python/) |
| 60 | + - [C++部署示例](./cpu-gpu/cpp/) |
| 61 | +- Graphcore IPU |
| 62 | + - [部署模型准备](./graphcore) |
| 63 | + - [Python部署示例](./graphcore/python/) |
| 64 | + - [C++部署示例](./graphcore/cpp/) |
| 65 | +- 昆仑 XPU |
| 66 | + - [部署模型准备](./kunlunxin) |
| 67 | + - [Python部署示例](./kunlunxin/python/) |
| 68 | + - [C++部署示例](./kunlunxin/cpp/) |
| 69 | +- 昇腾 Ascend |
| 70 | + - [部署模型准备](./ascend) |
| 71 | + - [Python部署示例](./ascend/python/) |
| 72 | + - [C++部署示例](./ascend/cpp/) |
| 73 | +- 瑞芯微 Rockchip |
| 74 | + - [部署模型准备](./rockchip/) |
| 75 | + - [Python部署示例](./rockchip/rknpu2/) |
| 76 | + - [C++部署示例](./rockchip/rknpu2/) |
| 77 | +- 晶晨 Amlogic |
| 78 | + - [部署模型准备](./amlogic/a311d/) |
| 79 | + - [C++部署示例](./amlogic/a311d/cpp/) |
| 80 | +- 算能 Sophgo |
| 81 | + - [部署模型准备](./sophgo/) |
| 82 | + - [Python部署示例](./sophgo/python/) |
| 83 | + - [C++部署示例](./sophgo/cpp/) |
| 84 | + |
| 85 | +### 2.3 更多部署方式 |
| 86 | + |
| 87 | +- [Android ARM CPU部署](./android) |
| 88 | +- [服务化Serving部署](./serving) |
| 89 | +- [web部署](./web) |
| 90 | +- [模型自动化压缩工具](./quantize) |
| 91 | + |
| 92 | + |
| 93 | +## 4. 常见问题 |
| 94 | +<div id="常见问题"></div> |
| 95 | + |
| 96 | +遇到问题可查看常见问题集合,搜索FastDeploy issue,*或给FastDeploy提交[issue](https://github.com/PaddlePaddle/FastDeploy/issues)*: |
| 97 | + |
| 98 | +[常见问题集合](https://github.com/PaddlePaddle/FastDeploy/tree/develop/docs/cn/faq) |
| 99 | +[FastDeploy issues](https://github.com/PaddlePaddle/FastDeploy/issues) |
0 commit comments