[Other] PPOCR models support model clone function (PaddlePaddle#1072)

* Refactor PaddleSeg with preprocessor && postprocessor

* Fix bugs

* Delete redundancy code

* Modify by comments

* Refactor according to comments

* Add batch evaluation

* Add single test script

* Add ppliteseg single test script && fix eval(raise) error

* fix bug

* Fix evaluation segmentation.py batch predict

* Fix segmentation evaluation bug

* Fix evaluation segmentation bugs

* Update segmentation result docs

* Update old predict api and DisableNormalizeAndPermute

* Update resize segmentation label map with cv::INTER_NEAREST

* Add Model Clone function for PaddleClas && PaddleDet && PaddleSeg

* Add multi thread demo

* Add python model clone function

* Add multi thread python && C++ example

* Fix bug

* Update python && cpp multi_thread examples

* Add cpp && python directory

* Add README.md for examples

* Delete redundant code

* Create README_CN.md

* Rename README_CN.md to README.md

* Update README.md

* Update README.md

* Update VERSION_NUMBER

* Update requirements.txt

* Update README.md

* update version in doc:

* [Serving]Update Dockerfile (PaddlePaddle#1037)

Update Dockerfile

* Add license notice for RVM onnx model file (PaddlePaddle#1060)

* [Model] Add GPL-3.0 license (PaddlePaddle#1065)

Add GPL-3.0 license

* PPOCR model support model clone

* Update README.md

* Update PPOCRv2 && PPOCRv3 clone code

* Update PPOCR python __init__

* Add multi thread ocr example code

* Update README.md

* Update README.md

* Update ResNet50_vd_infer multi process code

* Add PPOCR multi process && thread example

* Update README.md

* Update README.md

* Update multi-thread docs

Co-authored-by: Jason <[email protected]>
Co-authored-by: leiqing <[email protected]>
Co-authored-by: heliqi <[email protected]>
Co-authored-by: WJJ1995 <[email protected]>

Author: 5 people

examples/vision/matting/rvm/README.md

@@ -17,10 +17,10 @@ For developers' testing, models exported by RobustVideoMatting are provided belo
 
 | Model                                                               | Parameter Size    | Accuracy    | Note |
 |:---------------------------------------------------------------- |:----- |:----- | :------ |
-| [rvm_mobilenetv3_fp32.onnx](https://bj.bcebos.com/paddlehub/fastdeploy/rvm_mobilenetv3_fp32.onnx) | 15MB | - |
-| [rvm_resnet50_fp32.onnx](https://bj.bcebos.com/paddlehub/fastdeploy/rvm_resnet50_fp32.onnx) | 103MB | - |
-| [rvm_mobilenetv3_trt.onnx](https://bj.bcebos.com/paddlehub/fastdeploy/rvm_mobilenetv3_trt.onnx) | 15MB | - |
-| [rvm_resnet50_trt.onnx](https://bj.bcebos.com/paddlehub/fastdeploy/rvm_resnet50_trt.onnx) | 103MB | - |
+| [rvm_mobilenetv3_fp32.onnx](https://bj.bcebos.com/paddlehub/fastdeploy/rvm_mobilenetv3_fp32.onnx) | 15MB ||exported from [RobustVideoMatting](https://github.com/PeterL1n/RobustVideoMatting/commit/81a1093)，GPL-3.0 License |
+| [rvm_resnet50_fp32.onnx](https://bj.bcebos.com/paddlehub/fastdeploy/rvm_resnet50_fp32.onnx) | 103MB | |exported from [RobustVideoMatting](https://github.com/PeterL1n/RobustVideoMatting/commit/81a1093)，GPL-3.0 License |
+| [rvm_mobilenetv3_trt.onnx](https://bj.bcebos.com/paddlehub/fastdeploy/rvm_mobilenetv3_trt.onnx) | 15MB | |exported from [RobustVideoMatting](https://github.com/PeterL1n/RobustVideoMatting/commit/81a1093)，GPL-3.0 License |
+| [rvm_resnet50_trt.onnx](https://bj.bcebos.com/paddlehub/fastdeploy/rvm_resnet50_trt.onnx) | 103MB | |exported from [RobustVideoMatting](https://github.com/PeterL1n/RobustVideoMatting/commit/81a1093)，GPL-3.0 License |
 
 **Note**：
 - If you want to use TensorRT for inference, download onnx model file with the trt suffix is necessary.

fastdeploy/vision/ocr/ppocr/classifier.cc

@@ -53,6 +53,12 @@ bool Classifier::Initialize() {
   return true;
 }
 
+std::unique_ptr<Classifier> Classifier::Clone() const {
+  std::unique_ptr<Classifier> clone_model = utils::make_unique<Classifier>(Classifier(*this));
+  clone_model->SetRuntime(clone_model->CloneRuntime());
+  return clone_model;
+}
+
 bool Classifier::Predict(const cv::Mat& img, int32_t* cls_label, float* cls_score) {
   std::vector<int32_t> cls_labels(1);
   std::vector<float> cls_scores(1);

fastdeploy/vision/ocr/ppocr/classifier.h

@@ -19,6 +19,7 @@
 #include "fastdeploy/vision/ocr/ppocr/utils/ocr_postprocess_op.h"
 #include "fastdeploy/vision/ocr/ppocr/cls_postprocessor.h"
 #include "fastdeploy/vision/ocr/ppocr/cls_preprocessor.h"
+#include "fastdeploy/utils/unique_ptr.h"
 
 namespace fastdeploy {
 namespace vision {
@@ -41,6 +42,13 @@ class FASTDEPLOY_DECL Classifier : public FastDeployModel {
   Classifier(const std::string& model_file, const std::string& params_file = "",
              const RuntimeOption& custom_option = RuntimeOption(),
              const ModelFormat& model_format = ModelFormat::PADDLE);
+  
+  /** \brief Clone a new Classifier with less memory usage when multiple instances of the same model are created
+   *
+   * \return new Classifier* type unique pointer
+   */
+  virtual std::unique_ptr<Classifier> Clone() const;
+
   /// Get model's name
   std::string ModelName() const { return "ppocr/ocr_cls"; }
 
@@ -53,6 +61,7 @@ class FASTDEPLOY_DECL Classifier : public FastDeployModel {
    */
   virtual bool Predict(const cv::Mat& img,
                       int32_t* cls_label, float* cls_score);
+  
   /** \brief BatchPredict the input image and get OCR classification model cls_result.
    *
    * \param[in] images The list of input image data, comes from cv::imread(), is a 3-D array with layout HWC, BGR format.

fastdeploy/vision/ocr/ppocr/dbdetector.cc

@@ -53,6 +53,12 @@ bool DBDetector::Initialize() {
   return true;
 }
 
+std::unique_ptr<DBDetector> DBDetector::Clone() const {
+  std::unique_ptr<DBDetector> clone_model = utils::make_unique<DBDetector>(DBDetector(*this));
+  clone_model->SetRuntime(clone_model->CloneRuntime());
+  return clone_model;
+}
+
 bool DBDetector::Predict(const cv::Mat& img,
                          std::vector<std::array<int, 8>>* boxes_result) {
   std::vector<std::vector<std::array<int, 8>>> det_results;

fastdeploy/vision/ocr/ppocr/dbdetector.h

@@ -19,6 +19,7 @@
 #include "fastdeploy/vision/ocr/ppocr/utils/ocr_postprocess_op.h"
 #include "fastdeploy/vision/ocr/ppocr/det_postprocessor.h"
 #include "fastdeploy/vision/ocr/ppocr/det_preprocessor.h"
+#include "fastdeploy/utils/unique_ptr.h"
 
 namespace fastdeploy {
 namespace vision {
@@ -42,8 +43,16 @@ class FASTDEPLOY_DECL DBDetector : public FastDeployModel {
   DBDetector(const std::string& model_file, const std::string& params_file = "",
              const RuntimeOption& custom_option = RuntimeOption(),
              const ModelFormat& model_format = ModelFormat::PADDLE);
+
+  /** \brief Clone a new DBDetector with less memory usage when multiple instances of the same model are created
+   *
+   * \return new DBDetector* type unique pointer
+   */
+  virtual std::unique_ptr<DBDetector> Clone() const;
+
   /// Get model's name
   std::string ModelName() const { return "ppocr/ocr_det"; }
+
    /** \brief Predict the input image and get OCR detection model result.
    *
    * \param[in] img The input image data, comes from cv::imread(), is a 3-D array with layout HWC, BGR format.
@@ -52,6 +61,7 @@ class FASTDEPLOY_DECL DBDetector : public FastDeployModel {
    */
   virtual bool Predict(const cv::Mat& img,
                        std::vector<std::array<int, 8>>* boxes_result);
+
    /** \brief BatchPredict the input image and get OCR detection model result.
    *
    * \param[in] images The list input of image data, comes from cv::imread(), is a 3-D array with layout HWC, BGR format.

fastdeploy/vision/ocr/ppocr/ppocr_pybind.cc

@@ -27,6 +27,9 @@ void BindPPOCRv3(pybind11::module& m) {
                           fastdeploy::vision::ocr::Recognizer*>())
       .def_property("cls_batch_size", &pipeline::PPOCRv3::GetClsBatchSize, &pipeline::PPOCRv3::SetClsBatchSize)
       .def_property("rec_batch_size", &pipeline::PPOCRv3::GetRecBatchSize, &pipeline::PPOCRv3::SetRecBatchSize)
+      .def("clone", [](pipeline::PPOCRv3& self) {
+        return self.Clone();
+      })
       .def("predict", [](pipeline::PPOCRv3& self,
                          pybind11::array& data) {
         auto mat = PyArrayToCvMat(data);
@@ -56,6 +59,9 @@ void BindPPOCRv2(pybind11::module& m) {
                           fastdeploy::vision::ocr::Recognizer*>())
       .def_property("cls_batch_size", &pipeline::PPOCRv2::GetClsBatchSize, &pipeline::PPOCRv2::SetClsBatchSize)
       .def_property("rec_batch_size", &pipeline::PPOCRv2::GetRecBatchSize, &pipeline::PPOCRv2::SetRecBatchSize)
+      .def("clone", [](pipeline::PPOCRv2& self) {
+        return self.Clone();
+      })
       .def("predict", [](pipeline::PPOCRv2& self,
                          pybind11::array& data) {
         auto mat = PyArrayToCvMat(data);

fastdeploy/vision/ocr/ppocr/ppocr_v2.cc

@@ -74,6 +74,17 @@ bool PPOCRv2::Initialized() const {
   }
   return true; 
 }
+
+std::unique_ptr<PPOCRv2> PPOCRv2::Clone() const {
+  std::unique_ptr<PPOCRv2> clone_model = utils::make_unique<PPOCRv2>(PPOCRv2(*this));
+  clone_model->detector_ = detector_->Clone().release();
+  if (classifier_ != nullptr) {
+    clone_model->classifier_ = classifier_->Clone().release();
+  }
+  clone_model->recognizer_ = recognizer_->Clone().release();
+  return clone_model;
+}
+
 bool PPOCRv2::Predict(cv::Mat* img,
                             fastdeploy::vision::OCRResult* result) {
   return Predict(*img, result);

fastdeploy/vision/ocr/ppocr/ppocr_v2.h

@@ -24,6 +24,7 @@
 #include "fastdeploy/vision/ocr/ppocr/dbdetector.h"
 #include "fastdeploy/vision/ocr/ppocr/recognizer.h"
 #include "fastdeploy/vision/ocr/ppocr/utils/ocr_postprocess_op.h"
+#include "fastdeploy/utils/unique_ptr.h"
 
 namespace fastdeploy {
 /** \brief This pipeline can launch detection model, classification model and recognition model sequentially. All OCR pipeline APIs are defined inside this namespace.
@@ -52,6 +53,12 @@ class FASTDEPLOY_DECL PPOCRv2 : public FastDeployModel {
   PPOCRv2(fastdeploy::vision::ocr::DBDetector* det_model,
                 fastdeploy::vision::ocr::Recognizer* rec_model);
 
+  /** \brief Clone a new PPOCRv2 with less memory usage when multiple instances of the same model are created
+   *
+   * \return new PPOCRv2* type unique pointer
+   */
+  std::unique_ptr<PPOCRv2> Clone() const;
+
   /** \brief Predict the input image and get OCR result.
    *
    * \param[in] im The input image data, comes from cv::imread(), is a 3-D array with layout HWC, BGR format.
@@ -69,6 +76,7 @@ class FASTDEPLOY_DECL PPOCRv2 : public FastDeployModel {
    */
   virtual bool BatchPredict(const std::vector<cv::Mat>& images,
                std::vector<fastdeploy::vision::OCRResult>* batch_result);
+  
   bool Initialized() const override;
   bool SetClsBatchSize(int cls_batch_size);
   int GetClsBatchSize();
@@ -83,7 +91,6 @@ class FASTDEPLOY_DECL PPOCRv2 : public FastDeployModel {
  private:
   int cls_batch_size_ = 1;
   int rec_batch_size_ = 6;
-  /// Launch the detection process in OCR.
 };
 
 namespace application {

fastdeploy/vision/ocr/ppocr/ppocr_v3.h

@@ -49,6 +49,20 @@ class FASTDEPLOY_DECL PPOCRv3 : public PPOCRv2 {
     // The only difference between v2 and v3
     recognizer_->GetPreprocessor().rec_image_shape_[1] = 48;
   }
+
+  /** \brief Clone a new PPOCRv3 with less memory usage when multiple instances of the same model are created
+   *
+   * \return new PPOCRv3* type unique pointer
+   */
+  std::unique_ptr<PPOCRv3> Clone() const {
+    std::unique_ptr<PPOCRv3> clone_model = utils::make_unique<PPOCRv3>(PPOCRv3(*this));
+    clone_model->detector_ = detector_->Clone().release();
+    if (classifier_ != nullptr) {
+      clone_model->classifier_ = classifier_->Clone().release();
+    }
+    clone_model->recognizer_ = recognizer_->Clone().release();
+  return clone_model;
+  }
 };
 
 }  // namespace pipeline

fastdeploy/vision/ocr/ppocr/recognizer.cc

@@ -56,6 +56,12 @@ bool Recognizer::Initialize() {
   return true;
 }
 
+std::unique_ptr<Recognizer> Recognizer::Clone() const {
+  std::unique_ptr<Recognizer> clone_model = utils::make_unique<Recognizer>(Recognizer(*this));
+  clone_model->SetRuntime(clone_model->CloneRuntime());
+  return clone_model;
+}
+
 bool Recognizer::Predict(const cv::Mat& img, std::string* text, float* rec_score) {
   std::vector<std::string> texts(1);
   std::vector<float> rec_scores(1);

fastdeploy/vision/ocr/ppocr/recognizer.h

@@ -19,6 +19,7 @@
 #include "fastdeploy/vision/ocr/ppocr/utils/ocr_postprocess_op.h"
 #include "fastdeploy/vision/ocr/ppocr/rec_preprocessor.h"
 #include "fastdeploy/vision/ocr/ppocr/rec_postprocessor.h"
+#include "fastdeploy/utils/unique_ptr.h"
 
 namespace fastdeploy {
 namespace vision {
@@ -43,8 +44,16 @@ class FASTDEPLOY_DECL Recognizer : public FastDeployModel {
              const std::string& label_path = "",
              const RuntimeOption& custom_option = RuntimeOption(),
              const ModelFormat& model_format = ModelFormat::PADDLE);
+  
   /// Get model's name
   std::string ModelName() const { return "ppocr/ocr_rec"; }
+
+  /** \brief Clone a new Recognizer with less memory usage when multiple instances of the same model are created
+   *
+   * \return new Recognizer* type unique pointer
+   */
+  virtual std::unique_ptr<Recognizer> Clone() const;
+
   /** \brief Predict the input image and get OCR recognition model result.
    *
    * \param[in] img The input image data, comes from cv::imread(), is a 3-D array with layout HWC, BGR format.
@@ -53,6 +62,7 @@ class FASTDEPLOY_DECL Recognizer : public FastDeployModel {
    * \return true if the prediction is successed, otherwise false.
    */
   virtual bool Predict(const cv::Mat& img, std::string* text, float* rec_score);
+
   /** \brief BatchPredict the input image and get OCR recognition model result.
    *
    * \param[in] images The list of input image data, comes from cv::imread(), is a 3-D array with layout HWC, BGR format.
@@ -62,6 +72,7 @@ class FASTDEPLOY_DECL Recognizer : public FastDeployModel {
    */
   virtual bool BatchPredict(const std::vector<cv::Mat>& images,
                std::vector<std::string>* texts, std::vector<float>* rec_scores);
+  
   virtual bool BatchPredict(const std::vector<cv::Mat>& images,
                std::vector<std::string>* texts, std::vector<float>* rec_scores,
                size_t start_index, size_t end_index,