Add ML module

[abdelaziz-mahdy]

Question

i know that ML module is not supported yet, any way i could help with that since i want to use CascadeClassifier i think its in the ml module (i didnt check )

if i cant help with it, any time plan for it?

[rainyl]

@abdelaziz-mahdy

If you just need CascadeClassifier, it is included in objdetect module, and has been implemented, https://docs.opencv.org/4.x/d1/de5/classcv_1_1CascadeClassifier.html

opencv_dart/lib/src/objdetect/objdetect.dart

Line 15 in 4af53ea

class CascadeClassifier extends CvStruct<cvg.CascadeClassifier> {

opencv_dart/test/objdetect_test.dart

Line 6 in 4af53ea

test('cv.CascadeClassifier', () {

It seems training machine learning models using python and other tools is a more common and fashion way nowadays, and I have no much free time, so ML module was not implemented and not planned recently, maybe it will be added several months later when I have some time.

Anyway, it is true that some people need ML module, I(and other users) will be grateful if you can implement it.

[abdelaziz-mahdy]

Thank you very much for the links and the awesome package, I will check what needs to be implemented and try to do it

[rainyl]

😄

[abdelaziz-mahdy]

i am trying to make some guide lines so that i follow if its okay

1. Branch Selection:

   • Should I work on the main branch or the native assets branch? since The native assets branch has many changes.

2. Workflow with ffigen:

   • Add the desired implementation in the .h files.
   • Implement the logic in the .cpp files.
   • Add the interface in Dart.
   • Create the test cases for the Dart interface.

3. Checklist:

   • Determine the correct branch to work on. main
   • Add desired implementation in .h files.
   • Implement logic in .cpp files.
   • Run ffigen to generate bindings
   • Add interface in Dart.
   • Create test cases for the Dart interface.

Questions:

1. OpenCV on iOS:
   • Have you tried using OpenCV on a clean project in iOS?
   • I remember having an issue with iOS and FFI where the C++ files get ignored, and the developer has to import them manually. I will check if this is still a problem. The native assets should fix this issue.

note: this is my package where i had the problem https://github.com/abdelaziz-mahdy/pytorch_lite/blob/latest-ffi/ios/pytorch_lite.podspec but i see that you created .framework which should not have this problem so ignore this question,

[rainyl]

wow, you are the author of pytorch_lite, cool 👍

For the questions:

1. Branch Selection: develop on a new branch based on main, it is easy to update native-assets once finished on main.
2. OpenCV on iOS: as you said, for ios the .framework is used, I have no apple devices so not tested, but other developers tested and it worked.

I have edited your checklist, hope you won't mind it.

BTW, you can find more build instructions in workflow file, thanks for your efforts in advance.

[rainyl]

One more thing, to speedup the build of opencv_dart, opencv itself is built in another repo with ML disabled, I have enabled it but the workflow has not finished yet, generally it will take about 40 min, once finished, the published tag will be 4.9.0+3 so you need to change the opencv version in conanfile.py to "4.9.0+3" at

opencv_dart/conanfile.py

Line 10 in 4af53ea

OPENCV_VERSION = "4.9.0+2"

You can check the build process here:

[abdelaziz-mahdy]

Oh thank you for that, is there is a guide you follow of what the headers are and the implementation for the functions? I checked gocv but looks like there is no ml module or I can't find it,

If there is something I can take a reference that will make it much easier

edit: also i see that the build finished but the release doesnt have the files https://github.com/rainyl/opencv.full/releases/tag/4.9.0%2B3

[rainyl]

Yes, this project was originally developed based on gocv, but now nearly every API has been refactored to cache exceptions, the most important guide is the documentation of opencv, i.e., wrap the classes and functions with C-style functions.

You can take a look at cv.Subdiv2D in this PR: #60

headers:

opencv_dart/src/imgproc/imgproc.h

Lines 168 to 187 in d469061

	CvStatus Subdiv2D_NewEmpty(Subdiv2D *rval);
	CvStatus Subdiv2D_NewWithRect(Rect rect, Subdiv2D *rval);
	void Subdiv2D_Close(Subdiv2D *self);
	CvStatus Subdiv2D_EdgeDst(Subdiv2D self, int edge, Point2f *dstpt, int *rval);
	CvStatus Subdiv2D_EdgeOrg(Subdiv2D self, int edge, Point2f *orgpt, int *rval);
	CvStatus Subdiv2D_FindNearest(Subdiv2D self, Point2f pt, Point2f *nearestPt, int *rval);
	CvStatus Subdiv2D_GetEdge(Subdiv2D self, int edge, int nextEdgeType, int *rval);
	CvStatus Subdiv2D_GetEdgeList(Subdiv2D self, Vec4f **rval, int *size);
	CvStatus Subdiv2D_GetLeadingEdgeList(Subdiv2D self, VecInt *leadingEdgeList);
	CvStatus Subdiv2D_GetTriangleList(Subdiv2D self, Vec6f **rval, int *size);
	CvStatus Subdiv2D_GetVertex(Subdiv2D self, int vertex, int *firstEdge, Point2f *rval);
	CvStatus Subdiv2D_GetVoronoiFacetList(Subdiv2D self, VecInt idx, VecVecPoint2f *facetList,
	VecPoint2f *facetCenters);
	CvStatus Subdiv2D_InitDelaunay(Subdiv2D self, Rect rect);
	CvStatus Subdiv2D_Insert(Subdiv2D self, Point2f pt, int *rval);
	CvStatus Subdiv2D_InsertVec(Subdiv2D self, VecPoint2f ptvec);
	CvStatus Subdiv2D_Locate(Subdiv2D self, Point2f pt, int *edge, int *vertex, int *rval);
	CvStatus Subdiv2D_NextEdge(Subdiv2D self, int edge, int *rval);
	CvStatus Subdiv2D_RotateEdge(Subdiv2D self, int edge, int rotate, int *rval);
	CvStatus Subdiv2D_SymEdge(Subdiv2D self, int edge, int *rval);

implementation:

opencv_dart/src/imgproc/imgproc.cpp

Lines 751 to 888 in d469061

	CvStatus Subdiv2D_NewEmpty(Subdiv2D *rval)
	{
	BEGIN_WRAP
	*rval = {new cv::Subdiv2D()};
	END_WRAP
	}
	CvStatus Subdiv2D_NewWithRect(Rect rect, Subdiv2D *rval)
	{
	BEGIN_WRAP
	*rval = {new cv::Subdiv2D(cv::Rect(rect.x, rect.y, rect.width, rect.height))};
	END_WRAP
	}
	void Subdiv2D_Close(Subdiv2D *self){CVD_FREE(self)}
	CvStatus Subdiv2D_EdgeDst(Subdiv2D self, int edge, Point2f *dstpt, int *rval)
	{
	BEGIN_WRAP
	auto p = cv::Point2f();
	*rval = self.ptr->edgeDst(edge, &p);
	*dstpt = {p.x, p.y};
	END_WRAP
	}
	CvStatus Subdiv2D_EdgeOrg(Subdiv2D self, int edge, Point2f *orgpt, int *rval)
	{
	BEGIN_WRAP
	auto p = cv::Point2f();
	*rval = self.ptr->edgeOrg(edge, &p);
	*orgpt = {p.x, p.y};
	END_WRAP
	}
	CvStatus Subdiv2D_FindNearest(Subdiv2D self, Point2f pt, Point2f *nearestPt, int *rval)
	{
	BEGIN_WRAP
	auto p = cv::Point2f();
	*rval = self.ptr->findNearest(cv::Point2f(pt.x, pt.y), &p);
	*nearestPt = {p.x, p.y};
	END_WRAP
	}
	CvStatus Subdiv2D_GetEdge(Subdiv2D self, int edge, int nextEdgeType, int *rval)
	{
	BEGIN_WRAP
	*rval = self.ptr->getEdge(edge, nextEdgeType);
	END_WRAP
	}
	CvStatus Subdiv2D_GetEdgeList(Subdiv2D self, Vec4f **rval, int *size)
	{
	BEGIN_WRAP
	auto v = std::vector<cv::Vec4f>();
	self.ptr->getEdgeList(v);
	*size = v.size();
	auto rv = new Vec4f[v.size()];
	for (int i = 0; i < v.size(); i++) {
	rv[i] = {v[i].val[0], v[i].val[1], v[i].val[2], v[i].val[3]};
	}
	*rval = rv;
	END_WRAP
	}
	CvStatus Subdiv2D_GetLeadingEdgeList(Subdiv2D self, VecInt *leadingEdgeList)
	{
	BEGIN_WRAP
	auto v = new std::vector<int>();
	self.ptr->getLeadingEdgeList(*v);
	*leadingEdgeList = {v};
	END_WRAP
	}
	CvStatus Subdiv2D_GetTriangleList(Subdiv2D self, Vec6f **rval, int *size)
	{
	BEGIN_WRAP
	auto v = std::vector<cv::Vec6f>();
	self.ptr->getTriangleList(v);
	*size = v.size();
	auto rv = new Vec6f[v.size()];
	for (int i = 0; i < v.size(); i++) {
	rv[i] = {v[i].val[0], v[i].val[1], v[i].val[2], v[i].val[3], v[i].val[4], v[i].val[5]};
	}
	*rval = rv;
	END_WRAP
	}
	CvStatus Subdiv2D_GetVertex(Subdiv2D self, int vertex, int *firstEdge, Point2f *rval)
	{
	BEGIN_WRAP
	auto p = self.ptr->getVertex(vertex, firstEdge);
	*rval = {p.x, p.y};
	END_WRAP
	}
	CvStatus Subdiv2D_GetVoronoiFacetList(Subdiv2D self, VecInt idx, VecVecPoint2f *facetList,
	VecPoint2f *facetCenters)
	{
	BEGIN_WRAP
	auto vf = std::vector<std::vector<cv::Point2f>>();
	auto vfc = std::vector<cv::Point2f>();
	self.ptr->getVoronoiFacetList(*idx.ptr, vf, vfc);
	*facetList = {new std::vector<std::vector<cv::Point2f>>(vf)};
	*facetCenters = {new std::vector<cv::Point2f>(vfc)};
	END_WRAP;
	}
	CvStatus Subdiv2D_InitDelaunay(Subdiv2D self, Rect rect)
	{
	BEGIN_WRAP
	self.ptr->initDelaunay(cv::Rect(rect.x, rect.y, rect.width, rect.height));
	END_WRAP
	}
	CvStatus Subdiv2D_Insert(Subdiv2D self, Point2f pt, int *rval)
	{
	BEGIN_WRAP
	*rval = self.ptr->insert(cv::Point2f(pt.x, pt.y));
	END_WRAP
	}
	CvStatus Subdiv2D_InsertVec(Subdiv2D self, VecPoint2f ptvec)
	{
	BEGIN_WRAP
	self.ptr->insert(*ptvec.ptr);
	END_WRAP
	}
	CvStatus Subdiv2D_Locate(Subdiv2D self, Point2f pt, int *edge, int *vertex, int *rval)
	{
	BEGIN_WRAP
	*rval = self.ptr->locate(cv::Point2f(pt.x, pt.y), *edge, *vertex);
	END_WRAP
	}
	CvStatus Subdiv2D_NextEdge(Subdiv2D self, int edge, int *rval)
	{
	BEGIN_WRAP
	*rval = self.ptr->nextEdge(edge);
	END_WRAP
	}
	CvStatus Subdiv2D_RotateEdge(Subdiv2D self, int edge, int rotate, int *rval)
	{
	BEGIN_WRAP
	*rval = self.ptr->rotateEdge(edge, rotate);
	END_WRAP
	}
	CvStatus Subdiv2D_SymEdge(Subdiv2D self, int edge, int *rval)
	{
	BEGIN_WRAP
	*rval = self.ptr->symEdge(edge);
	END_WRAP
	}

Edit: Now published

[abdelaziz-mahdy]

Yes, this project was originally developed based on gocv, but now nearly every API has been refactored to cache exceptions, the most important guide is the documentation of opencv, i.e., wrap the classes and functions with C-style functions.

You can take a look at cv.Subdiv2D in this PR: #60

headers:

opencv_dart/src/imgproc/imgproc.h

Lines 168 to 187 in d469061

	CvStatus Subdiv2D_NewEmpty(Subdiv2D *rval);
	CvStatus Subdiv2D_NewWithRect(Rect rect, Subdiv2D *rval);
	void Subdiv2D_Close(Subdiv2D *self);
	CvStatus Subdiv2D_EdgeDst(Subdiv2D self, int edge, Point2f *dstpt, int *rval);
	CvStatus Subdiv2D_EdgeOrg(Subdiv2D self, int edge, Point2f *orgpt, int *rval);
	CvStatus Subdiv2D_FindNearest(Subdiv2D self, Point2f pt, Point2f *nearestPt, int *rval);
	CvStatus Subdiv2D_GetEdge(Subdiv2D self, int edge, int nextEdgeType, int *rval);
	CvStatus Subdiv2D_GetEdgeList(Subdiv2D self, Vec4f **rval, int *size);
	CvStatus Subdiv2D_GetLeadingEdgeList(Subdiv2D self, VecInt *leadingEdgeList);
	CvStatus Subdiv2D_GetTriangleList(Subdiv2D self, Vec6f **rval, int *size);
	CvStatus Subdiv2D_GetVertex(Subdiv2D self, int vertex, int *firstEdge, Point2f *rval);
	CvStatus Subdiv2D_GetVoronoiFacetList(Subdiv2D self, VecInt idx, VecVecPoint2f *facetList,
	VecPoint2f *facetCenters);
	CvStatus Subdiv2D_InitDelaunay(Subdiv2D self, Rect rect);
	CvStatus Subdiv2D_Insert(Subdiv2D self, Point2f pt, int *rval);
	CvStatus Subdiv2D_InsertVec(Subdiv2D self, VecPoint2f ptvec);
	CvStatus Subdiv2D_Locate(Subdiv2D self, Point2f pt, int *edge, int *vertex, int *rval);
	CvStatus Subdiv2D_NextEdge(Subdiv2D self, int edge, int *rval);
	CvStatus Subdiv2D_RotateEdge(Subdiv2D self, int edge, int rotate, int *rval);
	CvStatus Subdiv2D_SymEdge(Subdiv2D self, int edge, int *rval);

implementation:

opencv_dart/src/imgproc/imgproc.cpp

Lines 751 to 888 in d469061

	CvStatus Subdiv2D_NewEmpty(Subdiv2D *rval)
	{
	BEGIN_WRAP
	*rval = {new cv::Subdiv2D()};
	END_WRAP
	}
	CvStatus Subdiv2D_NewWithRect(Rect rect, Subdiv2D *rval)
	{
	BEGIN_WRAP
	*rval = {new cv::Subdiv2D(cv::Rect(rect.x, rect.y, rect.width, rect.height))};
	END_WRAP
	}
	void Subdiv2D_Close(Subdiv2D *self){CVD_FREE(self)}
	CvStatus Subdiv2D_EdgeDst(Subdiv2D self, int edge, Point2f *dstpt, int *rval)
	{
	BEGIN_WRAP
	auto p = cv::Point2f();
	*rval = self.ptr->edgeDst(edge, &p);
	*dstpt = {p.x, p.y};
	END_WRAP
	}
	CvStatus Subdiv2D_EdgeOrg(Subdiv2D self, int edge, Point2f *orgpt, int *rval)
	{
	BEGIN_WRAP
	auto p = cv::Point2f();
	*rval = self.ptr->edgeOrg(edge, &p);
	*orgpt = {p.x, p.y};
	END_WRAP
	}
	CvStatus Subdiv2D_FindNearest(Subdiv2D self, Point2f pt, Point2f *nearestPt, int *rval)
	{
	BEGIN_WRAP
	auto p = cv::Point2f();
	*rval = self.ptr->findNearest(cv::Point2f(pt.x, pt.y), &p);
	*nearestPt = {p.x, p.y};
	END_WRAP
	}
	CvStatus Subdiv2D_GetEdge(Subdiv2D self, int edge, int nextEdgeType, int *rval)
	{
	BEGIN_WRAP
	*rval = self.ptr->getEdge(edge, nextEdgeType);
	END_WRAP
	}
	CvStatus Subdiv2D_GetEdgeList(Subdiv2D self, Vec4f **rval, int *size)
	{
	BEGIN_WRAP
	auto v = std::vector<cv::Vec4f>();
	self.ptr->getEdgeList(v);
	*size = v.size();
	auto rv = new Vec4f[v.size()];
	for (int i = 0; i < v.size(); i++) {
	rv[i] = {v[i].val[0], v[i].val[1], v[i].val[2], v[i].val[3]};
	}
	*rval = rv;
	END_WRAP
	}
	CvStatus Subdiv2D_GetLeadingEdgeList(Subdiv2D self, VecInt *leadingEdgeList)
	{
	BEGIN_WRAP
	auto v = new std::vector<int>();
	self.ptr->getLeadingEdgeList(*v);
	*leadingEdgeList = {v};
	END_WRAP
	}
	CvStatus Subdiv2D_GetTriangleList(Subdiv2D self, Vec6f **rval, int *size)
	{
	BEGIN_WRAP
	auto v = std::vector<cv::Vec6f>();
	self.ptr->getTriangleList(v);
	*size = v.size();
	auto rv = new Vec6f[v.size()];
	for (int i = 0; i < v.size(); i++) {
	rv[i] = {v[i].val[0], v[i].val[1], v[i].val[2], v[i].val[3], v[i].val[4], v[i].val[5]};
	}
	*rval = rv;
	END_WRAP
	}
	CvStatus Subdiv2D_GetVertex(Subdiv2D self, int vertex, int *firstEdge, Point2f *rval)
	{
	BEGIN_WRAP
	auto p = self.ptr->getVertex(vertex, firstEdge);
	*rval = {p.x, p.y};
	END_WRAP
	}
	CvStatus Subdiv2D_GetVoronoiFacetList(Subdiv2D self, VecInt idx, VecVecPoint2f *facetList,
	VecPoint2f *facetCenters)
	{
	BEGIN_WRAP
	auto vf = std::vector<std::vector<cv::Point2f>>();
	auto vfc = std::vector<cv::Point2f>();
	self.ptr->getVoronoiFacetList(*idx.ptr, vf, vfc);
	*facetList = {new std::vector<std::vector<cv::Point2f>>(vf)};
	*facetCenters = {new std::vector<cv::Point2f>(vfc)};
	END_WRAP;
	}
	CvStatus Subdiv2D_InitDelaunay(Subdiv2D self, Rect rect)
	{
	BEGIN_WRAP
	self.ptr->initDelaunay(cv::Rect(rect.x, rect.y, rect.width, rect.height));
	END_WRAP
	}
	CvStatus Subdiv2D_Insert(Subdiv2D self, Point2f pt, int *rval)
	{
	BEGIN_WRAP
	*rval = self.ptr->insert(cv::Point2f(pt.x, pt.y));
	END_WRAP
	}
	CvStatus Subdiv2D_InsertVec(Subdiv2D self, VecPoint2f ptvec)
	{
	BEGIN_WRAP
	self.ptr->insert(*ptvec.ptr);
	END_WRAP
	}
	CvStatus Subdiv2D_Locate(Subdiv2D self, Point2f pt, int *edge, int *vertex, int *rval)
	{
	BEGIN_WRAP
	*rval = self.ptr->locate(cv::Point2f(pt.x, pt.y), *edge, *vertex);
	END_WRAP
	}
	CvStatus Subdiv2D_NextEdge(Subdiv2D self, int edge, int *rval)
	{
	BEGIN_WRAP
	*rval = self.ptr->nextEdge(edge);
	END_WRAP
	}
	CvStatus Subdiv2D_RotateEdge(Subdiv2D self, int edge, int rotate, int *rval)
	{
	BEGIN_WRAP
	*rval = self.ptr->rotateEdge(edge, rotate);
	END_WRAP
	}
	CvStatus Subdiv2D_SymEdge(Subdiv2D self, int edge, int *rval)
	{
	BEGIN_WRAP
	*rval = self.ptr->symEdge(edge);
	END_WRAP
	}

nice i like how clean this is done, i will check what i can do tomorrow and open a pr if i was able to implement it.

thank again for your help, this info will hopefully make it much easier for me to make the pr <3

[abdelaziz-mahdy]

ffigen fails in files not related to the file i made

is that a problem from my side?

running conan build . -b missing compiled successfully

adding
#include <stddef.h> before core file made it work, dont know if that the correct fix

[rainyl]

Well, I developed on windows and it worked normally, maybe macos needs stddef.h, feel free to add it.

[rainyl]

I have published a new branch ml with some data structures wrapped, but they were not tested, just for some detailed instructions and methods to provide some help, but forgive me that I have no more time to finish it, take a look and hoping it can help you~