How do I create a UDF and enable datafusion to parse it in datafusion rust?

[KeeganJin]

Hi all,
I want to create a UDF in a stream processing system with the ability to call a ML operator. since I didn't find any support related to that. I wonder if anyone here could help?
The table looks like this:
device_id | data
1 | data_01
2 | data_02

I want to perform ML operator on the "data" column, which leads me to UDF. The SQL query is like:
SELECT ML_Operator(data) FROM table, where the ML_Operator should be the user-defined function.

I wonder if it is possible in datafusion, and if possible how would the query be parsed? Thanks!

[jiangzhx]

hi @KeeganJin

There is an example of how to create a UDF at https://github.com/apache/arrow-datafusion/blob/main/datafusion-examples/examples/simple_udf.rs. Additionally, examples of creating UDAFs and UDWFs can be found in the same 'examples' directory.

[alamb]

Maybe https://www.linkedin.com/pulse/guide-user-defined-functions-apache-arrow-datafusion-dade-aderemi/ would help?