2021 Q3 Roadmap

[tobegit3hub]

OneStore engine

• Import offline data
• Sync data between online and offline
• Manage and sync catalog

Feature improvement

• Bulk load
• Reslove memory fragement
• Generage DDL by SQL and schema
• Develop the script to build demo automaticly

Docs

• Deployment docs
• Usage docs
• Architecture and technical docs

Batch Execution Engine

• Support automatically packing for OpenMLDB Spark distribution
• Support Spark distribution for Linux, MacOS and Allinone versions
• Support reading and write tables from catalog service like Hive metastore
• Integrated with data exploration service like Presto and Hive
• Resolve the issue to run WindowSkewOpt for multiple windows
• Make UnsafeRowOpt stable and use by default
• Fix all the static check warning and increase 30% unit tests code coverage

Online Execution Engine

• Support distributed execution with pulling data from multiple tablets.
• Support time column index optimization for filter and join operator
• Support parameterized query

SQL ability

• Support more math, string, comparison, flow control and other functions.

bugfix and some other things

[jingchen2222]

OpenMLDB Execution Engine

https://github.com/4paradigm/OpenMLDB/projects/3

Running Issues

SQL ability

Support more math, string, comparison, flow control and other functions.

• (RUNNING) feat: support insert multiple rows into a table using a single SQL insert statement. #398
• SQL function: FirstValue and LastValue #222
• feat: support and validate query table with multiple-levels path #365
• support standard index option in create table statement #228 feat: refactor index option and move index option validation code to SqlClusterRouter #220
• sql: support like predicate #224
• Support UNION STATEMENT on physical plan #227
• (RUNNING) feat: support IN(...) predicate expression #234
• SqlCompiler: provide comment & hint info #235

Online Execution Engine

• Support parameterized query: Support batch query with parameter #287

  • Support online batch engine with parameters or placeholder #262
  • JDBC execute batch query with PrepareStatement #167
  • Support parameterized query in Python SDK #290

• Support Distributed query on batch mode: Distributed query on batch mode #320

  • (RUNNING) Support General aggegrate function over table COUNT, MAX, MIN, SUM #219
  • fix: engine plan optimization for where and group with the same partition #317
  • feat: support distributed query on BatchMode under some restrictions #318
  • feat: refactor of sql query router #362

• Support time column index optimization for filter and join operator

  • Engine optimize: optimize time expression based on time column index #221

• Resign the Error System #error

  • Make SQL error tips friendly #229
  • feat: improve error message system. #406

Benchmark

• OpenMLDB Benchmark Strategy #377
  • feat: run benchmark on GitHub workflow, compare & upload test results  #432

SDK and Api Server

• feat: api server support parameterized query #397
• feat: improve cli and make the console output more clean and clear #426

build & workflow & style

• reduce library (*.so) size #239

[jingchen2222]

SQL Ability

Query

Insert

Create Table

Create Procedure