H2O Benchmarks degradation analysis OmniSci/Modin

[aregm]

Analyze degradations in H2O join benchmarks and groupby 10 query.

[gshimansky]

Working h2o join benchmark for Modin h2oai/db-benchmark#162

[ienkovich]

For join queries on Modin-on-OmniSci most of the time (up to 90%) is spent on ResultSet conversion to Arrow format. The columnar converter is not used in these cases.

[ienkovich]

Slow conversion is caused by an arrow dictionary build. In Q1-Q4 we have one such dictionary built in ~2.2 sec. For Q5 we have two such dictionaries.

[ienkovich]

Example conversion timing (for join Q1):

2788ms total duration for convertToArrow                                           
  2788ms start(0ms) getArrowBatch ArrowResultSetConverter.cpp:509                  
    2090ms start(0ms) T12 initialize column builders ArrowResultSetConverter.cpp:526
      0ms start(0ms) initializeColumnBuilder ArrowResultSetConverter.cpp:949       
      0ms start(0ms) initializeColumnBuilder ArrowResultSetConverter.cpp:949       
      0ms start(0ms) initializeColumnBuilder ArrowResultSetConverter.cpp:949       
      0ms start(0ms) initializeColumnBuilder ArrowResultSetConverter.cpp:949       
        0ms start(0ms) build dictionary ArrowResultSetConverter.cpp:958            
      0ms start(0ms) initializeColumnBuilder ArrowResultSetConverter.cpp:949       
        0ms start(0ms) build dictionary ArrowResultSetConverter.cpp:958            
      2090ms start(0ms) initializeColumnBuilder ArrowResultSetConverter.cpp:949    
        2090ms start(0ms) build dictionary ArrowResultSetConverter.cpp:958         
      0ms start(2090ms) initializeColumnBuilder ArrowResultSetConverter.cpp:949    
      0ms start(2090ms) initializeColumnBuilder ArrowResultSetConverter.cpp:949    
        0ms start(2090ms) build dictionary ArrowResultSetConverter.cpp:958         
      0ms start(2090ms) initializeColumnBuilder ArrowResultSetConverter.cpp:949    
    160ms start(2090ms) T06 columnar converter ArrowResultSetConverter.cpp:698     
    537ms start(2251ms) T07 row converter ArrowResultSetConverter.cpp:762          
      218ms start(2484ms) T08 append rows to arrow ArrowResultSetConverter.cpp:787
      84ms start(2704ms) T10 finish builders ArrowResultSetConverter.cpp:817       
    0ms start(2788ms) T11 make record batch ArrowResultSetConverter.cpp:830

[ienkovich]

There is an opportunity to improve performance for join query 5 by performing parallel arrow dictionaries build. Submitted intel-ai/omniscidb#110 which decreases Q5 execution time from 7 sec to 5 sec.

[anmyachev]

I think it's no longer relevant. @Garra1980 should we close the issue?