Skip to content
This repository was archived by the owner on Feb 20, 2023. It is now read-only.

Add Sqlancer Test Type and Installation Requirements - #1500

Open
dniu16 wants to merge 4 commits into
cmu-db:masterfrom
dniu16:sqlancer_test
Open

Add Sqlancer Test Type and Installation Requirements#1500
dniu16 wants to merge 4 commits into
cmu-db:masterfrom
dniu16:sqlancer_test

Conversation

@dniu16

@dniu16 dniu16 commented Feb 26, 2021

Copy link
Copy Markdown
Contributor

This PR adds the sqlancer test type following the instruction here: https://github.com/cmu-db/noisepage/tree/master/script/testing. The test can be run using the command "PYTHONPATH=.. python3 -m script.testing.sqlancer" under the noisepage directory.

In addition, maven is installed in installation/package.sh since it is needed to build sqlancer.

@dniu16 dniu16 self-assigned this Feb 26, 2021
@dniu16 dniu16 added the feature Adds a requested feature label Feb 26, 2021
from ..util.constants import DIR_TMP

# git settings for sqlancer.
SQLANCER_GIT_URL = "https://github.com/dniu16/sqlancer.git"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@apavlo apavlo added the ready-for-ci Indicate that this build should be run through CI. label Mar 12, 2021
@noisepage-checks

Copy link
Copy Markdown

Major Decrease in Performance

STOP: this PR has a major negative performance impact

tps (%change) benchmark_type wal_device details
-2.05% tpcc RAM disk
Detailsmaster tps=8894.4, commit tps=8712.46, query_mode=extended, benchmark_type=tpcc, scale_factor=32.0000, terminals=32, client_time=60, weights={'Payment': 43, 'Delivery': 4, 'NewOrder': 45, 'StockLevel': 4, 'OrderStatus': 4}, wal_device=RAM disk, max_connection_threads=32
-1.06% tpcc None
Detailsmaster tps=9997.24, commit tps=9891.11, query_mode=extended, benchmark_type=tpcc, scale_factor=32.0000, terminals=32, client_time=60, weights={'Payment': 43, 'Delivery': 4, 'NewOrder': 45, 'StockLevel': 4, 'OrderStatus': 4}, wal_device=None, max_connection_threads=32
-2.83% tpcc HDD
Detailsmaster tps=491.73, commit tps=477.83, query_mode=extended, benchmark_type=tpcc, scale_factor=32.0000, terminals=32, client_time=60, weights={'Payment': 43, 'Delivery': 4, 'NewOrder': 45, 'StockLevel': 4, 'OrderStatus': 4}, wal_device=HDD, max_connection_threads=32
2.33% tatp RAM disk
Detailsmaster tps=3462.28, commit tps=3543.02, query_mode=extended, benchmark_type=tatp, scale_factor=1.0000, terminals=16, client_time=60, weights={'GetAccessData': 35, 'UpdateLocation': 14, 'GetNewDestination': 10, 'GetSubscriberData': 35, 'DeleteCallForwarding': 2, 'InsertCallForwarding': 2, 'UpdateSubscriberData': 2}, wal_device=RAM disk, max_connection_threads=32
-3.25% tatp None
Detailsmaster tps=3799.59, commit tps=3676.15, query_mode=extended, benchmark_type=tatp, scale_factor=1.0000, terminals=16, client_time=60, weights={'GetAccessData': 35, 'UpdateLocation': 14, 'GetNewDestination': 10, 'GetSubscriberData': 35, 'DeleteCallForwarding': 2, 'InsertCallForwarding': 2, 'UpdateSubscriberData': 2}, wal_device=None, max_connection_threads=32
-7.3% tatp HDD
Detailsmaster tps=351.98, commit tps=326.28, query_mode=extended, benchmark_type=tatp, scale_factor=1.0000, terminals=16, client_time=60, weights={'GetAccessData': 35, 'UpdateLocation': 14, 'GetNewDestination': 10, 'GetSubscriberData': 35, 'DeleteCallForwarding': 2, 'InsertCallForwarding': 2, 'UpdateSubscriberData': 2}, wal_device=HDD, max_connection_threads=32

@codecov

codecov Bot commented Mar 13, 2021

Copy link
Copy Markdown

Codecov Report

Merging #1500 (7309ef8) into master (68c0069) will decrease coverage by 0.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1500      +/-   ##
==========================================
- Coverage   81.79%   81.78%   -0.02%     
==========================================
  Files         706      706              
  Lines       50425    50425              
==========================================
- Hits        41246    41239       -7     
- Misses       9179     9186       +7     
Impacted Files Coverage Δ
src/storage/arrow_serializer.cpp 80.00% <0.00%> (-6.25%) ⬇️
src/include/metrics/metrics_store.h 61.66% <0.00%> (-1.67%) ⬇️
src/include/storage/index/bplustree.h 89.49% <0.00%> (-0.34%) ⬇️
src/storage/data_table.cpp 98.29% <0.00%> (+1.13%) ⬆️
src/include/storage/sql_table.h 100.00% <0.00%> (+2.32%) ⬆️
src/transaction/transaction_manager.cpp 95.23% <0.00%> (+2.72%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 68c0069...7309ef8. Read the comment docs.

@lmwnshn lmwnshn removed the ready-for-ci Indicate that this build should be run through CI. label Mar 14, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

feature Adds a requested feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants