2016-08-22-tidb-weekly.md

layout	title
post	Weekly Update

Last week, we landed 26 PRs in the TiDB repositories and 15 PRs in the TiKV repositories.

Notable changes to TiDB

• Upgrade the query optimizer.
• Upgrade the lexer.
• Replace golang protobuf with gogo protobuf.
• Optimize the distributed executor.
• Repair the Time and Decimal types to improve the compatibility with MySQL.
• Support the Set names binary statement.
• Support Covering Index.
• Optimize the table scanning when the condition is false constant.
• Fix several bugs.

Notable changes to TiKV

• Add the leader lease read support for better performance, see benchmark.
• Use delete_file_in_range from RocksDB to destroy Regions quickly to avoid blocking Raft storage threads.
• Support GC for obsolete data versions.
• Coprocessor supports covering index for Select Where and the aggregation operations.
• Check whether the key is already rolled back when prewrite to fix a transaction bug.
• Support the --log-file flag to redirect log to the log file.

Notable changes to Placement Driver

• Refine the join flag to support multiple join scenarios.
• Use unix socket in test to avoid the "Address Already in Use" error.
• Redirect requests to the Leader if the current member is a Follower.

Benchmark

Use sysbench to benchmark leader lease read and previous Raft quorum read in 3-node TiKV.

Insert

# Prepare data
sysbench --test=./lua-tests/db/oltp.lua --mysql-host=${host} --mysql-port=${port} \
 --mysql-user=${user} --mysql-password=${password} --oltp-tables-count=$1 \
 --oltp-table-size=5120000 --rand-init=on prepare

# Run benchmark
sysbench --test=./lua-tests/db/insert.lua --mysql-host=${host} --mysql-port=${port} \
 --mysql-user=${user} --mysql-password=${password} --oltp-tables-count=1 \
 --oltp-table-size= 5120000 --num-threads=${threads} --report-interval=60 \
 --max-requests=1280000 --percentile=99 run

|Threads|Leader lease read qps|Leader lease read avg/.99 latency|Raft quorum read qps|Raft quorum read/.99 latency| |---|---|---|---|---|---| |32|2296|13.93/15.28|1315|24.33/94| |64|2199|29.1/145|1325|48.29/473| |128|1854|69/931|1290|99/697|

As we can see, the qps is increased by about 70%, and the latency is decreased by about 40%.

Select

# Prepare data
sysbench --test=./lua-tests/db/oltp.lua --mysql-host=${host} --mysql-port=${port} \
 --mysql-user=${user} --mysql-password=${password} --oltp-tables-count=1 \
 --oltp-table-size=5120000 --rand-init=on prepare

# Run benchmark
sysbench --test=./lua-tests/db/select.lua --mysql-host=${host} --mysql-port=${port} \
 --mysql-user=${user} --mysql-password=${password} --oltp-tables-count=1 \
 --oltp-table-size=5120000 --num-threads=${threads} --report-interval=60 \
 --max-requests=1280000 --percentile=99 run

|Threads|Leader lease read qps|Leader lease read avg/.99 latency|Raft quorum read qps|Raft quorum read/.99 latency| |---|---|---|---|---|---| |32|21010|1.52/7.53|12221|2.62/6.69| |64|25948|2.47/10.20|12637|5.06/11.62| |128|27283|4.69/13.68|11069|11.56/35.88|

As we can see, the qps is increased by about 130%, and the latency is decreased by about 50%.

New contributors

• hhkbp2