2016-09-05-tidb-weekly.md

title	date	summary	tags
Weekly update (August 29 ~ September 04, 2016)	2016-09-05	Last week, we landed 29 PRs in the TiDB repositories and 24 PRs in the TiKV repositories.	TiDB TiKV Placement Driver

Weekly update (August 29 ~ September 04, 2016)

Last week, we landed 29 PRs in the TiDB repositories and 24 PRs in the TiKV repositories.

Notable changes to TiDB

• Support the unhex and the ceiling/ceil functions
• Improve the Parser to handle \r\n.
• Solve the potential concurrency issues
• Support Load Data
• Use the Pipeline model to filter data through indexes to improve the performance
• Improve the code to reduce memory allocation and improve the performance
• Fix several bugs.

Notable changes to TiKV

• Coprocessor supports the new decimal type.
• Use the Raft column family to save Raft meta and logs. See Benchmark.
• Tune the write column family to reduce memory usage.

Notable changes to Placement Driver

• Check duplicated store addresses to prevent user from bootstrapping cluster in the wrong way, see issues 287, 288.
• Support the remove store API to remove a dead TiKV store.
• Use glide instead of the original godep to manage vendor.
• Remove join itself to prevent user from starting a removed PD server again.

Benchmark

Use sysbench to benchmark using the (CF_RAFT) column family to save the Raft log and previously the default (CF_DEFAULT) column family in 3-node TiKV.

# Prepare data
sysbench --test=./lua-tests/db/oltp.lua --mysql-host=${host} --mysql-port=${port} \
 --mysql-user=${user} --mysql-password=${password} --oltp-tables-count=1 \
 --oltp-table-size=${table_size} --rand-init=on prepare

# Run benchmark
sysbench --test=./lua-tests/db/insert.lua --mysql-host=${host} --mysql-port=${port} \
 --mysql-user=${user} --mysql-password=${password} --oltp-tables-count=1 \
 --oltp-table-size=${table_size} --num-threads=${threads} --report-interval=60 \
 --max-requests=1280000 --percentile=99 run

Threads	Table Size	CF_DEFAULT qps	CF_DEFAULT avg/.99 latency	CF_RAFT qps	CF_RAFT avg/.99 latency
32	6400000	3885	8.24/13.48	3979	8.04/13.70
64	7680000	3653	17.52/34.10	4477	14.29/24.49
128	8960000	3422	37.39/70.10	4642	27.57/57.45

As we can see, the qps is increased by about 22%, and the latency is decreased by about 18%.

New contributors

• Dagang Wei