From fbb2eaba3685eeef8747c8bf6ac16e21ca49182b Mon Sep 17 00:00:00 2001 From: Jeehoon Kang Date: Wed, 14 Mar 2018 02:56:34 +0900 Subject: [PATCH 1/3] Introduce concurrent circular buffer --- text/2018-03-14-circbuf.md | 128 +++++++++++++++++++++++++++++++++++++ 1 file changed, 128 insertions(+) create mode 100644 text/2018-03-14-circbuf.md diff --git a/text/2018-03-14-circbuf.md b/text/2018-03-14-circbuf.md new file mode 100644 index 0000000..52d5704 --- /dev/null +++ b/text/2018-03-14-circbuf.md @@ -0,0 +1,128 @@ +# Summary + +Introduce crate `crossbeam-circbuf` which provides SPSC and SPMC channels based on concurrent +circular buffer. Here is [a prototype implementation][crossbeam-circbuf-prototype]. + + +# Motivation + +Crossbeam already has a [work-stealing deque][crossbeam-deque] for being used in task schedulers, +which is successfully deployed in [Rayon][crossbeam-deque-rayon] and +[Tokio][crossbeam-deque-tokio]. However, Tokio actually doesn't use the full functionality of deque, +leaving `Deque::pop()` unused. This is because Tokio targets on IO and should be fair in executing +tasks, while `Deque::pop()` will pop the most recent task repeatedly. + +Basically, `crossbeam-circbuf` is the result of removing `Deque::pop()` from `crossbeam-deque`. +Since the synchronization between `Deque::pop()` and `Stealer::steal()` is the most intricate and +complex part of a work-stealing deque, removing `Deque::pop()` from the work-stealing deque will +greatly simplify the code base and make it faster. + +It is worth noting that [Go's scheduler also uses circular buffer, basically][go-scheduler]. + + +# Detailed design + +## Changes from deque + +Besides removing `pop()`, we renamed things and added three methods to circular buffer: + +### Name changes + +- `Deque` renamed into `CircBuf` +- `Stealer` renamed into `Receiver` +- `Deque::push()` renamed into `CircBuf::send()` +- `Deque::steal()`, `Stealer::steal()` renamed into `CircBuf::try_recv()`, + `Receiver::try_recv()`. Now they return `Result, RecvError>`, where `Ok(Some(v))` means + the value `v` is returned, `Ok(None)` means the circular buffer is empty, and + `Err(RecvError::Retry)` means you lost a race and may want to retry. + +### New methods + +- `CircBuf::recv()`, `Receiver::recv()`: retries to receive a value until get a value or check that + the circular buffer is empty. +- `Receiver::recv_exclusive()`: receives a value, assuming that there are no concurrent receiving + methods invocations. It's necessary for providing efficient SPSC receivers. + + +## SPSC and SPMC channels + +We provide SPSC and SPMC channels as a thin wrapper around circular buffer. Their API looks like: + +```rust +use concurrent_circbuf::spsc; +use std::thread; + +// Since there's only one receiver, `spsc::new()` just creates it. +let (tx, rx) = spsc::new::(); + +tx.send('a'); +tx.send('b'); +tx.send('c'); + +assert_eq!(rx.recv(), Some('a')); +drop(tx); + +thread::spawn(move || { + assert_eq!(rx.recv(), Some('b')); + assert_ne!(rx.try_recv(), Ok(None)); // it's not empty +}).join().unwrap(); +``` + +```rust +use concurrent_circbuf::spmc::{Channel, Receiver}; +use std::thread; + +// Since there can be multiple receivers, `Channel::new()` creates an SPMC channel, and the channel +// can create multiple receivers. +let c = Channel::::new(); +let r = c.receiver(); + +c.send('a'); +c.send('b'); +c.send('c'); + +assert_eq!(c.recv(), Some('a')); +drop(c); + +thread::spawn(move || { + assert_eq!(r.recv(), Some('b')); + assert_ne!(r.try_recv(), Ok(None)); // Ok(Some('c')) or Err(RecvError::Retry) +}).join().unwrap(); +``` + + +## Performance + +We benchmarked the performance of SPSC and SPMC using [`crossbeam-channel`'s +benchmark][crossbeam-channel-benchmark]. + +Results on an Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz (12 cores, 24 hw-threads) running Linux +4.11.12-1-MANJARO: + +![Graphs](https://user-images.githubusercontent.com/1201316/37359892-175db1dc-2732-11e8-992e-c4748ac919d5.png) + +It says `crossbeam-circbuf` outperforms not only `crossbeam-deque` but also `crossbeam-channel`, +`crossbeam::sync::SegQueue`, and `std::sync::mpsc` for unbounded SPSC and SPMC scenarios. + + +# Drawbacks and Alternatives + +We are creating even another crate. Is it worth? Maybe you can use `crossbeam-deque` in spite of +`crossbeam-circbuf`'s performance advantages. + +API distinguishes `recv()` and `try_recv()`, the only difference being that `recv()` always succeeds +in returning value or checking the emptiness, while `try_recv()` may lose a race and bail out. While +they have different performance characteristics (`recv()` being ~20% faster than spinning +`try_recv()` in benchmark), is it actually worth distinguishing? + +# Unresolved questions + +Not that I'm aware of now. + + +[crossbeam-deque]: https://github.com/crossbeam-rs/crossbeam-deque +[crossbeam-deque]: https://github.com/rayon-rs/rayon/pull/528 +[crossbeam-tokio]: https://github.com/tokio-rs/tokio/pull/185 +[go-scheduler]: https://github.com/golang/go/blob/master/src/runtime/proc.go#L4731 +[crossbeam-channel-benchmark]: https://github.com/crossbeam-rs/crossbeam-channel/tree/master/benchmarks +[crossbeam-circbuf-prototype]: https://github.com/jeehoonkang/concurrent-circbuf/ From 4366dde145e398fcf727181756cf662d7420888e Mon Sep 17 00:00:00 2001 From: Jeehoon Kang Date: Wed, 14 Mar 2018 02:59:33 +0900 Subject: [PATCH 2/3] Fix link error --- text/2018-03-14-circbuf.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/2018-03-14-circbuf.md b/text/2018-03-14-circbuf.md index 52d5704..4685cc4 100644 --- a/text/2018-03-14-circbuf.md +++ b/text/2018-03-14-circbuf.md @@ -121,8 +121,8 @@ Not that I'm aware of now. [crossbeam-deque]: https://github.com/crossbeam-rs/crossbeam-deque -[crossbeam-deque]: https://github.com/rayon-rs/rayon/pull/528 -[crossbeam-tokio]: https://github.com/tokio-rs/tokio/pull/185 +[crossbeam-deque-rayon]: https://github.com/rayon-rs/rayon/pull/528 +[crossbeam-deque-tokio]: https://github.com/tokio-rs/tokio/pull/185 [go-scheduler]: https://github.com/golang/go/blob/master/src/runtime/proc.go#L4731 [crossbeam-channel-benchmark]: https://github.com/crossbeam-rs/crossbeam-channel/tree/master/benchmarks [crossbeam-circbuf-prototype]: https://github.com/jeehoonkang/concurrent-circbuf/ From f75c7053ee33db2157160afbb96c085252c66c13 Mon Sep 17 00:00:00 2001 From: Jeehoon Kang Date: Wed, 14 Mar 2018 03:02:13 +0900 Subject: [PATCH 3/3] Revise a sentence --- text/2018-03-14-circbuf.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/text/2018-03-14-circbuf.md b/text/2018-03-14-circbuf.md index 4685cc4..0f74b89 100644 --- a/text/2018-03-14-circbuf.md +++ b/text/2018-03-14-circbuf.md @@ -24,7 +24,8 @@ It is worth noting that [Go's scheduler also uses circular buffer, basically][go ## Changes from deque -Besides removing `pop()`, we renamed things and added three methods to circular buffer: +Besides removing `pop()`, we renamed existing structs and methods and added a few methods to +circular buffer: ### Name changes