Skip to content

Commit 8ca47df

Browse files
committed
fix(io_context): decouple concurrency_hint == 1 from lockless mode (#310)
A concurrency_hint of 1 silently enabled full single-threaded lockless mode: scheduler mutex/condvar, per-descriptor locks, IOCP dispatch mutex, and io_uring ring/dispatch mutexes were all disabled, plus io_uring got SINGLE_ISSUER/DEFER_TASKRUN. This made cross-thread post() undefined behavior, contrary to asio (where a plain integer hint never disables locking) and to what migrating users expect. A hint now only tunes performance and never changes the safety contract: - Remove the three hint == 1 -> single_threaded coupling sites (backend-tag template ctor, io_context(unsigned) ctor, and normalize_options, which is deleted along with the now-dead configure_single_threaded_() member). - Rename the public opt-in io_context_options::single_threaded -> single_threaded_lockless, which remains the sole way to enable lockless mode (still gating the io_uring SINGLE_ISSUER flags). The explicit, descriptive name replaces an innocuous flag that read like a mere performance tweak. - Default ctor clamp max(2u, hardware_concurrency()) -> max(1u, ...): the "2" floor existed only to dodge the removed trap; floor at 1 still guards a 0 return. Tests: the resolver/stream_file single-threaded restriction tests switch to explicit opts.single_threaded_lockless; new testHintOneIsThreadSafe proves cross-thread post() into a hint == 1 context is TSan-clean across all backends (verified: fails under TSan with the coupling restored). Docs and benchmarks: header Doxygen (option, Thread Safety, default ctor), the io-context and configuration guide pages, and all perf/bench/corosio sources updated for the new option name. Fixes #310
1 parent 19d76f3 commit 8ca47df

17 files changed

Lines changed: 131 additions & 86 deletions

‎doc/modules/ROOT/pages/4.guide/4c.io-context.adoc‎

Lines changed: 16 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -53,16 +53,17 @@ corosio::io_context ioc;
5353
----
5454

5555
Creates an `io_context` with a concurrency hint of
56-
`std::max(2u, std::thread::hardware_concurrency())` — the default constructor
57-
never selects single-threaded mode. Single-threaded (lockless) mode is keyed
58-
on a `concurrency_hint` of exactly `1`; pass `1` explicitly to opt into it.
56+
`std::max(1u, std::thread::hardware_concurrency())`. The concurrency hint only
57+
tunes performance and never changes the safety contract; single-threaded
58+
(lockless) mode is opt-in exclusively via the `single_threaded_lockless` option
59+
(see xref:4.guide/4c2.configuration.adoc#single-threaded-mode[Single-Threaded Mode]).
5960

6061
=== With Concurrency Hint
6162

6263
[source,cpp]
6364
----
64-
corosio::io_context ioc(1); // Single-threaded, no synchronization
65-
corosio::io_context ioc(4); // Up to 4 threads, thread-safe
65+
corosio::io_context ioc(1); // one-thread hint, still fully thread-safe
66+
corosio::io_context ioc(4); // up to 4 threads
6667
----
6768

6869
The concurrency hint affects:
@@ -73,7 +74,10 @@ The concurrency hint affects:
7374
concurrently. It is not a library-managed thread pool, and it is distinct
7475
from `thread_pool_size`.
7576

76-
Use `1` for single-threaded programs to avoid synchronization overhead.
77+
The hint never disables locking — a context built with `concurrency_hint == 1`
78+
remains fully thread-safe, so other threads may `post()` into it. To drop
79+
synchronization overhead in a genuinely single-threaded program, set
80+
`io_context_options::single_threaded_lockless = true` instead.
7781

7882
== Running the Event Loop
7983

@@ -245,8 +249,9 @@ int main()
245249

246250
== Thread Safety
247251

248-
The `io_context` can be used from multiple threads when constructed with
249-
a concurrency hint greater than 1:
252+
The `io_context` is thread-safe by default, regardless of the concurrency hint:
253+
multiple threads may call `run()` concurrently, and any thread may `post()` work
254+
into it.
250255

251256
[source,cpp]
252257
----
@@ -261,7 +266,9 @@ for (auto& t : threads)
261266
----
262267

263268
Multiple threads can call `run()` concurrently. The `io_context` distributes
264-
work across threads.
269+
work across threads. The one exception is lockless mode: constructing with
270+
`io_context_options::single_threaded_lockless = true` drops these guarantees (see
271+
xref:4.guide/4c2.configuration.adoc#single-threaded-mode[Single-Threaded Mode]).
265272

266273
WARNING: Individual I/O objects (sockets, timers) are not thread-safe.
267274
Don't access the same socket from multiple threads without synchronization.

‎doc/modules/ROOT/pages/4.guide/4c2.configuration.adoc‎

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -80,12 +80,14 @@ corosio::native_io_context<corosio::epoll> ioc(opts);
8080
blocking file I/O and DNS resolution. Ignored on IOCP where
8181
file I/O uses native overlapped I/O.
8282

83-
| `single_threaded`
83+
| `single_threaded_lockless`
8484
| false
8585
| all
8686
| Disable all scheduler mutex and condition variable operations.
8787
Eliminates synchronization overhead when only one thread calls
88-
`run()`. See <<single-threaded-mode>> for restrictions.
88+
`run()`. The only way to enable lockless mode; the concurrency
89+
hint never disables locking. See <<single-threaded-mode>> for
90+
restrictions.
8991
|===
9092

9193
Options that do not apply to the active backend are silently ignored.
@@ -144,16 +146,23 @@ and DNS resolution use a shared thread pool.
144146
* *No file I/O*: leave at 1 (the pool is created lazily).
145147

146148
[#single-threaded-mode]
147-
=== Single-Threaded Mode (`single_threaded`)
149+
=== Single-Threaded Mode (`single_threaded_lockless`)
148150

149151
Disables all mutex and condition variable operations inside the
150152
scheduler and per-socket descriptor states. This eliminates
151153
15-25% of overhead on the post-and-dispatch hot path.
152154

155+
Setting `single_threaded_lockless = true` is the *only* way to enable
156+
lockless mode. The `concurrency_hint` passed at construction never
157+
disables locking — in particular, `concurrency_hint == 1` keeps the
158+
context fully thread-safe (unlike some other libraries, where a hint
159+
of `1` silently elides synchronization). Enabling this option
160+
transfers responsibility for the restrictions below to the caller.
161+
153162
[source,cpp]
154163
----
155164
corosio::io_context_options opts;
156-
opts.single_threaded = true;
165+
opts.single_threaded_lockless = true;
157166
158167
corosio::io_context ioc(opts);
159168
ioc.run(); // only one thread may call this

‎include/boost/corosio/backend.hpp‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -359,7 +359,10 @@ struct iocp_t
359359
360360
@param ctx The execution context that owns the scheduler.
361361
@param concurrency_hint Hint for the number of threads that
362-
will call `run()`. Pass `1` for single-threaded mode.
362+
will call `run()`. Only tunes performance; it never
363+
changes the safety contract. Single-threaded (lockless)
364+
mode is opt-in exclusively via
365+
@ref io_context_options::single_threaded_lockless.
363366
364367
@return Reference to the newly created scheduler.
365368
*/

‎include/boost/corosio/io_context.hpp‎

Lines changed: 23 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -97,26 +97,33 @@ struct io_context_options
9797
*/
9898
unsigned thread_pool_size = 1;
9999

100-
/** Enable single-threaded mode (disable scheduler locking).
100+
/** Enable single-threaded, lockless mode (disable scheduler locking).
101101
102102
When true, the scheduler skips all mutex lock/unlock and
103103
condition variable operations on the hot path. This
104104
eliminates synchronization overhead when only one thread
105105
calls `run()`.
106106
107+
Enabling this drops the io_context's thread-safety guarantees:
108+
the caller takes on responsibility for the restrictions below.
109+
Leaving it `false` (the default) keeps the context fully
110+
thread-safe.
111+
107112
@par Restrictions
108113
- Only one thread may call `run()` (or any run variant).
109114
- Posting work from another thread is undefined behavior.
110115
- DNS resolution returns `operation_not_supported`.
111116
- POSIX file I/O returns `operation_not_supported`.
112117
- Signal sets should not be shared across contexts.
113118
114-
@note Constructing an `io_context` with `concurrency_hint == 1`
115-
automatically enables single-threaded mode regardless of
116-
this field's value, matching asio's convention. To opt out,
117-
pass `concurrency_hint > 1`.
119+
@note This field is the *only* way to enable lockless mode. The
120+
`concurrency_hint` passed at construction never changes the
121+
safety contract — it only tunes performance. In particular,
122+
`concurrency_hint == 1` keeps full thread safety (unlike
123+
asio's named `ASIO_CONCURRENCY_HINT_UNSAFE` constants, which
124+
this field mirrors).
118125
*/
119-
bool single_threaded = false;
126+
bool single_threaded_lockless = false;
120127

121128
/** Enable IORING_SETUP_SQPOLL on the io_uring backend.
122129
@@ -126,7 +133,7 @@ struct io_context_options
126133
path. Most useful for sustained traffic. Idle thread parks
127134
after `sq_thread_idle_ms` of no activity.
128135
129-
Independent of `single_threaded`. Default: off.
136+
Independent of `single_threaded_lockless`. Default: off.
130137
131138
Ignored on non-io_uring backends.
132139
*/
@@ -196,8 +203,10 @@ class timer_service;
196203
197204
@par Thread Safety
198205
Distinct objects: Safe.@n
199-
Shared objects: Safe, if using a concurrency hint greater
200-
than 1.
206+
Shared objects: Safe, unless the context was constructed with
207+
`io_context_options::single_threaded_lockless = true` (lockless
208+
mode), in which case only one thread may use it. The
209+
`concurrency_hint` does not affect thread safety.
201210
202211
@see epoll_t, select_t, kqueue_t, iocp_t
203212
*/
@@ -211,9 +220,6 @@ class BOOST_COROSIO_DECL io_context : public capy::execution_context
211220
io_context_options const& opts,
212221
unsigned concurrency_hint);
213222

214-
/// Switch the scheduler to single-threaded (lockless) mode.
215-
void configure_single_threaded_();
216-
217223
protected:
218224
detail::scheduler* sched_;
219225

@@ -223,11 +229,11 @@ class BOOST_COROSIO_DECL io_context : public capy::execution_context
223229

224230
/** Construct with default concurrency and platform backend.
225231
226-
Uses `std::thread::hardware_concurrency()` clamped to a minimum
227-
of 2 as the concurrency hint, so the default constructor never
228-
silently engages single-threaded mode (see
229-
@ref io_context_options::single_threaded). Pass an explicit
230-
`concurrency_hint == 1` to opt into single-threaded mode.
232+
Uses `std::thread::hardware_concurrency()` (floored to 1, in
233+
case it reports 0) as the concurrency hint. The hint only tunes
234+
performance and never affects thread safety; single-threaded
235+
(lockless) mode is opt-in exclusively via
236+
@ref io_context_options::single_threaded_lockless.
231237
*/
232238
io_context();
233239

@@ -266,8 +272,6 @@ class BOOST_COROSIO_DECL io_context : public capy::execution_context
266272
{
267273
(void)backend;
268274
sched_ = &Backend::construct(*this, concurrency_hint);
269-
if (concurrency_hint == 1)
270-
configure_single_threaded_();
271275
}
272276

273277
/** Construct with an explicit backend tag and runtime options.

‎perf/bench/corosio/accept_churn_bench.cpp‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -132,7 +132,7 @@ bench_sequential_churn_lockless(bench::state& state)
132132
using acceptor_type = corosio::native_tcp_acceptor<Backend>;
133133

134134
corosio::io_context_options opts;
135-
opts.single_threaded = true;
135+
opts.single_threaded_lockless = true;
136136
corosio::native_io_context<Backend> ioc(opts, 1);
137137
acceptor_type acc(ioc);
138138
acc.open();
@@ -397,7 +397,7 @@ bench_burst_churn_lockless(bench::state& state)
397397
state.counters["burst_size"] = burst_size;
398398

399399
corosio::io_context_options opts;
400-
opts.single_threaded = true;
400+
opts.single_threaded_lockless = true;
401401
corosio::native_io_context<Backend> ioc(opts, 1);
402402
acceptor_type acc(ioc);
403403
acc.open();

‎perf/bench/corosio/fan_out_bench.cpp‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -336,7 +336,7 @@ bench_fork_join_lockless(bench::state& state)
336336
state.counters["fan_out"] = fan_out;
337337

338338
corosio::io_context_options opts;
339-
opts.single_threaded = true;
339+
opts.single_threaded_lockless = true;
340340
corosio::native_io_context<Backend> ioc(opts, 1);
341341

342342
std::vector<socket_type> clients;
@@ -413,7 +413,7 @@ bench_nested_lockless(bench::state& state)
413413
state.counters["subs_per_group"] = subs_per_group;
414414

415415
corosio::io_context_options opts;
416-
opts.single_threaded = true;
416+
opts.single_threaded_lockless = true;
417417
corosio::native_io_context<Backend> ioc(opts, 1);
418418

419419
std::vector<socket_type> clients;
@@ -508,7 +508,7 @@ bench_concurrent_parents_lockless(bench::state& state)
508508
state.counters["fan_out"] = fan_out;
509509

510510
corosio::io_context_options opts;
511-
opts.single_threaded = true;
511+
opts.single_threaded_lockless = true;
512512
corosio::native_io_context<Backend> ioc(opts, 1);
513513

514514
std::vector<socket_type> clients;

‎perf/bench/corosio/http_server_bench.cpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -179,7 +179,7 @@ bench_single_connection_lockless(bench::state& state)
179179
using socket_type = corosio::native_tcp_socket<Backend>;
180180

181181
corosio::io_context_options opts;
182-
opts.single_threaded = true;
182+
opts.single_threaded_lockless = true;
183183
corosio::native_io_context<Backend> ioc(opts, 1);
184184
auto [client, server] = corosio::test::make_socket_pair<
185185
socket_type, corosio::native_tcp_acceptor<Backend>>(ioc);

‎perf/bench/corosio/io_context_bench.cpp‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -263,7 +263,7 @@ void
263263
bench_single_threaded_lockless(bench::state& state)
264264
{
265265
corosio::io_context_options opts;
266-
opts.single_threaded = true;
266+
opts.single_threaded_lockless = true;
267267

268268
corosio::native_io_context<Backend> ioc(opts, 1);
269269
auto ex = ioc.get_executor();
@@ -295,7 +295,7 @@ void
295295
bench_interleaved_lockless(bench::state& state)
296296
{
297297
corosio::io_context_options opts;
298-
opts.single_threaded = true;
298+
opts.single_threaded_lockless = true;
299299

300300
int handlers_per_iteration = 100;
301301

‎perf/bench/corosio/local_socket_latency_bench.cpp‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -167,7 +167,7 @@ bench_unix_pingpong_latency_lockless(bench::state& state)
167167
state.counters["message_size"] = static_cast<double>(message_size);
168168

169169
corosio::io_context_options opts;
170-
opts.single_threaded = true;
170+
opts.single_threaded_lockless = true;
171171
corosio::native_io_context<Backend> ioc(opts, 1);
172172
socket_type client(ioc), server(ioc);
173173
if (auto ec = corosio::connect_pair(client, server))
@@ -201,7 +201,7 @@ bench_unix_concurrent_latency_lockless(bench::state& state)
201201
state.counters["num_pairs"] = num_pairs;
202202

203203
corosio::io_context_options opts;
204-
opts.single_threaded = true;
204+
opts.single_threaded_lockless = true;
205205
corosio::native_io_context<Backend> ioc(opts, 1);
206206

207207
std::vector<socket_type> clients;

‎perf/bench/corosio/local_socket_throughput_bench.cpp‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -195,7 +195,7 @@ bench_unix_throughput_lockless(bench::state& state)
195195
state.counters["chunk_size"] = static_cast<double>(chunk_size);
196196

197197
corosio::io_context_options opts;
198-
opts.single_threaded = true;
198+
opts.single_threaded_lockless = true;
199199
corosio::native_io_context<Backend> ioc(opts, 1);
200200
socket_type writer(ioc), reader(ioc);
201201
if (auto ec = corosio::connect_pair(writer, reader))
@@ -259,7 +259,7 @@ bench_unix_bidirectional_throughput_lockless(bench::state& state)
259259
state.counters["chunk_size"] = static_cast<double>(chunk_size);
260260

261261
corosio::io_context_options opts;
262-
opts.single_threaded = true;
262+
opts.single_threaded_lockless = true;
263263
corosio::native_io_context<Backend> ioc(opts, 1);
264264
socket_type sock1(ioc), sock2(ioc);
265265
if (auto ec = corosio::connect_pair(sock1, sock2))

0 commit comments

Comments
 (0)