From 5b3ef98065b7e590666f7a22d4f165cab5b156d6 Mon Sep 17 00:00:00 2001 From: WalterMadelim <143272683+WalterMadelim@users.noreply.github.com> Date: Thu, 18 Sep 2025 16:59:25 +0800 Subject: [PATCH 1/5] Update parallelism.md The original description is not comprehensive. I add a quote from Julia's doc. --- docs/src/tutorials/algorithms/parallelism.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/src/tutorials/algorithms/parallelism.md b/docs/src/tutorials/algorithms/parallelism.md index 52eacd76690..35721fd366b 100644 --- a/docs/src/tutorials/algorithms/parallelism.md +++ b/docs/src/tutorials/algorithms/parallelism.md @@ -90,8 +90,10 @@ julia> ids ### Thread safety When working with threads, you must avoid race conditions, in which two -threads attempt to write to the same variable at the same time. In the above -example we avoided a race condition by using `ReentrantLock`. See the +threads attempt to write to the same variable at the same time. Also be +very careful about reading any data if another thread might write to it, +as it could result in segmentation faults or worse. In the above example +we avoided a race condition by using `ReentrantLock`. See the [Multi-threading](https://docs.julialang.org/en/v1/manual/multi-threading/) section of the Julia documentation for more details. From b011e14f9804aa3a54de4d244c7ad8c0071be7c7 Mon Sep 17 00:00:00 2001 From: Oscar Dowson Date: Fri, 19 Sep 2025 09:53:31 +1200 Subject: [PATCH 2/5] Update thread safety section for clarity Clarify the explanation of race conditions and thread safety. --- docs/src/tutorials/algorithms/parallelism.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/src/tutorials/algorithms/parallelism.md b/docs/src/tutorials/algorithms/parallelism.md index 35721fd366b..0a0cb464284 100644 --- a/docs/src/tutorials/algorithms/parallelism.md +++ b/docs/src/tutorials/algorithms/parallelism.md @@ -89,11 +89,11 @@ julia> ids ### Thread safety -When working with threads, you must avoid race conditions, in which two -threads attempt to write to the same variable at the same time. Also be -very careful about reading any data if another thread might write to it, -as it could result in segmentation faults or worse. In the above example -we avoided a race condition by using `ReentrantLock`. See the +When working with threads, you must avoid data races. A data race is when +multiple threads access the same variable at the same time, at least one +modifies it, and the accesses are not coordinated by synchronization. In the +`ids` example above, all threads attempt to modify the `ids` vector, but we +avoided a race condition by using `ReentrantLock`. See the [Multi-threading](https://docs.julialang.org/en/v1/manual/multi-threading/) section of the Julia documentation for more details. From 82414af8f326e1f58991aecc3356fe781882c0ea Mon Sep 17 00:00:00 2001 From: Oscar Dowson Date: Fri, 19 Sep 2025 12:02:17 +1200 Subject: [PATCH 3/5] Revise data races section and JuMP thread safety Clarified the explanation of data races and provided examples of incorrect multi-threading usage with JuMP models. --- docs/src/tutorials/algorithms/parallelism.md | 76 +++++++++++++++++--- 1 file changed, 65 insertions(+), 11 deletions(-) diff --git a/docs/src/tutorials/algorithms/parallelism.md b/docs/src/tutorials/algorithms/parallelism.md index 0a0cb464284..ae9cc003e76 100644 --- a/docs/src/tutorials/algorithms/parallelism.md +++ b/docs/src/tutorials/algorithms/parallelism.md @@ -87,19 +87,73 @@ julia> ids For more information, read [PSA: Thread-local state is no longer recommended](https://julialang.org/blog/2023/07/PSA-dont-use-threadid/). -### Thread safety - -When working with threads, you must avoid data races. A data race is when -multiple threads access the same variable at the same time, at least one -modifies it, and the accesses are not coordinated by synchronization. In the -`ids` example above, all threads attempt to modify the `ids` vector, but we -avoided a race condition by using `ReentrantLock`. See the -[Multi-threading](https://docs.julialang.org/en/v1/manual/multi-threading/) +### Data races + +When working with threads, you must avoid data races. A data race occurs when +multiple threads access the same variable at the same time, at least one thread +modifies the variable, and the order of the reads and writes are not properly +coordinated. + +Here's an example of a data race: +````julia +julia> begin + x = Ref(0) + Threads.@threads for i in 1:Threads.nthreads() + for i in 1:1_000 + x[] += 1 + end + end + x[] + end +1106 +```` +The expected answer is `4_000` (because there are four threads each incrementing +1,000 times), the actual result is much smaller. Moreover, the result is +non-deterministic; if we re-ran this code, we would get a different value each +time. + +We got the wrong answer because multiple threads are reading and writing `x[]` +at the same time without coordination. For example, the following sequence +could occur: + + * Assume `x[]` currently has a value of `3` + * Thread A reads `x[]` to get `3` + * Thread B reads `x[]` to get `3` + * Thread A writes `x[] = 3 + 1 = 4` + * Thread B writes `x[] = 3 + 1 = 4` + * The final value of `x[]` is `4` + +The write from Thread A is overwritten, and so the value of `x[]` has increased +by `1` instead of `2`. + +Similar to the earlier `ids` example, we can fix this data race using a +`ReentrantLock`. The lock ensures that only one thread can update (read and then +write) `x` at a time. Now we get the correct answer: +````julia +julia> begin + x = Ref(0) + l = ReentrantLock() + Threads.@threads for i in 1:Threads.nthreads() + for i in 1:1_000 + lock(l) do + x[] += 1 + end + end + end + x[] + end +4000 +```` + +See the [Multi-threading](https://docs.julialang.org/en/v1/manual/multi-threading/) section of the Julia documentation for more details. -JuMP models are not thread-safe. Code that uses multi-threading to -simultaneously modify or optimize a single JuMP model across threads may error, -crash Julia, or silently produce incorrect results. +### JuMP models are not thread-safe + +An object is thread-safe if it can be modified by separate threads without +causing a data race. JuMP models are not thread-safe. Code that uses +multi-threading to simultaneously modify or optimize a single JuMP model +across threads may error, crash Julia, or silently produce incorrect results. For example, the following incorrect use of multi-threading crashes Julia: ```julia From 343203f9b57f48f42a650c7274cbd4742226d168 Mon Sep 17 00:00:00 2001 From: Oscar Dowson Date: Fri, 19 Sep 2025 13:56:24 +1200 Subject: [PATCH 4/5] Add explanation of locking mechanism in parallelism --- docs/src/tutorials/algorithms/parallelism.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/docs/src/tutorials/algorithms/parallelism.md b/docs/src/tutorials/algorithms/parallelism.md index ae9cc003e76..4fd3f0796d8 100644 --- a/docs/src/tutorials/algorithms/parallelism.md +++ b/docs/src/tutorials/algorithms/parallelism.md @@ -108,7 +108,7 @@ julia> begin 1106 ```` The expected answer is `4_000` (because there are four threads each incrementing -1,000 times), the actual result is much smaller. Moreover, the result is +1,000 times), but the actual result is much smaller. Moreover, the result is non-deterministic; if we re-ran this code, we would get a different value each time. @@ -145,6 +145,20 @@ julia> begin 4000 ```` +With the lock, the sequence of events goes something like: + + * Assume `x[]` currently has a value of `3` + * Thread A acquires the lock + * Thread A reads `x[]` to get `3` + * Thread B asks for the lock, but is denied because A is currently using it + * Thread A writes `x[] = 3 + 1 = 4` + * Thread A releases the lock + * Thread B acquires the lock + * Thread B reads `x[]` to get `4` + * Thread B writes `x[] = 4 + 1 = 5` + * Thread B releases the lock + * The final value of `x[]` is `5` + See the [Multi-threading](https://docs.julialang.org/en/v1/manual/multi-threading/) section of the Julia documentation for more details. From 2765723d6306e105e0c904a49923ed814adf1592 Mon Sep 17 00:00:00 2001 From: Oscar Dowson Date: Sat, 20 Sep 2025 12:59:24 +1200 Subject: [PATCH 5/5] Update docs/src/tutorials/algorithms/parallelism.md --- docs/src/tutorials/algorithms/parallelism.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/tutorials/algorithms/parallelism.md b/docs/src/tutorials/algorithms/parallelism.md index 4fd3f0796d8..b2a49e0b934 100644 --- a/docs/src/tutorials/algorithms/parallelism.md +++ b/docs/src/tutorials/algorithms/parallelism.md @@ -112,7 +112,7 @@ The expected answer is `4_000` (because there are four threads each incrementing non-deterministic; if we re-ran this code, we would get a different value each time. -We got the wrong answer because multiple threads are reading and writing `x[]` +We got the wrong answer because multiple threads are reading and writing `x` at the same time without coordination. For example, the following sequence could occur: