Discussion on Zephyr's Poor Real-Time Performance

[lenghonglin]

Recently, the well-known embedded systems expert Jacob Beningo wrote the '2024 Real-Time Operating System Performance Report.' The report tested several commonly used RTOSes, including PX5, ThreadX, Zephyr, and FreeRTOS, to evaluate real-time performance through test cases such as coroutine scheduling, preemptive scheduling, memory allocation, message handling, and synchronization processing. Based on the results, Zephyr's real-time performance was nearly the worst. What could be the reason for this?

Beningo-RtosPerformance-2024-Report.pdf

[cocoeoli]

From this result, Zephyr still has a lot of space for performance optimization.

[heinwessels]

You should read the follow up article by the same person: https://www.beningo.com/is-rtos-performance-affected-by-an-mpu/#

Essentially, in the initial benchmark you posted Zephyr had a Memory Protection Unit enabled (MPU) enabled, which has an performance impact because it adds safety features.

Quote from the article:

Zephyr jumped from a relative performance against PX5 RTOS of 42% to 70.1%! That’s a 28% improvement in the benchmark and a nearly 67% improvement against itself! All because the MPU and a few additional checks were disabled!
You might argue that disabling the MPU in the benchmark is cheating, but since the other RTOSes in the study did not have them enabled, it really levels the playing field and makes it possible to compare the RTOSes more accurately.

[aniltirli]

As mentioned in that article Zephyr's defaults are really MPU, asserttions and error-checking enabled ? I am not sure about that, in my understanding from the documentation, i think you need to set those in config file to enable them, am i wrong ?

[nashif]

quick update as things improve in zephyr and some of the issues are being addressed. Here is the state of things:

++ with changes currently in review and pending merge.

[DBS06]

@nashif Hi, is it possible to have access to the test code which you are using?
I am doing right now a similar benchmarks for different RTOSes and I am curious to see the configuration and the test code.

[henrikbrixandersen]

@DBS06 Have you seen https://github.com/zephyrproject-rtos/zephyr/tree/main/tests/benchmarks/thread_metric ?

[DBS06]

@DBS06 Have you seen https://github.com/zephyrproject-rtos/zephyr/tree/main/tests/benchmarks/thread_metric ?

Thanks a lot!!!
Actually I am aware of zephyr/tests/benchmarks/ but I wasn't aware of this specific one. I started my benchmark-project 3 months ago and had to stop it because other things were more urgent.
So thanks a lot!

[nashif]

Update just before feature freeze of 4.1..... (c8f640a)

Few changes currently in review would improve things even more, stay tuned.