SPI speed ~2x slower than it should be on RPi 4

[gtrainavicius]

The SPI transfer speed used on RPi4 is 2 times slower (50kHz) than requested for example here: https://github.com/raspberrypi/linux/blob/rpi-4.19.y/sound/soc/bcm/pisound.c#L329 (100kHz)

On RPi4, requesting 100000 Hz produces 50kHz communcation speed. Changing the requested speed to 200000 Hz, on RPi4 it first starts at 80kHz, then at some point it switches to 100kHz, so that means RPi4 is capable of using exactly 100kHz.

On RPi3B+, requesting 100000 yields 62.5kHz speed. Requesting 200000 gives 125kHz. I assume RPi3B+ does not have in-between speeds like RPi4.

The question (and bug) is - why does RPi4 pick 50kHz when requested 100kHz instead, even though it can be seen being capable of 80kHz and 100kHz speeds.

To reproduce
Without any hat connected, you may run spidev_test to test drive /dev/spidev.0.0. Either an oscilloscope or logic analyzer can be connected to SCK line to measure the frequency, or it can very roughly be inferred based on the reported data rate and the duration of execution. Here are test results of sending 80000 bytes in 16 byte transfers using 100kHz and 200kHz requested speeds on RPi4 and 3B+, they contain SPI logic analyzer timed output, as well as the spidev_test invocation, results and additional notes at the very bottom.

spi_test_results.zip

100kHz RPi4

pi@raspberrypi:~/spidev_test $ time { ./spidev_test -S 16 -I 5000 -D /dev/spidev0.0 -s 100000; }
spi mode: 0x0
bits per word: 8
max speed: 100000 Hz (100 KHz)
rate: tx 41.1kbps, rx 41.1kbps
rate: tx 41.8kbps, rx 41.8kbps
rate: tx 41.8kbps, rx 41.8kbps
total: tx 78.1KB, rx 78.1KB

real    0m18.392s
user    0m0.056s
sys     0m0.286s

Actual transfer rate: 4351.44 B/s
Theoretical 50kHz rate: 6250 B/s
Theoretical 100kHz rate: 12500 B/s

Logic analyzer measured SCK rate: 40kHz

100kHz RPi3B+

pi@raspberrypi:~/spidev_test $ time { ./spidev_test -S 16 -I 5000 -D /dev/spidev0.0 -s 100000; }
spi mode: 0x0
bits per word: 8
max speed: 100000 Hz (100 KHz)
rate: tx 54.4kbps, rx 54.4kbps
rate: tx 64.8kbps, rx 64.8kbps
total: tx 78.1KB, rx 78.1KB

real    0m11.838s
user    0m0.050s
sys     0m0.214s

Actual transfer rate: 6763.58 B/s
Theoretical 50kHz rate: 6250 B/s
Theoretical 100kHz rate: 12500 B/s

Logic analyzer measured SCK rate: 62.5kHz

200kHz RPi4

pi@raspberrypi:~/spidev_test $ time { ./spidev_test -S 16 -I 5000 -D /dev/spidev0.0 -s 200000; }
spi mode: 0x0
bits per word: 8
max speed: 200000 Hz (200 KHz)
rate: tx 69.1kbps, rx 69.1kbps
total: tx 78.1KB, rx 78.1KB

real    0m9.369s
user    0m0.040s
sys     0m0.312s

Actual transfer rate: 8544.44 B/s
Theoretical 50kHz rate: 6250 B/s
Theoretical 100kHz rate: 12500 B/s

Logic analyzer measured SCK rate: 80kHz

200kHz RPi3B+

pi@raspberrypi:~/spidev_test $ time { ./spidev_test -S 16 -I 5000 -D /dev/spidev0.0 -s 200000; }
spi mode: 0x0
bits per word: 8
max speed: 200000 Hz (200 KHz)
rate: tx 125.1kbps, rx 125.1kbps
total: tx 78.1KB, rx 78.1KB

real    0m6.120s
user    0m0.045s
sys     0m0.232s

Actual transfer rate: 13086.32 B/s
Theoretical 50kHz rate: 6250 B/s
Theoretical 100kHz rate: 12500 B/s

Logic analyzer measured SCK rate: 125kHz

As can be seen from the results of same command being executed on RPi3B+ and RPi4, the tests finish significantly faster on RPi3B+.

Expected behaviour
100kHz gets picked when requesting 100kHz speed, or at least something closer like 80kHz.

System
RPi 3B+ and RPi 4 using latest Raspbian Lite running:

Linux raspberrypi 4.19.88-v7+ #1284 SMP Wed Dec 11 13:46:41 GMT 2019 armv7l GNU/Linux

[popcornmix]

The SPI clock is a divided down version of the core clock.
The core clock reduces when arm is not busy.

If you want to force the full speed of SPI you need to about core clock from clocking down.
There are a few ways of doing this:
In config.txt

force_turbo=1

or

core_freq_min=500
core_freq=500

Or from the arm:

sudo sh -c "echo performance > cpu0/cpufreq/scaling_governor" .

[gtrainavicius]

Is 50 kHz the best it can do when slowed down? In all cases when running the test, the system was otherwise idle - if requesting 200kHz, the system managed to run SPI at 80kHz, this seems like a bug - it should be able to get something closer to 100kHz when being asked for 100kHz...

[popcornmix]

The core clock changes outside of the spi driver's knowledge.
Therefore it assumes the highest frequency and will run slower when the core runs slower.

Typically core_freq=500 core_freq_min=200.
So when idle you will get 2/5 of the SPI frequency you requested.

[pelwell]

The SPI interfaces (and I2C and UART1) share the same clock as the VPU cores; if the VPU clock frequency changes, so does the SPI clock. The Linux SPI driver is unaware of these clock changes, so to avoid a bus speed which is too high the clock divisor is calculated for the turbo speed, but when running at the normal speed the divisor is too high with the result that the bus clock is too slow. Locking the VPU/core clock to a fixed value allows the divisor to be correctly calculated without affecting the ARM clock speeds.

[edo1]

What are default frequencies for RPi 3B+ and RPi 4?
I use gpu_freq=250, is it enough to have stable SPI frequency on PRi 3B+? Or I have to specify core_freq_min as well?
What about RPi 4?

Does force_turbo=1 affect system stability/overheating?

[popcornmix]

Pi4 uses 200MHz for and and all older Pi's 250MHz.

gpu_freq=250 will have stable frequency on Pi0-3.

Does force_turbo=1 affect system stability/overheating?

You'll have marginally higher temperatures due to the higher clock when idle, and no change when arm is busy.

[gtrainavicius]

I see, I assumed the communication or some other drivers would be handling clock changes not to impact the communication too much.

Is there something that can be added to a device tree overlay EEPROM to ensure a minimum clock speed, so we can make sure that the SPI communication is within minimum and maximum allowable range?

[pelwell]

See raspberrypi/firmware#1308.

[marckleinebudde]

FYI: there are two functions that might be of interest here. You can register a notifier for clock and/or cpu frequenctychange
https://elixir.bootlin.com/linux/v5.5.7/source/drivers/cpufreq/cpufreq.c#L341
https://elixir.bootlin.com/linux/v5.5.7/source/drivers/clk/clk.c#L4136

[lurch]

@marckleinebudde I suspect those are for monitoring CPU-side clock changes, whereas the issue here is GPU-side clock changes (since the SPI, I2C and UART1 peripherals are running off the VPU clock rather than the CPU clock, as @pelwell describes above).

[marckleinebudde]

@lurch I'm not familiar with the raspi clock tree, but if the GPU-side clock is not mapped to the linux clock framework, this doesn't work.