Under specific circumstances one step of the simulator can take 800 seconds

[medrimonia]

We experienced at least twice that one single tick of the simulator took 800 seconds.

An example of this is can be found at 12:38:224 of K-RR-A3, CIT Brains vs RoboFEI where nothing special seems to be appearing.

[2944.414|0757.320] Info: STATUS: Avg speed factor: 0.267 (over last 20.04 seconds)
[2944.414|0757.320] Info: STATUS: state: STATE_READY, remaining time: 600
[3751.897|0758.224] Info: STATUS: Avg speed factor: 0.001 (over last 807.48 seconds)
[3751.897|0758.224] Info: STATUS: state: STATE_READY, remaining time: 600
[3771.945|0763.264] Info: STATUS: Avg speed factor: 0.251 (over last 20.05 seconds)
[3771.945|0763.264] Info: STATUS: state: STATE_READY, remaining time: 600

The same error occurred at 12:38:360 in K-RR-E1, MRL-HSL vs RoboFEI in another state of the game.

[3233.029|0755.728] Info: STATUS: Avg speed factor: 0.234 (over last 20.01 seconds)
[3233.029|0755.728] Info: STATUS: state: STATE_PLAYING, remaining time: 572
[3240.819|0757.536] Info: Ball touched again by red player 3.
[4031.769|0758.360] Info: STATUS: Avg speed factor: 0.003 (over last 798.74 seconds)
[4031.769|0758.360] Info: STATUS: state: STATE_PLAYING, remaining time: 570
[4051.827|0762.656] Info: STATUS: Avg speed factor: 0.214 (over last 20.06 seconds)
[4051.827|0762.656] Info: STATUS: state: STATE_PLAYING, remaining time: 565

I'm wondering, whether it's possible that it's related with an error related to cache that we obtain systematically:

[30230:30255:0625/023527.458433:ERROR:disk_cache.cc(184)] Unable to create cache
[30230:30255:0625/023527.458488:ERROR:shader_disk_cache.cc(606)] Shader Cache Creation failed: -2
[30266:30274:0625/023527.489241:ERROR:simple_backend_impl.cc(81)] Failed to create directory: /home/ubuntu/.cache/Cyberbotics/Webots/QtWebEngine/Default/Cache
[30266:30274:0625/023527.489299:ERROR:simple_backend_impl.cc(81)] Failed to create directory: /home/ubuntu/.cache/Cyberbotics/Webots/QtWebEngine/Default/Cache
[30266:30274:0625/023527.489313:ERROR:simple_backend_impl.cc(757)] Simple Cache Backend: wrong file structure on disk: 1 path: /home/ubuntu/.cache/Cyberbotics/Webots/QtWebEngine/Default/Cache
[30266:30274:0625/023527.489384:ERROR:simple_backend_impl.cc(81)] Failed to create directory: /home/ubuntu/.cache/Cyberbotics/Webots/QtWebEngine/Default/Cache
[30266:30274:0625/023527.489416:ERROR:simple_backend_impl.cc(81)] Failed to create directory: /home/ubuntu/.cache/Cyberbotics/Webots/QtWebEngine/Default/Cache
[30266:30274:0625/023527.489426:ERROR:simple_backend_impl.cc(757)] Simple Cache Backend: wrong file structure on disk: 1 path: /home/ubuntu/.cache/Cyberbotics/Webots/QtWebEngine/Default/Cache
[30266:30270:0625/023527.489448:ERROR:disk_cache.cc(184)] Unable to create cache

[omichel]

I have never seen this error before... But that could indeed explain the problem. Are you sure you have sufficient space on the disk?

[medrimonia]

Yes I am certain we have sufficient space on the disk, three things I can already mention is that:

1. We also had this error on other computers, the common factor was just running the game over ssh (when we run webots locally, we don't have this issue)
2. Looking at old logs, we have been having this message from the beginning
3. It's not crashing and the 800 seconds appeared only with games involving the RoboFEI model until know and only once, so it's not gamebreaking for us, but still quite curious.

[omichel]

Yes, what is really strange is that it happens only with one model of robot... Can we download this model somewhere to try to reproduce the problem?

[medrimonia]

Here is a link to the RoboFEI robot model.

Note that they tried moving to an 'embedded' (i.e not regenerating) version of the proto, but that led to crash of the referee. So they reverted in order to at least be able to participate.