testlib: Workaound mysterious disappearing buttons

[mvollmer]

No description provided.

[mvollmer]

@martinpitt, @jelly, fyi. This is magic...

[mvollmer]

Ok, this shows how the button went from white to blue just by taking the second screenshot. (The "reference image" in this run is the first screenshot, the "new pixels" are the second screenshot.

This is just the page actually changing from one to the other, in parts that are normally ignored. (The plots and IO rate in this case)

So maybe we shouldn't loudly warn if the two screenshots are different.

[mvollmer]

This shows the same in cockpit-files. The second screenshot has the blue background, the first is white.

[mvollmer]

@martinpitt, this mysteriously seems to work... It gets rid of the SuperUser-testBasic disappearing button flake, and the TestFiles.testPermissions one.

[martinpitt]

Wow, what a hack.. I am worried though about making all screenshots twice, that's quite some overhead? Could we perhaps hide that under some chromium_hack=False option and only enable it for the two problematic cases? Then at least we retain something greppable, and avoid that this bug happens more often/in more situations.

[mvollmer]

Could we perhaps hide that under some chromium_hack=False option and only enable it for the two problematic cases?

Yes, certainly.