[Epic] Flaky tests roundup #9080
Comments
MartinZikmund
added
kind/bug
This test appears to be Wasm only. I'm disabling in #9571 |
|
Images for When_Transforms_Paused_With_FillBehaviorStop_Then_Hold: Teardown: |
jeromelaban
added
the
area/tests 🧪
GitHub Issue (If applicable): related to #9080 <!-- Link to relevant GitHub issue if applicable. All PRs should be associated with an issue (GitHub issue or internal), unless the change is documentation related. --> ## PR Type What kind of change does this PR introduce? <!-- Please uncomment one or more that apply to this PR - Bugfix - Feature - Code style update (formatting) - Refactoring (no functional changes, no api changes) - Build or CI related changes - Documentation content changes - Project automation - Other... Please describe: --> ## What is the current behavior? <!-- Please describe the current behavior that you are modifying, or link to a relevant issue. --> ## What is the new behavior? <!-- Please describe the new behavior after your modifications. --> ## PR Checklist Please check if your PR fulfills the following requirements: - [ ] Docs have been added/updated which fit [documentation template](https://github.com/unoplatform/uno/blob/master/doc/.feature-template.md) (for bug fixes / features) - [ ] [Unit Tests and/or UI Tests](https://github.com/unoplatform/uno/blob/master/doc/articles/uno-development/working-with-the-samples-apps.md) for the changes have been added (for bug fixes / features) (if applicable) - [ ] Validated PR `Screenshots Compare Test Run` results. - [ ] Contains **NO** breaking changes - [ ] Associated with an issue (GitHub or internal) and uses the [automatic close keywords](https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue). - [ ] Commits must be following the [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/#summary) specification. <!-- If this PR contains a breaking change, please describe the impact and migration path for existing applications below. Please note that breaking changes are likely to be rejected --> ## Other information <!-- Please provide any additional information if necessary --> Internal Issue (If applicable): <!-- Link to relevant internal issue if applicable. All PRs should be associated with an issue (GitHub issue or internal) -->
|
|
|
Edit: It appears that |
|
I took a look at
The |
|
We already have code for that, but it is racing with code in msbuild. uno/src/SourceGenerators/Uno.UI.Tasks/Content/Uno.UI.Tasks.targets Lines 33 to 40 in d1d468f
The evaluation from Uno.UI.Tasks.targets was done first (because it's imported first), then it got cleared by msbuild code from Microsoft.Common.CurrentVersion.targets |
ghost
mentioned this issue
Current behavior
This epic tracks flaky tests so they can be addressed one by one.
When a flaky test is added in this list, it can be set to
[Ignore]on the platform or platforms where it tends to fail.When_Large_List_Scroll_To_End_Then_Back_Up_And_First_Itemis flaky in WASM CIWhen_ShapeVisual_ViewBox_Shape_Combinations(When_ShapeVisual_ViewBox_Shape_Combinationsis unstable #18752)When_RectangleUI test is very flaky on iOSBreadcrumbBarleak tests are flaky on iOSGiven_AppWindow.WhenResizeis failing in CI on Linux, passing locallyKnob_Translationfails in CI #18375NSInternalInconsistencyException Reason: Attempted to scroll the collection view to an out-of-bounds item#17102ValidateElementsHaveCorrectKeysis not passing (incorrectly disabled)VerifyKeyboardAcceleratorCanOverrideControlAccelerator,VerifyTryInvokeKeyboardAcceleratorBehaviorForLocallyScopedAccelerator,ValidateKeyboardAcceleratorEventNotInvokedWhenCollapsedandValidateKeyboardAcceleratorBehaviorWithMultipleModifierson Skia are unstableWhen_Image_Is_Monochromaticfails on iOS .NET 9 RC1Given_ImageBrush.WhenStretchfail on iOSWhen_Hungarian_Culture_Column_Order,When_Czech_Culture_Column_OrderandWhen_CanadaFrench_Culture_Column_Orderfail on net9 wasm. It could be related to a missing/filtered ICU, or a headless browser issue, it works locally.MoveItemsBetweenPrimaryAndSecondaryCommandsis unstable on androidWhen_DownSamplingis flaky and disabled in test: Disable flaky test #16991SelectingTimeSetsSelectedTimeis flaky on Wasm #16167When_UnloadAndReload_Then_ReMaterializeItemsWhen_ThemeChangeon WasmTestProgressRing_InitialStatefailsFlyoutTest_UnloadedFlyoutTest_Targetis flaky on iOSGiven_CalendarDatePicker.When_Theme_Changesis flaky on iOS #13417PasswordBox_RevealInScrollVieweris flaky on iOSEventSequence_Tests.TestHyperlinkmisses a click event on android #15734Verify_MenuBarItem_Bounds_Native_Popupsis flaky on Android - started being flaky after pull request fix(flyout): dismiss all flyouts on window focus lost and SizeChanged #13630 (@ramezgerges)TextBox_Readonlyis flaky on iOSWhen_Drag_Then_DoesNotScrollis not working on Android, the thumb drags when it should notCanChangeDisplayModeBeforeLoadedis flaky on SkiaHolding_Tests.When_InListViewWithoutItemClickfails for missing item on androidHolding_Tests.When_Basicfails on androidHolding_Tests.When_InScrollfails on androidHolding_Tests.When_InListViewWithItemClickfails on androidTestTextBoxReleaseOutFocusedis flaky on WASMCheck_ListView_Swallows_Measureis flaky on Android #15829When_MultipleItemsis not failing sometimes (Timing related ?)Flyout_TemplatedParentis flaky on WASMWebViewtests are flaky on iOS #15828When_MediaPlayerElement_SetSource_Check_Playfails on Skia #15471TestTranslatedTapis flaky on WASM (could be related to TestTranslatedClick fails on faster build agents #13844)ValidateOverflowPlacementdoes not pass on AndroidTestTranslatedTapis failing on androidDoesNotShowMenuIfSecondaryElementsAreCollapseddoes not pass on AndroidWhen_NestedInSVAndOutOfViewportOnInitialLoad_Then_MaterializedEvenWhenScrollingOnMinorAxisis flaky on WASMWhen_NestedIRSlowlyChangeViewport_Then_MaterializedNeededItems(Timeout even with retries) (Youssef: Can't repro)When_Verify_Canvas_With_Outer_Clip(Assert.Fail failed. ImageAssert.HasColorAt @ line 73) (Youssef: Verified - it always fails)UITests.Windows_UI_Xaml_Media.GradientBrushTests.GradientsPagecrashes the android emulator completely (not just the app)VerifyNavigationViewItemToolTipPaneDisplayMode(Youssef: Verified - it always crashes locally)VerifyCollectionChangeGetsRespected(CleanupSystem.Exception: Event was not raised.inMUXApiTestBase.Cleanup()) (Youssef: test not disabled on master, and always passes for me)ListView_ObservableCollection_Unused_Spacefails regularly on iOS (Expected: 132.0f But was: 88.0f)When_Open_Close_Root_Twice_Keep_Statefails on android for missing TreeView items (Youssef: fails locally with null ref)When_UsingMultipleLanguagesunit test fails when running with net7 or later (DateFormatter is buggy and CultureInfo itself returns wrong values)ExecuteScriptAsynctests are flaky on iOS #11997NavigationViewInfoBadgeTestPagecauses CI not to pass #13257When_Grid_RowCollection_Changesfails with invalidchild.SizePassedToArrangeOverrideNativeCommandBar_Sizefails (The first orientation change fails to be applied)ListView_SelectedItemsfails (Timeout ?)When_Flipview_DataTemplateSelector(AssertFailedException: Assert.Fail failed. ImageAssert.HasColorAt @ line 101)When_Add_Removefailure:ColorPickerWhen_Add_Removefailure:CalendarViewWhen_ItemsControl.When_NestedItemsControl_RecycleTemplate(Assert.AreEqual failed. Expected a difference no greater than <1> between expected value <50> and actual value <0>.)RepeaterTests.VerifyCorrectionsInNonScrollableDirectionVerifyFocusedItemIsRecycledOnCollectionReset(Skia fixed in test: enable working tests and add ItemsRepeater fixes for RepeaterTests #13074)When_WebMessageReceived(Timed out waiting for condition to be met. When_WebMessageReceived():207)When_Flipview_Items_Modified(Timed out waiting for equality condition to be met. Expected -1 but last received value was 0.)When_Render_Border_GetPixelsAsync(Assert.AreEqual failed. Expected:<<10, 10>>. Actual:<<10.5, 10>>. Invalid Layouted.)ValidateFractionalWidthDoesNotCrash(Assert.Fail failed. Timeout waiting on SpectrumRectangle.Fill to be set.) (Youssef: Test is actually skipped in WinUI. There is not much info on why it's skipped there, but I don't see anything that should trigger property changed for Fill)When_StretchAndAlignmenton Android 28 (Youssef: Passing for me locally)TextBox_Readonly_ShouldNotBringUpKeyboardon Android 30 (The keyboard does not show up on emulators, breaking the test)When_PressOnContainerAndReleaseOnNested_Mouseon WASM (fails in CI)When_RepeatDuration: It was enabled in chore: enable When_RepeatDuration on skia #13060, but then failures are found and reverted in Revert "chore: enable When_RepeatDuration on skia" #13089MediaPlayertests are failing too often #13384RepeaterTests.VerifyRepeaterDoesNotLeakItemContainers(Skia started failing in build: Move build to net8.0/net9.0 instead of net8.0/net7.0 #16523)In Progress
TestTextBoxReleaseOutFocusedis flaky on WASMTestTextBoxTapis very flaky on WASMGiven_SymbolIcon.Validate_Sizefails on all targets (Expected:<20>. Actual:<627.5>) (fix: Fix Given_SymbolIcon.Validate_Size test failure #13069)VerifyDropdownItemTemplateWithNoControl(failed cleanupSystem.Exception: Event was not raised.) (Youssef: passing for me - test: Enable skipped BreadcrumbTests on Android #13077)VerifyBreadcrumbDefaultAPIValues(failed cleanupSystem.Exception: Event was not raised.) (Youssef: passing for me - test: Enable skipped BreadcrumbTests on Android #13077)ValidateOverflowButtonHidesWhenAppropriateWithNoAppBarButtons,ValidateMoreButtonCanShowWithoutSizeChanging,ValidateOverflowButtonHidesWhenAppropriateWithPrimaryAppBarButtonsare failing randomly on android. (Youssef: passing for me - test: Enable CommandBar tests on Android #13078)When_EmbeddedResource(resource is not included when build ondotnet build) (Youssef: This appears to only be an issue for Uno.UI solution, ie, not a problem for external customers. Fixing in build: Make sure AssignTargetPathsDependsOn is set properly in Uno solution build #13079)DoubleAnimation_Testson android. These tests fail on android when running one by one, but don't when run with others. (Youssef: Can't repro. Enabling in test: Enable DoubleAnimation_Tests #13082)When_Opacity_Paused_With_FillBehaviorHold_Then_Hold- on Android (Youssef: Can't repro. Enabling in test: Enable DoubleAnimation_Tests #13082)When_Transforms_Paused_With_FillBehaviorStop_Then_Hold. This test tends to fail when run alone, indicating a race condition, see in the comments for screenshots of the test. (Youssef: Can't repro. Enabling in test: Enable DoubleAnimation_Tests #13082)Given_ImageBrushStretch(IncorrectRenderTargetBitmapcontent) (youssef: Issue not related to RenderTargetBitmap. Fixing test in test: Unskip and fix When_Stretch test #13081)Done
When_xLoad_StaticResourcetest is failing #10164Basics_AutomatedTransformation(The specified child already has a parent. You must call removeView() on the child's parent first.) (test: UnskipBasics_AutomatedTransformationon Android #13061)When_Render_Then_CanRenderOnCanvas(Assert.Fail failed. ImageAssert.HasColorAt @ line 134) (works locally on Skia, fails on Android and being fixed in fix: Fix rendering of RenderTargetBitmap #13070)When_Reorder_To_Last_2(chore: enable When_Reorder_To_Last_2 on WASM #13093)RepeaterTests.VerifyUIElementsInItemsSource(test: enable working tests and add ItemsRepeater fixes for RepeaterTests #13074)RepeaterTests.NestedRepeaterWithDataTemplateScenario(test: enable working tests and add ItemsRepeater fixes for RepeaterTests #13074)RepeaterTests.VerifyStoreScenarioCache(test: enable working tests and add ItemsRepeater fixes for RepeaterTests #13074)CanOpenFlyoutByTapping,CanCloseFlyoutBySelectingADate, andPressingDoesNotOpenMenuFlyouttests are failing #10165 (fix(calendardatepicker): fix open on tap, close on escape #13056)The text was updated successfully, but these errors were encountered: