Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky test on Windows: test_git::test_concurrent_read_write_commit #3069

Closed
emesterhazy opened this issue Feb 16, 2024 · 1 comment
Closed

Comments

@emesterhazy
Copy link
Member

emesterhazy commented Feb 16, 2024

This test seems flaky. @martinvonz confirmed him and others have been seeing this for a while, but he hasn't been able to repro it on a Windows machine.

Here's an example flake:

Here's an excerpt from the logs:

---- test_git::test_concurrent_read_write_commit stdout ----
thread '<unnamed>' panicked at lib\tests\test_git.rs:2931:45:
unexpected error: A lock could not be obtained for reference 

I did a code search on github for the string "A lock could not be obtained for reference" and only appears in GitOxide here:

GitOxide might not be the issue since Martin mentioned there's a failing run from 6 months ago with the same issue (the logs are missing so I can't confirm though).

Edit: I realized I should probably post the full log before it's gc'd by github actions.

failures:

---- test_git::test_concurrent_read_write_commit stdout ----
thread '<unnamed>' panicked at lib\tests\test_git.rs:2931:45:
unexpected error: A lock could not be obtained for reference "refs/jj/keep/c5c6efd6ac240102e7f047234c3cade55eedd621"
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library\std\src\panicking.rs:645
   1: core::panicking::panic_fmt
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library\core\src\panicking.rs:72
   2: runner::test_git::test_concurrent_read_write_commit::{{closure}}::{{closure}}::{{closure}}
   3: core::iter::adapters::filter_map::filter_map_try_fold::{{closure}}
   4: core::iter::traits::iterator::Iterator::try_fold
   5: <core::iter::adapters::filter_map::FilterMap<I,F> as core::iter::adapters::SourceIter>::as_inner
   6: <I as alloc::vec::in_place_collect::SpecInPlaceCollect<T,I>>::collect_in_place
   7: alloc::vec::Vec<T,A>::is_empty
   8: <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter
   9: core::iter::traits::iterator::Iterator::collect
  10: itertools::Itertools::collect_vec
  11: runner::test_git::test_concurrent_read_write_commit::{{closure}}::{{closure}}
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread 'test_git::test_concurrent_read_write_commit' panicked at lib\tests\test_git.rs:2884:5:
a scoped thread panicked
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library\std\src\panicking.rs:645
   1: core::panicking::panic_fmt
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library\core\src\panicking.rs:72
   2: std::thread::scoped::scope
   3: runner::test_git::test_concurrent_read_write_commit::{{closure}}
   4: runner::test_git::test_concurrent_read_write_commit::{{closure}}
   5: core::ops::function::FnOnce::call_once
   6: core::ops::function::FnOnce::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library\core\src\ops\function.rs:250
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.


failures:
    test_git::test_concurrent_read_write_commit

test result: FAILED. 298 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 7.43s

error: test failed, to rerun pass `-p jj-lib --test runner`
Error: Process completed with exit code 1.
@emesterhazy
Copy link
Member Author

emesterhazy commented Feb 16, 2024

Possibly related to #1536.

yuja added a commit to yuja/jj that referenced this issue Feb 17, 2024
…tion

Apparently, gix has 100ms timeout. Since this test tries to create contended
situation, it's possible that the ref lock can't be acquired. I've added
upper bound to the retry loop at b37293f "tests: add upper bound to
test_concurrent_read_write_commit() loop", so ignoring arbitrary errors
should be okay.

The problem can be reproduced on my Linux machine by inserting 10ms sleep() to
gix and increasing the concurrency.

Fixes jj-vcs#3069
@yuja yuja closed this as completed in 5eea88d Feb 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant