-
Notifications
You must be signed in to change notification settings - Fork 62
feat: Improve flaky tests and add error messages #1653
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: christian.lutnik <[email protected]>
Summary of ChangesHello @chrfwow, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request focuses on improving the stability and diagnostic capabilities of existing tests. It addresses flakiness in a specific test by introducing more precise timing measurements and detailed error reporting. Additionally, it enhances the robustness of end-to-end tests by adopting a thread-safe data structure for event collection, ensuring reliable behavior in concurrent environments. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request improves test robustness by addressing potential concurrency issues and enhancing assertion messages. The change from LinkedList to ConcurrentLinkedQueue for handling events in E2E tests is a good move to prevent race conditions. The assertion message in FlagdProviderSyncResourcesTest is now more informative. I've added a couple of suggestions: one to improve test performance and readability by using a lazy-evaluated assertion message, and another to remove a redundant and potentially problematic re-initialization of a shared state field, which could itself introduce flakiness.
| public EventSteps(State state) { | ||
| super(state); | ||
| state.events = new LinkedList<>(); | ||
| state.events = new ConcurrentLinkedQueue<>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The events field in the State class is already initialized where it's declared. Re-initializing it here in the EventSteps constructor is redundant and can introduce flakiness. If other step definitions in the same scenario modify the events list before this constructor is called, those changes will be lost. It's best to remove this line and rely on the single initialization in the State class.
| Assertions.assertTrue( | ||
| waitTime.get() < MAX_TIME_TOLERANCE * 2, | ||
| "Wakeup should be almost instant, but took " + waitTime.get() | ||
| + " ms, which is more than the max of" | ||
| + (MAX_TIME_TOLERANCE * 2) + " ms"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For better performance and readability, it's a good practice to use a lambda for the assertion message in JUnit 5. This enables lazy evaluation, meaning the message string is only constructed if the assertion fails. This avoids unnecessary string concatenation in passing tests.
| Assertions.assertTrue( | |
| waitTime.get() < MAX_TIME_TOLERANCE * 2, | |
| "Wakeup should be almost instant, but took " + waitTime.get() | |
| + " ms, which is more than the max of" | |
| + (MAX_TIME_TOLERANCE * 2) + " ms"); | |
| Assertions.assertTrue( | |
| waitTime.get() < MAX_TIME_TOLERANCE * 2, | |
| () -> "Wakeup should be almost instant, but took " + waitTime.get() | |
| + " ms, which is more than the max of" | |
| + (MAX_TIME_TOLERANCE * 2) + " ms"); |
Signed-off-by: christian.lutnik <[email protected]>
Makes some tests more robust