Development: E2E tests for programming exam participations using Git
#10326
Conversation
WalkthroughThis pull request refactors and enhances the Playwright test suite for exam participation and programming exercises using Git submissions. It updates test files by adding new imports, a dedicated test suite for Git-based programming exercises, and a new exam creation helper. The changes also remove an intermediary helper in favor of direct calls to the new GitExerciseParticipation class, update test configuration scripts with debugging and video recording settings, and modify page objects to better validate exercise scores and handle Git submissions. Changes
Sequence Diagram(s)sequenceDiagram
participant T as Test Runner
participant G as GitExerciseParticipation
participant E as ExamParticipationPage
T->>G: [If SSH] setupSSHCredentials(context, sshAlgorithm)
T->>G: makeSubmission(..., cloneMethod, sshAlgorithm)
G->>G: pushGitSubmissionFiles(...) %% internal submission processing
G-->>T: Submission completed
T->>E: checkExerciseScore(expectedResult)
Possibly related PRs
Suggested labels
Suggested reviewers
✨ Finishing Touches
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
src/test/playwright/support/pageobjects/exercises/programming/GitExerciseParticipation.ts (1)
13-13: Consider simplifying static-only class.
The linter flags this class for containing only static members. Converting it to a module of standalone functions or a simple object may offer cleaner code and fewer dependencies.🧰 Tools
🪛 Biome (1.9.4)
[error] 13-103: Avoid classes that contain only static members.
Prefer using simple functions instead of classes with only static members.
(lint/complexity/noStaticOnlyClass)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (7)
src/test/playwright/e2e/exam/ExamParticipation.spec.ts(3 hunks)src/test/playwright/e2e/exercise/programming/ProgrammingExerciseParticipation.spec.ts(5 hunks)src/test/playwright/package.json(1 hunks)src/test/playwright/playwright.config.ts(1 hunks)src/test/playwright/support/pageobjects/exam/ExamParticipationPage.ts(1 hunks)src/test/playwright/support/pageobjects/exercises/programming/GitExerciseParticipation.ts(1 hunks)src/test/playwright/support/pageobjects/exercises/programming/ProgrammingExerciseOverviewPage.ts(3 hunks)
🧰 Additional context used
🪛 Biome (1.9.4)
src/test/playwright/support/pageobjects/exercises/programming/GitExerciseParticipation.ts
[error] 13-103: Avoid classes that contain only static members.
Prefer using simple functions instead of classes with only static members.
(lint/complexity/noStaticOnlyClass)
⏰ Context from checks skipped due to timeout of 90000ms (7)
- GitHub Check: Call Build Workflow / Build .war artifact
- GitHub Check: Call Build Workflow / Build and Push Docker Image
- GitHub Check: Codacy Static Code Analysis
- GitHub Check: server-tests
- GitHub Check: server-style
- GitHub Check: client-tests
- GitHub Check: Analyse
🔇 Additional comments (12)
src/test/playwright/support/pageobjects/exercises/programming/GitExerciseParticipation.ts (2)
47-59: SSH setup appears valid.
The approach to retrieving and setting SSH keys for the user is straightforward and ensures the key is added to the test environment.
70-102: Well-structured Git submission process.
The sequence (deleting files, creating new files, committing, and pushing) is logically correct and ensures a clean working directory before submission. Please confirm error handling for cases when file operations or Git commands fail.src/test/playwright/e2e/exam/ExamParticipation.spec.ts (3)
3-3: Additional import looks fine.
This import forProgrammingExerciseAssessmentTypeis consistent with the new usage in the exam creation.
17-20: New imports for Git-based exam flows.
These imports correctly bring in classes and enums used for Git submissions in the new programming exam tests.
267-319: Dynamic testing of multiple Git clone methods.
Running the same test suite for HTTPS, HTTPS with token, and SSH ensures thorough coverage. The setup and teardown logic for SSH keys is well-placed under conditional hooks, promoting test stability.src/test/playwright/playwright.config.ts (1)
35-37: Video recording on first retry is a good choice.
This configuration helps diagnose flaky tests without generating excessive video artifacts.src/test/playwright/support/pageobjects/exercises/programming/ProgrammingExerciseOverviewPage.ts (2)
18-22: LGTM! Method rename improves clarity of intent.The method rename from
getResultScoretocheckResultScorebetter reflects its assertion functionality, making the code more self-documenting.
76-80: LGTM! Enum values are now more descriptive.The explicit string values in the
GitCloneMethodenum improve readability and make the code more maintainable.src/test/playwright/support/pageobjects/exam/ExamParticipationPage.ts (1)
128-132: LGTM! New method follows consistent pattern.The
checkExerciseScoremethod follows the same pattern ascheckResultScore, maintaining consistency across the codebase.src/test/playwright/e2e/exercise/programming/ProgrammingExerciseParticipation.spec.ts (2)
79-83: LGTM! Test code simplified using direct calls.The test now directly uses
GitExerciseParticipation.makeSubmission, making it cleaner and more maintainable.
106-110: LGTM! Improved SSH key verification.The code now properly verifies the absence of SSH key and displays appropriate alert before setup.
src/test/playwright/package.json (1)
20-20: LGTM! Enhanced debugging capabilities.Adding
DEBUG=pw:apito the sequential test script will help in diagnosing API-related issues during test runs.
src/test/playwright/support/pageobjects/exercises/programming/GitExerciseParticipation.ts
Show resolved
Hide resolved
Checklist
General
Client
Motivation and Context
We want to test the scenario where students participate in exams that contain programming exercises with automatic assessment and make submissions using Git.
Description
Adds 3 E2E tests in Playwright about participating exams with automatic programming exercise assessment and submitting solutions using Git. It includes Git submission using HTTPS + username/password, HTTPS + token and SSH using ED25519 key.
Steps for Testing
Steps for running the tests:
npm install && npm run playwright:setupnpx playwright test e2e/exercise/programming/ProgrammingExerciseParticipation.spec.ts -g "Programming exam with Git submissions"npm run playwright:opento open the Playwright UI, search for the "Programming exam with Git submissions" test suite and run itTestserver States
Note
These badges show the state of the test servers.
Green = Currently available, Red = Currently locked
Review Progress
Code Review
Manual Tests
Summary by CodeRabbit
New Features
Refactor
Tests