-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Description
Summary
Several E2E tests are currently marked as test.fixme() because the mock API doesn't properly simulate Vercel AI SDK streaming behavior. Additionally, the /knowledge-base page warmup times out during E2E seed setup.
Tests to Fix
Chat Tests (__tests__/e2e/tests/chat.spec.ts)
All require proper mock API that triggers Vercel AI SDK streaming state:
-
stop button appears during streaming and works(line 79)- Stop button only appears when
isLoading=trueduring active streaming - Mock doesn't trigger streaming state properly
- Stop button only appears when
-
displays error state and retry button on failure(line 119)- Error banner with Retry button requires SSE error format
- Mock returns 500 JSON instead of streaming SSE errors
-
retry button actually resends the message(line 137)- PR Fix offline retry flow - button now actually retries messages #844 regression test
- Requires error → retry → success flow to work
-
can queue messages while streaming(line 169)- Queue button only visible during active streaming
- Requires long-running streamed response
Seed Setup (__tests__/e2e/tests/seed.spec.ts)
/knowledge-basewarmup timeout (line 94)- Page times out during seed warmup phase
- May be related to page complexity or slow initial load
- Currently excluded from warmup list
Root Cause
The mock chat API in __tests__/e2e/fixtures/mock-chat-api.ts doesn't fully replicate Vercel AI SDK's streaming response format. The SDK uses specific SSE patterns that trigger UI state changes (loading spinners, stop buttons, error banners).
Proposed Solutions
- Update mock API to properly emit Vercel AI SDK streaming format
- Research
@ai-sdk/reactinternals to understand exact SSE patterns needed - Consider using MSW (Mock Service Worker) for more realistic API mocking
- Investigate
/knowledge-basepage performance - may need optimization or longer timeout
Acceptance Criteria
- All 4 chat tests pass (not fixme)
-
/knowledge-baseincluded in warmup without timeout - E2E tests run in CI without flakiness
Related
- PR ✨ Add developer-friendly warnings for skipped E2E tests #853 - E2E credential warnings (added fixme pattern)
- PR Fix offline retry flow - button now actually retries messages #844 - Original retry button fix
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels