Skip to content

🧪 Fix E2E tests marked as fixme (mock API + knowledge-base warmup) #854

@TechNickAI

Description

@TechNickAI

Summary

Several E2E tests are currently marked as test.fixme() because the mock API doesn't properly simulate Vercel AI SDK streaming behavior. Additionally, the /knowledge-base page warmup times out during E2E seed setup.

Tests to Fix

Chat Tests (__tests__/e2e/tests/chat.spec.ts)

All require proper mock API that triggers Vercel AI SDK streaming state:

  1. stop button appears during streaming and works (line 79)

    • Stop button only appears when isLoading=true during active streaming
    • Mock doesn't trigger streaming state properly
  2. displays error state and retry button on failure (line 119)

    • Error banner with Retry button requires SSE error format
    • Mock returns 500 JSON instead of streaming SSE errors
  3. retry button actually resends the message (line 137)

  4. can queue messages while streaming (line 169)

    • Queue button only visible during active streaming
    • Requires long-running streamed response

Seed Setup (__tests__/e2e/tests/seed.spec.ts)

  1. /knowledge-base warmup timeout (line 94)
    • Page times out during seed warmup phase
    • May be related to page complexity or slow initial load
    • Currently excluded from warmup list

Root Cause

The mock chat API in __tests__/e2e/fixtures/mock-chat-api.ts doesn't fully replicate Vercel AI SDK's streaming response format. The SDK uses specific SSE patterns that trigger UI state changes (loading spinners, stop buttons, error banners).

Proposed Solutions

  1. Update mock API to properly emit Vercel AI SDK streaming format
  2. Research @ai-sdk/react internals to understand exact SSE patterns needed
  3. Consider using MSW (Mock Service Worker) for more realistic API mocking
  4. Investigate /knowledge-base page performance - may need optimization or longer timeout

Acceptance Criteria

  • All 4 chat tests pass (not fixme)
  • /knowledge-base included in warmup without timeout
  • E2E tests run in CI without flakiness

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions