Skip to content

[Web][Backend] Voice command accessibility mode -> Web Speech API + preference endpoint (closes #287, #286)#617

Open
mustafa-5493 wants to merge 13 commits into
mainfrom
feat/voice-commands
Open

[Web][Backend] Voice command accessibility mode -> Web Speech API + preference endpoint (closes #287, #286)#617
mustafa-5493 wants to merge 13 commits into
mainfrom
feat/voice-commands

Conversation

@mustafa-5493

Copy link
Copy Markdown
Contributor

📝 Description

Closes #287 and parent #286. Implements requirement 1.1.3.8 end-to-end:

Backend

  • voiceCommandsEnabled boolean column on RegisteredUser (default false) + V6 Flyway migration
  • VoiceCommandsRequest DTO + RegisteredUserService.setVoiceCommandsEnabled
  • Owner-only PATCH /api/users/me/voice-commands (mirrors the leaderboard-visibility pattern)
  • voiceCommandsEnabled surfaced in UserProfileDTO so the web client reads it via /me

Web

  • parseVoiceCommand -> pure deterministic parser: zoom in/out, my location, find/search <q>, navigate to/go to <X>, report (<cat>) here
  • useSpeechRecognition -> thin React wrapper over window.SpeechRecognition / webkitSpeechRecognition, auto re-arms on quiet pauses, cleans up on unmount, surfaces errors as state
  • VoiceCommandContext -> app-level { active, lastEvent } state bridging the navbar mic toggle and the per-page command runner
  • VoiceCommandRunner -> lives inside the Leaflet MapContainer; dispatches recognized commands to existing Home setters (map.zoomIn(), setSearchTarget, setRouteDest, setShowCreatePanel, etc.) so a voice command is exactly equivalent to the click path
  • VoiceCommandMicButton -> navbar pill; hidden when feature unsupported or user hasn't opted in. Animates while listening (respects prefers-reduced-motion via motion-safe:)
  • VoiceCommandLiveRegion -> visually-hidden polite ARIA region; announces "Heard: …" / "Zoom in" / "Command not recognized: …" / errors
  • Accessibility section on /profile with a toggle calling useSetVoiceCommandsEnabled; unsupported-browser notice rendered inline when window.SpeechRecognition is missing

📊 Type

New feature

✅ Acceptance Criteria

  • Voice mode can be enabled/disabled from /profile; setting persists to backend and restores on next login from /api/users/me
  • Mic toggle button only shows when user is opted in and the browser supports the API
  • Voice commands trigger search, navigation, report, zoom, locate flows
  • Listening indicator (animated mic) visible while recording
  • ARIA live region announces recognition results
  • Graceful fallback message shown on unsupported browsers (Firefox/Safari without polyfill)
  • All voice-triggered actions remain accessible via keyboard/mouse (no fallback regression)
  • 481 frontend tests pass (15 new), backend tests cover service + controller paths

🧪 How to Test

Automated

cd backend && ./mvnw test            # service + controller tests for the new endpoint
cd frontend && npm run test:run      # 481 tests including 15 new parser tests

Manual (Chrome or Edge required for Web Speech API)

  1. Bring up the stack: docker compose -f docker-compose.local.yml --env-file backend/.env up -d --build
  2. Sign in, go to /profile → Accessibility section → enable "Voice commands"
  3. Reload -> toggle stays on (backend persisted)
  4. Mic icon appears in navbar. Click → grant mic permission → indicator pulses
  5. Say each:
    • "Zoom in" / "Zoom out" → map zooms
    • "My location" → map flies to user
    • "Find pharmacy" → map flies to first geocode result
    • "Navigate to Taksim" → route panel opens with Taksim as destination
    • "Report obstacle here" → create-report panel opens
    • Gibberish → live region announces "Command not recognized"
  6. Switch to Firefox → toggle in /profile shows unsupported-browser notice; mic icon hidden
  7. Hard-refresh / disable voice mode → mic icon disappears, listening session ends

📸 Screenshots

Ekran Resmi 2026-05-11 22 08 55

⏱️ Estimated Review Time

15 min (commit-by-commit on the Commits tab -> 12 atomic commits cover backend column → DTO → service → endpoint → tests → parser → hook → context → service-call → mic UI → runner → settings toggle)

@mustafa-5493 mustafa-5493 self-assigned this May 11, 2026
@mustafa-5493 mustafa-5493 added enhancement New feature or request frontend Visual features backend Features related to API's and internal implementations. labels May 11, 2026
@mustafa-5493 mustafa-5493 force-pushed the feat/voice-commands branch from b759044 to b28be73 Compare May 11, 2026 19:19
@mustafa-5493 mustafa-5493 requested a review from efemantarci May 11, 2026 19:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend Features related to API's and internal implementations. enhancement New feature or request frontend Visual features

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Web] Accessibility Mode: Voice Commands

1 participant