Skip to content

feat: SSE progress streaming + structured logging (feature parity)#3

Open
sktbrd wants to merge 1 commit intobgrana75:mainfrom
SkateHive:feat/sse-progress-logging-parity
Open

feat: SSE progress streaming + structured logging (feature parity)#3
sktbrd wants to merge 1 commit intobgrana75:mainfrom
SkateHive:feat/sse-progress-logging-parity

Conversation

@sktbrd
Copy link
Copy Markdown
Contributor

@sktbrd sktbrd commented Mar 20, 2026

Feature Parity PR

This brings the Oracle worker to 100% feature parity with the Mac Mini worker.

New Endpoints

Endpoint Method Description
/progress/:requestId GET Real-time SSE progress streaming
/logs?limit=10 GET Recent transcode logs (JSON)
/stats GET Transcode statistics

New File

  • src/logger.js — Structured logging with file persistence (logs/transcode-{hostname}.log)

SSE Progress Flow

Client subscribes: GET /progress/{correlationId}
  ↓
Client uploads: POST /transcode (with correlationId in form data)
  ↓
Server streams progress:
  5%  → receiving
  10% → transcoding (FFmpeg starts)
  0-80% → transcoding (FFmpeg real-time progress via stderr)
  85% → uploading (Pinata)
  85-100% → uploading (Pinata onUploadProgress)
  100% → complete

Why This Matters

  • Webapp progress bar now works on ALL servers (not just Mac Mini)
  • No more 404 errors on /progress endpoint
  • Dashboard monitoring via /logs and /stats
  • Structured logs for debugging failed transcodes

Backward Compatible

  • All new features are additive
  • Existing /transcode and /healthz endpoints unchanged
  • No breaking changes to request/response format

…ints

Feature parity with Mac Mini video worker:

NEW ENDPOINTS:
- GET /progress/:requestId - Real-time SSE progress streaming
- GET /logs?limit=10 - Recent transcode logs (JSON)
- GET /stats - Transcode statistics (total, success rate, avg duration)

NEW FILES:
- src/logger.js - Structured logging with file persistence

FEATURES:
- SSE progress: clients can subscribe before upload starts via correlationId
- FFmpeg progress tracking: real-time transcode percentage (0-80%)
- Upload progress: Pinata upload percentage (85-100%)
- Structured logs: saved to logs/transcode-{hostname}.log
- Dashboard data: logs + stats endpoints for monitoring
- Enhanced error logging: full Pinata error details

All servers (Mac Mini, Oracle, Pi) now have identical feature sets.
Webapp SSE progress bar works on all servers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant