Skip to content

technical_documentation.md

Latest commit

 

History

History
240 lines (190 loc) · 6.06 KB

technical_documentation.md

File metadata and controls

240 lines (190 loc) · 6.06 KB
Raw

Produce.it - Technical Documentation

System Architecture Overview

flowchart LR
    subgraph Frontend["Frontend (Next.js)"]
        A[MagicDropzone] --> B[API Call]
        C[Results Page] --> D[ScoreRing]
        C --> E[DiagnosticCard]
    end

    subgraph API["API Server (Flask)"]
        F["/api/analyze"] --> G[AnalysisPipeline]
    end

    subgraph Models["AI Models (PyTorch)"]
        H[StereoModel]
        I[LUFSModel]
        J[SpectralModel]
    end

    subgraph Analyzers["Audio Analyzers"]
        K[LUFSAnalyzer]
        L[StereoAnalyzer]
        M[SpectrogramProcessor]
    end

    B --> F
    G --> K --> I
    G --> L --> H
    G --> M --> J
    G --> C
Loading

AI Models

1. Stereo Assessment Model

Property Value
File src/crnn/stereo_model.pth (4KB)
Architecture MLP: 4 → 16 → 8 → 1
Activation ReLU + Sigmoid
Training Samples 165
Validation Loss 0.0079

Inputs (4 features):

  1. user_width - Stereo width of user track (0.0-1.0)
  2. user_correlation - Phase correlation (-1.0 to 1.0)
  3. ref_width - Reference/target width
  4. ref_correlation - Reference/target correlation

Output: Quality score (0.0-1.0) → multiplied by 100 for display

2. LUFS/Dynamics Model

Property Value
File src/crnn/lufs_model.pth (4KB)
Architecture MLP: 4 → 16 → 8 → 1
Activation ReLU + Dropout(0.1) + Sigmoid
Training Samples 130
Validation Loss 0.0204

Inputs (4 features, normalized):

  1. (user_lufs + 30) / 30 - Normalized user LUFS
  2. (ref_lufs + 30) / 30 - Normalized reference LUFS
  3. lufs_diff / 20 - LUFS difference normalized
  4. spec_std / 50 - Spectral standard deviation

Output: Quality score (0.0-1.0)

3. Spectral Model

Property Value
File src/crnn/spectral_model.pth (7KB)
Architecture MLP: 4 → 32 → 16 → 8 → 1
Activation ReLU + Dropout(0.1) + Sigmoid
Training Samples 190
Validation Loss 0.0073

Inputs (4 features, normalized):

  1. user_spec_mean / 50
  2. user_spec_std / 50
  3. ref_spec_mean / 50
  4. ref_spec_std / 50

Output: Quality score (0.0-1.0)

Training Data Pipeline

Reference Database (Gold Standard)

  • Source: 97 rock tracks from data/input/rock/
  • Output: data/output/reference_features/rock_references.csv

Reference Statistics:

Metric Mean Std Dev
LUFS -15.36 dB 3.03
Stereo Width 0.336 (33.6%) 0.166
Phase Correlation 0.331 0.349

Synthetic Degradation Engine

File: src/data/synthetic_degradation.py

Category Degradations
Stereo mono_collapse, phase_inversion, over_widening, stereo_imbalance
Dynamics over_compression, clipping, volume_reduction
Spectral lowpass_filter, highpass_filter, mud_boost, harsh_boost

Process:

  1. Load reference track
  2. Apply random degradation (intensity 0.3-1.0)
  3. Extract features from original and degraded
  4. Calculate quality score based on intensity
  5. Save as training pair

Generated: 485 samples (97 tracks × 5 degradations)

API Endpoints

POST /api/analyze

Upload and analyze audio file.

Request: multipart/form-data

  • file (required): Audio file
  • reference (optional): Reference track

Response:

{
  "file": "track.mp3",
  "overall_score": 62.9,
  "lufs": {
    "value": -13.7,
    "target": -15.36,
    "difference": 1.7,
    "ai_score": 57
  },
  "stereo": {
    "width": 0.24,
    "correlation": 0.52,
    "ai_score": 63
  },
  "spectral": {
    "mean": 4.7,
    "std": 17.0,
    "ai_score": 69
  },
  "diagnostics": []
}

GET /api/health

Returns loaded models and status.

GET /api/reference-stats

Returns rock reference database statistics.

Frontend Components

Component Purpose
MagicDropzone.tsx Drag & drop + file input, API submission
ScoreRing.tsx Animated circular score visualization
DiagnosticCard.tsx Issue/suggestion display cards
results/page.tsx Full results page with all scores

File Structure

src/
├── pipeline.py              # Main orchestrator
├── api_server.py            # Flask REST API
├── crnn/
│   ├── models.py            # Model class definitions
│   ├── stereo_model.pth     # Trained weights
│   ├── lufs_model.pth
│   ├── spectral_model.pth
│   ├── train_stereo.py
│   ├── train_lufs.py
│   └── train_spectral.py
├── data/
│   ├── synthetic_degradation.py
│   ├── generate_training_data.py
│   └── extract_reference_features.py
├── pre_mastering/
│   └── lufs_comparison.py   # LUFS analysis
├── stereo_field/
│   └── stereo_analyzer.py   # Stereo analysis
└── process_spectrograms/
    └── spectrogram_processor.py

frontend/
├── app/
│   ├── page.tsx             # Home page
│   └── results/page.tsx     # Results page
└── components/
    ├── MagicDropzone.tsx
    ├── ScoreRing.tsx
    └── DiagnosticCard.tsx

Running the System

./start.sh
# API: http://localhost:5001
# Frontend: http://localhost:3000

Limitations & Future Improvements

Current Limitation Future Solution
Small training set (485 samples) Collect more reference tracks
Synthetic degradations only Add real user track ratings
Rock genre bias Multi-genre reference databases
No audio file output Add auto-mastering feature