Production-style machine learning repository for early sepsis risk prediction from ICU time-series data, with deterministic preprocessing, sequence modeling, manifest-based serving, and deployment-safe public demo packaging.
This repository implements an end-to-end workflow for early sepsis detection:
- schema-aware ingestion for official PhysioNet PSV files and local/Kaggle-style CSV mirrors
- deterministic patient-level splitting and leakage-resistant preprocessing
- sliding window generation for sequence learning
- GRU, LSTM, and PatchTST-style sequence classifiers
- experiment comparison, calibration analysis, and manifest-backed threshold modes
- FastAPI inference endpoints and a public-safe Streamlit presentation app
Real clinical source data is intentionally excluded from version control.
- Intended use: research, engineering validation, and portfolio demonstration
- Not intended use: direct clinical decision support
- Demo output is explanatory and operationally oriented, not treatment guidance
- Public deployment paths sanitize sensitive filesystem details when environment is non-development
- Data ingestion and validation: src/early_sepsis/data/ingestion.py
- Deterministic split and preprocessing: src/early_sepsis/data/pipeline.py
- Window generation: src/early_sepsis/data/windowing.py
- Sequence models: src/early_sepsis/modeling/sequence_models.py
- Sequence training/evaluation/prediction: src/early_sepsis/modeling/sequence_pipeline.py
- Experiment analysis and calibration: src/early_sepsis/modeling/experiment_analysis.py
- Selected model manifest management: src/early_sepsis/modeling/model_manifest.py
- API serving: src/early_sepsis/serving/api.py
- Streamlit demo and startup checks: src/early_sepsis/demo/app.py, src/early_sepsis/demo/startup.py
| Layer | Primary modules | Purpose |
|---|---|---|
| Ingestion | src/early_sepsis/data/ingestion.py | Detect format, normalize schema aliases, validate rows/files |
| Feature pipeline | src/early_sepsis/data/preprocessing.py | Train-only stats, imputation, scaling, masks |
| Temporal dataset | src/early_sepsis/data/windowing.py | Build label-aware windows with configurable horizon |
| Modeling | src/early_sepsis/modeling/sequence_models.py | GRU/LSTM/PatchTST sequence classifiers |
| Training + metrics | src/early_sepsis/modeling/sequence_pipeline.py, src/early_sepsis/modeling/sequence_metrics.py | Fit, evaluate, threshold selection, checkpointing |
| Registry + selection | src/early_sepsis/modeling/model_manifest.py | Portable selected model manifest |
| Serving | src/early_sepsis/serving/sequence_service.py, src/early_sepsis/serving/api.py | Manifest-backed inference with dataset and shape guardrails |
| Presentation | src/early_sepsis/demo/app.py | Public-safe dashboard with artifact-backed visuals |
- Validate and ingest raw data.
- Split patients into train/validation/test cohorts.
- Preprocess splits using train-derived statistics only.
- Build temporal windows and labels.
- Train sequence model(s) and export checkpoints.
- Compare runs and select best checkpoint into manifest.
- Analyze calibration and synchronize threshold modes.
- Serve through FastAPI and/or present through Streamlit.
- Build compact public artifact bundle for deployment.
data/raw/
├── patient_0001.psv
├── patient_0002.psv
└── ...
data/local_csv/
└── sepsis_data.csv
Requirements enforced by ingestion:
- target must map to
SepsisLabel(alias handling is implemented) - patient identifier aliases are supported (falls back to file stem when absent)
- time aliases are supported (falls back to row order when absent)
- malformed rows are dropped with warning; strict mode can fail fast
Synthetic test/demo assets are included under tests/fixtures, assets/demo, and data/demo. Restricted source clinical data is not committed.
Run from repository root.
Recommended (uv):
uv python install 3.12
uv sync --extra devAlternative (venv + pip):
python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
pip install -e .[dev]Base settings class: src/early_sepsis/settings.py
Environment template: .env.example
Common runtime variables:
SEPSIS_ENVIRONMENTSEPSIS_SELECTED_SEQUENCE_MANIFEST_PATHSEPSIS_PUBLIC_ARTIFACTS_DIRSEPSIS_SERVING_DEFAULT_OPERATING_MODESEPSIS_DEMO_PUBLIC_MODESEPSIS_MODEL_ARTIFACT_PATHSEPSIS_API_HOSTSEPSIS_API_PORT
Config files:
- configs/data_pipeline.yaml
- configs/model_training.yaml
- configs/model_tuning.yaml
- configs/api.yaml
- configs/orchestration.yaml
Generate synthetic ICU data, preprocess, window, and run a short training smoke flow:
uv run python scripts/generate_synthetic_data.py --output-path tests/fixtures/generated_synthetic.csv --dataset-format csv --patient-count 24 --min-hours 10 --max-hours 20 --seed 42
uv run python scripts/preprocess_data.py --raw-path tests/fixtures/generated_synthetic.csv --dataset-format csv --output-dir artifacts/processed_cli_smoke --strict
uv run python scripts/create_windows.py --processed-dir artifacts/processed_cli_smoke --output-dir artifacts/windows_cli_smoke --window-length 8 --prediction-horizon 6
uv run python scripts/train_sequence.py --windows-dir artifacts/windows_cli_smoke --output-dir artifacts/models/sequence_cli_smoke --model-type gru --epochs 2 --batch-size 64 --disable-mlflowValidate source files:
uv run python scripts/validate_raw_data.py --raw-path data/raw --dataset-format auto
uv run python scripts/validate_raw_data.py --raw-path data/raw --dataset-format auto --strictRun deterministic preprocessing pipeline:
uv run python scripts/preprocess_data.py --config configs/data_pipeline.yaml --raw-path data/raw --dataset-format auto --output-dir artifacts/processed
uv run python scripts/print_split_summary.py --processed-dir artifacts/processedCreate train/validation/test window parquet datasets:
uv run python scripts/create_windows.py --processed-dir artifacts/processed --output-dir artifacts/windows --window-length 8 --prediction-horizon 6Optional toggles are available in CLI for masks/static features and padding mode.
Train sequence model:
uv run python scripts/train_sequence.py --config configs/model_training.yaml --windows-dir artifacts/windows --output-dir artifacts/models/sequence --model-type patchtstResolve latest checkpoint and evaluate:
$CHECKPOINT = (Get-ChildItem artifacts/models/sequence -Recurse -Filter best_checkpoint.pt | Sort-Object LastWriteTime -Descending | Select-Object -First 1).FullName
uv run python scripts/evaluate_sequence.py --checkpoint-path $CHECKPOINT --windows-dir artifacts/windows --split testGenerate sequence prediction parquet:
uv run python scripts/predict_sequence.py --checkpoint-path $CHECKPOINT --parquet-path artifacts/windows/test.parquet --output-path artifacts/predictions/sequence_predictions.parquetTabular baseline path (used by /predict endpoint):
uv run python scripts/train_local.py --data-path tests/fixtures/synthetic_tabular.csv --dataset-format csvRun Optuna tuning:
uv run python scripts/tune_sequence.py --config configs/model_tuning.yaml --windows-dir artifacts/windows --model-type patchtst --n-trials 20Aggregate experiments:
uv run python scripts/compare_experiments.py --model-root artifacts/models --output-dir artifacts/analysis/experimentsSelect best run into manifest:
uv run python scripts/select_best_model.py --model-root artifacts/models --selection-metric validation_auprc --dataset-tag physionet --manifest-path artifacts/models/registry/selected_model.jsonRun calibration analysis and synchronize thresholds:
uv run python scripts/analyze_calibration.py --manifest-path artifacts/models/registry/selected_model.json --split validation --output-dir artifacts/analysis/calibration --high-recall-target 0.90 --sync-manifest-thresholdsManual threshold synchronization and path normalization:
uv run python scripts/sync_manifest_thresholds.py --manifest-path artifacts/models/registry/selected_model.json --recommendations-path artifacts/analysis/calibration/threshold_recommendations.json --summary-path artifacts/analysis/calibration/calibration_summary.json
uv run python scripts/migrate_manifest_paths.py --manifest-path artifacts/models/registry/selected_model.jsonThreshold operating modes currently implemented across serving and demo:
defaultbalancedhigh_recall
Evaluation interpretation in this project:
- Threshold-invariant metrics: AUROC, AUPRC, Brier score, Expected Calibration Error, prevalence
- Threshold-dependent outputs: predicted labels, confusion matrix counts, precision, recall, F1, alert rate
Test-set comparison across sequence architectures (PhysioNet windows):
| Model | AUROC | AUPRC | Brier score | F1 |
|---|---|---|---|---|
| GRU | 0.6959 | 0.0196 | 0.0420 | 0.0442 |
| LSTM | 0.6518 | 0.0170 | 0.0189 | 0.0286 |
| PatchTST | 0.7439 | 0.0267 | 0.6409 | 0.0266 |
ECE by threshold mode for the selected PatchTST test evaluation:
| Threshold mode | Threshold | Expected Calibration Error (ECE) |
|---|---|---|
default |
0.95 | 0.6629 |
balanced |
0.99 | 0.6629 |
high_recall |
0.95 | 0.6629 |
Note: ECE is threshold-invariant in this repository's metric definitions, so it remains constant across threshold modes for a fixed model and dataset split.
Start API:
uv run python scripts/serve_api.pyHealth and model metadata:
Invoke-RestMethod -Method Get -Uri "http://127.0.0.1:8000/health" | ConvertTo-Json -Depth 8
Invoke-RestMethod -Method Get -Uri "http://127.0.0.1:8000/model-info" | ConvertTo-Json -Depth 8Tabular inference request (/predict):
$records = Get-Content tests/fixtures/synthetic_records.json -Raw | ConvertFrom-Json
$body = @{ records = $records; include_explanation = $false } | ConvertTo-Json -Depth 8
Invoke-RestMethod -Method Post -Uri "http://127.0.0.1:8000/predict" -ContentType "application/json" -Body $body | ConvertTo-Json -Depth 8Sequence inference request (/predict-sequence):
uv run python -c "import json,httpx,pandas as pd; m=json.load(open('artifacts/models/registry/selected_model.json','r',encoding='utf-8')); tag=m['dataset']['dataset_tag']; df=pd.read_parquet(f\"{m['dataset']['windows_dir']}/validation.parquet\"); r=df.iloc[0]; payload={'dataset_tag':tag,'operating_mode':'balanced','samples':[{'patient_id':r['patient_id'],'end_hour':int(r['end_hour']),'features':r['features'].tolist(),'missing_mask':r['missing_mask'].tolist() if r['missing_mask'] is not None else None,'static_features':r['static_features'].tolist() if r['static_features'] is not None else None}]}; resp=httpx.post('http://127.0.0.1:8000/predict-sequence',json=payload,timeout=60.0); print(resp.status_code); print(resp.text)"Start demo:
uv run streamlit run streamlit_app.pyEquivalent helper script:
uv run python scripts/run_demo.pyDemo behavior is artifact-backed and public-safe:
- selected model loaded from manifest
- threshold mode selector maps to manifest thresholds
- threshold-invariant and threshold-dependent outputs are separated in presentation
- calibration reliability fallback chart is sanitized and bounded to [0, 1]
- sensitive local paths are not exposed when environment is non-development
Inference source order:
- Public mode: bundled demo parquet, then evaluation split parquet, then saved walkthrough payload
- Non-public mode: evaluation split parquet, then bundled demo parquet, then saved walkthrough payload
Operational summary source order:
public_artifacts/demo/operational_windows_subset.parquetassets/demo/operational_windows_subset.parquet<manifest.dataset.windows_dir>/<split>.parquet- current inference parquet source (if available)
Curate compact demo windows and operational subset:
uv run python scripts/curate_demo_assets.py --manifest-path artifacts/models/registry/selected_model.json --candidate-rows-per-source 3000 --demo-count 36 --operational-count 600Audit demo-score diversity and display mapping:
uv run python scripts/audit_demo_inference.py --manifest-path artifacts/models/registry/selected_model.json --parquet-path assets/demo/sequence_demo_samples.parquet --display-round-decimals 6Build compact public bundle:
uv run python scripts/prepare_public_artifacts.py --manifest-path artifacts/models/registry/selected_model.json --output-dir public_artifactsRequired deployment files are documented in public_artifacts/README.md.
Entrypoint and runtime files:
Recommended secrets block:
[sepsis]
environment = "production"
demo_public_mode = true
selected_sequence_manifest_path = "public_artifacts/models/registry/selected_model.json"
public_artifacts_dir = "public_artifacts"
demo_sample_parquet_path = "public_artifacts/demo/sequence_demo_samples.parquet"
public_repo_url = "https://github.com/<owner>/<repo>"docker build -t early-sepsis-streamlit .
docker run --rm -p 8501:8501 -e PORT=8501 early-sepsis-streamlitdocker compose -f docker/docker-compose.yml up --build apiRun full test suite:
uv run pytest -qRun targeted suites used by serving/demo paths:
uv run pytest tests/test_serving_sequence.py tests/test_demo_presentation.py tests/test_demo_thresholds.py -qStatic checks:
uv run ruff check .
uv run mypy srcCurrent limitations:
- Research implementation only; not a clinical decision-support product
- Sequence performance and threshold recommendations are artifact-dependent and dataset-dependent
- Public demo prioritizes portability and safety over full-fidelity offline evaluation scale
- Training and tuning can be compute-intensive on CPU-only systems
Practical improvement backlog:
- add automated drift monitoring jobs for post-training score distribution tracking
- add richer model comparison visualization overlays for operating mode review
- harden deployment CI for artifact integrity checks before release
This project is licensed under the MIT License. See the LICENSE file for details.