From ff3ac16ad82368953d906c1e2827c140a1cc448b Mon Sep 17 00:00:00 2001 From: Jorge Miguel Silva Date: Sun, 10 Aug 2025 12:55:51 +0100 Subject: [PATCH] docs: streamline README and update usage --- README.md | 236 ++++++++++++++++++++---------------------------------- 1 file changed, 87 insertions(+), 149 deletions(-) diff --git a/README.md b/README.md index 2569b15..999a393 100644 --- a/README.md +++ b/README.md @@ -51,18 +51,45 @@ - [Quick Start](#quick-start) - [CLI](#cli) - [GUI](#gui) -- [Configuration](#configuration) - [Reports](#reports) - [Examples and Scripts](#examples-and-scripts) +- [Configuration](#configuration) - [Troubleshooting](#troubleshooting) - [Contributing](#contributing) --- -## Quick Start +## Installation + +PhenoQC requires Python 3.9+. + +### From PyPI ```bash pip install phenoqc +``` + +### From source + +```bash +git clone https://github.com/jorgeMFS/PhenoQC.git +cd PhenoQC +pip install -e . +``` + +For local development without installation you can run: + +```bash +python -m phenoqc.cli +``` + +Dependencies are listed in `requirements.txt`. + +--- + +## Quick Start + +```bash phenoqc --help # Minimal run @@ -91,133 +118,11 @@ phenoqc \ ## CLI -New flags: - -- `--impute-params '{"n_neighbors": 5}'` (JSON) -- `--impute-tuning on|off` -- `--label-column class` and `--imbalance-threshold 0.10` - -Reports generated under `--output` include a PDF with: - -- Summary & scores -- Optional Class Distribution (when label column is set) -- Additional Quality Dimensions (only when computed) -- Missing data summary, mapping success, and visuals - -### CLI additions - -- `--impute-params '{"n_neighbors": 5}'` (JSON) -- `--impute-tuning on|off` -- `--label-column class` and `--imbalance-threshold 0.10` - -### Config (new `imputation:` block) - -```yaml -imputation: - strategy: knn - params: - n_neighbors: 5 - weights: uniform - per_column: - Creatinine_mgdl: - strategy: mice - params: {max_iter: 15} - Cholesterol_mgdl: - strategy: svd - params: {rank: 3} - tuning: - enable: true - mask_fraction: 0.1 - scoring: MAE - max_cells: 20000 - random_state: 42 - grid: - n_neighbors: [3, 5, 7] -``` - -Per-column overrides accept their own params. The quick tuner supports KNN, MICE (max_iter) and SVD (rank); you can pass a grid to explore. - ---- - -## GUI - -Launch: - -```bash -python run_gui.py -``` - -Highlights: - -- Step 3: Optional label column and imbalance threshold -- Step 4: Default strategy, per-column overrides, parameters, and tuning -- Results: Class Distribution table/plot, Imputation Settings, Tuning Summary - ---- - -## Reports - -- Class Distribution: table and warning when minority proportion < threshold -- Imputation Settings: global strategy/params and tuning summary -- Additional Quality: only displayed if metrics are computed - ---- - -## Examples and Scripts - -- `scripts/e2e_small_quality_metrics_cli_test.py` – small demo focusing on quality metrics -- `scripts/e2e_medium_cli_test.py` – mid-sized end-to-end pipeline run -- `scripts/end_to_end_e2e_cli_test.py` – large end-to-end pipeline run -- `scripts/imputation_params_cli_test.py` – imputation params and optional tuning - ---- - ---- - -## Installation - -PhenoQC requires Python 3.9+. - -### Fresh install - -Install from PyPI: - -```bash -pip install phenoqc -``` - -### From source - -Clone the repository and install in editable mode: - -```bash -git clone https://github.com/jorgeMFS/PhenoQC.git -cd PhenoQC -pip install -e . -``` - -Running the CLI directly from the uninstalled `src/` tree will fail. For local -development without installation you can use: +PhenoQC provides a flexible command-line interface suited for automation. -```bash -python -m phenoqc.cli -``` - -**Dependencies** are listed in `requirements.txt` and include: - -- `pandas`, `jsonschema`, `requests`, `plotly`, `reportlab`, `streamlit`, - `pyyaml`, `watchdog`, `kaleido`, `tqdm`, `Pillow`, `scikit-learn`, - `fancyimpute`, `fastjsonschema`, `pronto`, `rapidfuzz`. - ---- - -## Usage - -PhenoQC can be invoked via its **CLI** or through the **GUI**: - -### 1. Command-Line Interface (CLI) +### Examples -#### Example: Process a Single File +#### Process a single file ```bash phenoqc \ @@ -232,7 +137,7 @@ phenoqc \ --ontologies HPO DO ``` -#### Example: Batch Process Multiple Files +#### Batch process multiple files ```bash phenoqc \ @@ -246,37 +151,70 @@ phenoqc \ --phenotype_columns '{"PrimaryPhenotype": ["HPO"], "DiseaseCode": ["DO"], "TertiaryPhenotype": ["MPO"]}' ``` -**Key Parameters:** +### Useful flags + +- `--impute-params '{"n_neighbors": 5}'` (JSON) +- `--impute-tuning on|off` +- `--label-column class` with `--imbalance-threshold 0.10` + +Reports generated under `--output` include: + +- Summary & scores +- Optional Class Distribution when a label column is set +- Additional Quality Dimensions when computed +- Missing data summary, mapping success, and visuals + +### Key parameters + +- `--input`: one or more data files or directories (`.csv`, `.tsv`, `.json`, `.zip`) +- `--output`: directory for saved reports (default `./reports/`) +- `--schema`: JSON schema for validation +- `--config`: YAML configuration (default `config.yaml`) +- `--custom_mappings`: custom term-mapping JSON (optional) +- `--impute`: strategy for missing data (`mean`, `median`, `mode`, `knn`, `mice`, `svd`, `none`) +- `--unique_identifiers`: columns that uniquely identify records +- `--phenotype_columns`: JSON mapping of columns to ontologies +- `--ontologies`: list of ontology IDs (e.g., `HPO DO MPO`) +- `--recursive`: enable recursive directory scanning -- `--input`: One or more data files or directories (`.csv`, `.tsv`, `.json`, `.zip`). -- `--output`: Directory for saving processed data and reports (default: `./reports/`). -- `--schema`: Path to the JSON schema for data validation. -- `--config`: YAML config file defining ontologies and settings (default: `config.yaml`). -- `--custom_mappings`: Path to a custom term-mapping JSON (optional). -- `--impute`: Strategy for missing data (e.g., `mean`, `median`, `mode`, `knn`, `mice`, `svd`, or `none`). -- `--unique_identifiers`: Columns that uniquely identify each record (e.g., `SampleID`). -- `--phenotype_columns`: JSON mapping of columns to ontologies: - e.g., `{"PrimaryPhenotype": ["HPO"], "DiseaseCode": ["DO"]}` -- `--ontologies`: List of ontology IDs (e.g., `HPO DO MPO`). -- `--recursive`: Enable recursive scanning of directories. +Per-column overrides accept their own params. The quick tuner supports KNN, MICE (`max_iter`) and SVD (`rank`); you can pass a grid to explore. --- -### 2. Graphical User Interface (GUI) +## GUI -Launch the Streamlit GUI for an interactive experience: +Launch the Streamlit interface: ```bash python run_gui.py ``` -**Workflow in the GUI**: +Workflow: + +1. Upload config & schema. +2. Upload data files or a `.zip` archive. +3. Choose unique identifiers & ontologies. +4. Set missing data strategy and optional tuning. +5. Review results and download reports. + +The GUI also surfaces class-distribution warnings and imputation summaries. + +--- + +## Reports -1. **Upload Config & Schema**: Provide a JSON schema and a YAML config to define validation and ontology settings. -2. **Upload Data**: Either upload individual `.csv`/`.tsv`/`.json` files or a `.zip` archive containing multiple files. -3. **Choose Unique Identifiers & Ontologies**: Select columns to map to ontologies (HPO, DO, etc.) and specify unique identifier columns (e.g., `SampleID`). -4. **Set Missing Data Strategy**: Choose an imputation strategy (mean, median, mode, advanced). -5. **Run QC**: Process data and review results. Download generated reports. +- Class Distribution: table and warning when minority proportion < threshold +- Imputation Settings: global strategy/params and tuning summary +- Additional Quality: only displayed if metrics are computed + +--- + +## Examples and Scripts + +- `scripts/e2e_small_quality_metrics_cli_test.py` – small demo focusing on quality metrics +- `scripts/e2e_medium_cli_test.py` – mid-sized end-to-end pipeline run +- `scripts/end_to_end_e2e_cli_test.py` – large end-to-end pipeline run +- `scripts/imputation_params_cli_test.py` – imputation params and optional tuning --- @@ -373,4 +311,4 @@ For more details, see the [GitHub Wiki](https://github.com/jorgeMFS/PhenoQC/wiki --- -*Last updated: January 13, 2025.* +*Last updated: August 10, 2025.*