Skip to content

Add subject-level visual QC outputs (histogram, segmentation, z-profile, object stats, ROI summaries, vessel overlays)#141

Merged
akhanf merged 10 commits intomainfrom
copilot/add-visual-qc-outputs
Apr 1, 2026
Merged

Add subject-level visual QC outputs (histogram, segmentation, z-profile, object stats, ROI summaries, vessel overlays)#141
akhanf merged 10 commits intomainfrom
copilot/add-visual-qc-outputs

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 31, 2026

Adds a new qc.smk rules module that generates per-subject, per-channel PNG quality-control figures covering the major QC categories described in the issue. All outputs land in the qc BIDS datatype directory.

New rules (spimquant/workflow/rules/qc.smk)

Rule Inputs Output
qc_intensity_histogram Raw OME-Zarr (per stain, at registration level) Linear + log histogram, CDF, saturation fraction; linear panel uses percentile-based axis bounds
qc_segmentation_overview OME-Zarr SPIM + seg mask.ozx at segmentation level 3-orientation slice montage (5 slices) + MIP with overlay; near-isotropic downsampling and aspect ratio via ZarrNii.get_zooms()
qc_vessels_overview OME-Zarr SPIM + vessel mask.ozx at segmentation level Same whole-brain slice montage + MIP for vessel masks (vesselslices.png)
qc_segmentation_roi_zoom OME-Zarr SPIM + seg mask.ozx + atlas dseg NIfTI (subject space) + label TSV Per-atlas-region full-resolution crops via ZarrNii/ZarrNiiAtlas: best axial slice with overlay (roimontage.png)
qc_vessels_roi_zoom OME-Zarr SPIM + vessel mask.ozx + atlas dseg NIfTI (subject space) + label TSV Same full-resolution ROI zoom montage for vessel masks (vesselroimontage.png)
qc_zprofile SPIM NIfTI + field-fraction NIfTI Per-Z-slice mean intensity (±1 SD) and field-fraction profile
qc_objectstats Aggregated regionprops parquet Volume distribution, log-volume, equivalent-radius, summary stats
qc_roi_summary Merged segstats TSV + atlas label TSV Top-20 region bar charts for field fraction, count, and density; background (atlas label 0) excluded

Workflow integration (Snakefile)

  • New all_qc target rule — intensity histograms always; segmentation-dependent outputs gated on do_seg; vessel outputs gated on do_vessels
  • rules.all_qc.input added to all_participant so QC runs by default
  • include: "rules/qc.smk" appended alongside existing rule includes

Scripts (spimquant/workflow/scripts/)

Six new scripts following the established snakemake-script pattern — access inputs/outputs/wildcards/params via the snakemake object, use matplotlib with the agg backend, and leverage zarrnii/pandas as appropriate:

  • qc_intensity_histogram.py — linear-scale histogram panel uses percentile-based axis bounds: x-axis capped at the 99th-percentile intensity (+ 5 % headroom) and y-axis capped at the tallest bar within that visible range, preventing long empty tails and background spikes from obscuring the distribution body; log-scale and CDF panels retain full-range bounds
  • qc_segmentation_overview.py — loads SPIM and mask via ZarrNii.from_ome_zarr() with downsample_near_isotropic=True; uses get_zooms() for physically correct imshow aspect ratios; also used by qc_vessels_overview
  • qc_segmentation_roi_zoom.py — uses ZarrNii and ZarrNiiAtlas to load full-resolution data; crops to each atlas ROI's bounding box via atlas.get_region_bounding_box() and crop_centered(); renders a 5-column grid montage; also used by qc_vessels_roi_zoom
  • qc_zprofile.py
  • qc_objectstats.py
  • qc_roi_summary.py — drops atlas label 0 (background) immediately after loading the segstats TSV so out-of-brain regions never appear in the summary bar charts

- New qc.smk with 5 QC rules: qc_intensity_histogram,
  qc_segmentation_overview, qc_zprofile, qc_objectstats, qc_roi_summary
- New Python scripts for each QC rule
- Add all_qc target rule to Snakefile and include it in all_participant
- All PNGs written to 'qc' datatype directory for each subject

Agent-Logs-Url: https://github.com/khanlab/SPIMquant/sessions/79447efe-f339-4deb-a3b4-6d9823ac5ccd

Co-authored-by: akhanf <11492701+akhanf@users.noreply.github.com>
Copilot AI changed the title [WIP] Add new smk file for visual QC outputs Add subject-level visual QC outputs (histogram, segmentation, z-profile, object stats, ROI summaries) Mar 31, 2026
Copilot AI requested a review from akhanf March 31, 2026 18:20
@akhanf
Copy link
Copy Markdown
Member

akhanf commented Mar 31, 2026

@copilot: the segslice overlay needs to have the aspect ratio scaled accordingly by voxel dimensions, or use the downsample_near_isotropic arg with from_ome_zarr..

also, need to have another output png similar to this, but zoomed in and cropped, so full detail can be seen, ideally montage across multiple ROIs.

this visualization can be done identically for the vessels mask as well, i.e. not just the stains_for_seg

- qc_segmentation_overview.py: replace aspect='auto' with physically
  correct aspect ratio computed from NIfTI header voxel dimensions
  (_slice_aspect helper for axial/coronal/sagittal orientations)
- qc_segmentation_roi_zoom.py: new script; crops SPIM+fieldfrac to
  each atlas ROI bounding box, shows best axial slice with overlay
  and correct aspect ratio
- qc.smk: add qc_vessels_overview (vessel fieldfrac, vesselslices.png),
  qc_segmentation_roi_zoom (roimontage.png),
  qc_vessels_roi_zoom (vesselroimontage.png)
- Snakefile: add vessel overview + seg/vessel ROI zooms to all_qc

Agent-Logs-Url: https://github.com/khanlab/SPIMquant/sessions/5069de35-e796-4137-9dfe-cbaa8a4d880e

Co-authored-by: akhanf <11492701+akhanf@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 31, 2026

@copilot: the segslice overlay needs to have the aspect ratio scaled accordingly by voxel dimensions, or use the downsample_near_isotropic arg with from_ome_zarr..

also, need to have another outpu...

Addressed in commit 1cd0826:

  1. Aspect ratio: Added _slice_aspect(zooms, step_axis) in qc_segmentation_overview.py that reads voxel dimensions from the NIfTI header (header.get_zooms()) and computes the physically correct imshow aspect ratio for each orientation (axial: dy/dx, coronal: dz/dx, sagittal: dz/dy). Applied to both slice panels and MIP columns.

  2. Zoomed ROI montage: New qc_segmentation_roi_zoom.py + qc_segmentation_roi_zoom rule (output: roimontage.png). Crops SPIM and field-fraction to each atlas region's bounding box (from the parcellation in subject space), picks the Z-slice with the most field-fraction signal, and lays out all ROIs in a 5-column grid with correct aspect ratio.

  3. Vessel support: New qc_vessels_overview rule (same qc_segmentation_overview.py script, reads datatype="vessels" fieldfrac → vesselslices.png) and qc_vessels_roi_zoom rule (same qc_segmentation_roi_zoom.py script → vesselroimontage.png). Both are added to all_qc, gated on do_vessels.

Copilot AI changed the title Add subject-level visual QC outputs (histogram, segmentation, z-profile, object stats, ROI summaries) Add subject-level visual QC outputs (histogram, segmentation, z-profile, object stats, ROI summaries, vessel overlays) Mar 31, 2026
@akhanf
Copy link
Copy Markdown
Member

akhanf commented Apr 1, 2026

@copilot I have modified the qc_segmentaton_roi_zoom to make use of ZarrNii and ZarrNiiAtlas so that full-res images can be generated. update the rest of the high-res plotting functions to also use ZarrNii and ZarrNii atlas, including the vessel segmentaiton plotting.

- qc_segmentation_overview.py: replace nibabel NIfTI loading with
  ZarrNii.from_ome_zarr() (downsample_near_isotropic=True); use
  get_zooms() for aspect ratio; import scipy.ndimage.zoom at top
- qc.smk qc_segmentation_overview: inputs now spim (OME-Zarr) +
  mask.ozx at segmentation_level; add level/mask_level/zarrnii_kwargs
  params; increase threads/mem_mb to match histogram rule
- qc.smk qc_vessels_overview: same changes, datatype='vessels' mask
- qc.smk qc_vessels_roi_zoom: update to same ZarrNii pattern as
  qc_segmentation_roi_zoom (OME-Zarr + mask.ozx + dseg_nii); add
  level param; increase threads/mem_mb

Agent-Logs-Url: https://github.com/khanlab/SPIMquant/sessions/ebb6ebb2-5a93-4243-ac48-35628311ecb3

Co-authored-by: akhanf <11492701+akhanf@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 1, 2026

@copilot I have modified the qc_segmentaton_roi_zoom to make use of ZarrNii and ZarrNiiAtlas so that full-res images can be generated. update the rest of the high-res plotting functions to also use Za...

Done in commit d489341.

  • qc_segmentation_overview.py: Replaced nibabel NIfTI loading with ZarrNii.from_ome_zarr() using downsample_near_isotropic=True for the SPIM and the same for the mask. Voxel spacings come from get_zooms() instead of the NIfTI header.

  • qc_segmentation_overview + qc_vessels_overview rules: Inputs changed from SPIM NIfTI + fieldfrac NIfTI to inputs["spim"].path (OME-Zarr) + mask.ozx at segmentation_level. Added level, mask_level (registration_level - segmentation_level, same arithmetic as fieldfrac.py), and zarrnii_kwargs params.

  • qc_vessels_roi_zoom rule: Updated to the same ZarrNii pattern as qc_segmentation_roi_zoom — uses inputs["spim"].path (OME-Zarr) + datatype="vessels" mask.ozx + dseg_nii NIfTI + label_tsv. Both rules now share qc_segmentation_roi_zoom.py.

@akhanf
Copy link
Copy Markdown
Member

akhanf commented Apr 1, 2026

@copilot: one final edit, the intensity histograms that are linearly scaled in y have the max x and max y set too high perhaps use a percentile to set the bounds instead.

- lin_xlim: cap at p99_val * 1.05 instead of last non-zero bin
- lin_ylim: cap at max count within the visible x range * 1.05
  so the distribution body is visible, not dominated by a spike
- log and CDF panels keep the existing disp_max bounds

Agent-Logs-Url: https://github.com/khanlab/SPIMquant/sessions/7db49e48-7c6c-4c09-b466-650c2ca2aef3

Co-authored-by: akhanf <11492701+akhanf@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 1, 2026

@copilot: one final edit, the intensity histograms that are linearly scaled in y have the max x and max y set too high perhaps use a percentile to set the bounds instead.

Done in commit 544b60e.

For the linear histogram panel:

  • X-axis: now capped at p99_val * 1.05 (99th percentile intensity + 5% headroom) instead of the last non-zero bin, eliminating the long empty tail
  • Y-axis: now capped at the tallest bar within that visible x range * 1.05, so the body of the distribution fills the panel rather than being flattened by a background spike

The log-scale and CDF panels keep the existing disp_max (last non-zero bin) so the full distribution is still visible there.

@akhanf
Copy link
Copy Markdown
Member

akhanf commented Apr 1, 2026

@copilot: final fix, for any instance-level analyses (e.g. roisummary etc) always start by dropping all the instances with atlas label 0 (background), as these are outside the brain and shouldn't be used

Background regions (index == 0) are outside the brain and should be
excluded from the per-ROI summary bar charts.

Agent-Logs-Url: https://github.com/khanlab/SPIMquant/sessions/e1add057-49be-45a2-8dbf-1e8d402fd215

Co-authored-by: akhanf <11492701+akhanf@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 1, 2026

@copilot: final fix, for any instance-level analyses (e.g. roisummary etc) always start by dropping all the instances with atlas label 0 (background), as these are outside the brain and shouldn't be u...

Done in commit 9aa06df. In qc_roi_summary.py, rows with index == 0 (background) are now dropped immediately after loading the segstats TSV, before the label-name merge and any plotting, so background regions never appear in the bar charts.

@akhanf akhanf marked this pull request as ready for review April 1, 2026 19:26
@akhanf akhanf merged commit 6deed05 into main Apr 1, 2026
2 checks passed
@akhanf akhanf deleted the copilot/add-visual-qc-outputs branch April 1, 2026 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants