Skip to content

Added concurrency using lst-parallel.py#34

Open
pradhanhitesh wants to merge 4 commits intoCompImg:mainfrom
pradhanhitesh:added-concurrency
Open

Added concurrency using lst-parallel.py#34
pradhanhitesh wants to merge 4 commits intoCompImg:mainfrom
pradhanhitesh:added-concurrency

Conversation

@pradhanhitesh
Copy link

@pradhanhitesh pradhanhitesh commented Jan 16, 2025

Description

The lst-parallel.py script helps execute multiple concurrent lst commands using the CPU and improves processing time for large datasets.

  1. The script is designed to take folder paths (--t1, --flair, --output) and the number of concurrent (--max_workers) processes as input to run. All other lst flags can also be added (such as --segment-only, --threshold, --stripped, --clipping, --fast-mode, --probability-map), which will subsequently be added while executing. In this script, I have added --save_temp, a flag that invokes the --temp flag for lst with the appropriate file path to save the temporary files.
  2. The script generates three folders that are: Output, Log, and Temp (if --save_temp is set). In the Output folder, subject-wise files are stored. In the Log folder, logs generated by the lst command is added and stored, along with time taken to finish the processing. If save_temp is set, Temp folder stores all the temporary files, subject-wise.
  3. The script also compiles the stats for all the subjects and stores in the under name lst_stats.csv in the Output directory.

Related Issue

Fixes #33

Type of Change

  • New feature

Checklist

  • I have performed a self-review of my code.
  • My changes generate no new warnings or errors.
  • I have updated the documentation if necessary.
  • My code follows the repository's coding standards.
  • I have included the relevant labels for this PR.

Additional Notes

Currently, I have not tested --annotate-only and --existing_seg flags in the lst-parallel.py script yet. Therefore, I have excluded them from the script, including --lesion-threshold. To run lst-parallel.py, pandas package is required, which is not supplied initially through pip install -e .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement] Implement Concurrency to Optimize Processing Time for Large File Operations

1 participant