Skip to content

[ENH] Add profile full_pipeline + fix stubrun #98

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
arnaudbore wants to merge 6 commits into
base: dev
Choose a base branch
from
Open

[ENH] Add profile full_pipeline + fix stubrun #98

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
c365f18
add profile full_pipeline + another way to run stubrun
arnaudbore Mar 26, 2026
6a5b5aa
fix profile for test
arnaudbore Mar 26, 2026
e63d92d
fix stubrun profile in test
arnaudbore Mar 26, 2026
c780ca3
fix action
arnaudbore Mar 26, 2026
ef8b0d8
update stubrun + config bundleseg
arnaudbore Mar 26, 2026
d463ae8
fix ascii and profiles
arnaudbore Mar 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion conf/modules.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,3 @@ includeConfig './modules/tractometry.config'
includeConfig './modules/harmonization.config'
includeConfig './modules/output_orig_space.config'
includeConfig './modules/output_template_space.config'
includeConfig './modules/stubrun.config'
8 changes: 7 additions & 1 deletion conf/modules/bundle_seg.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,13 @@ process {
}

withName: ".*:BUNDLE_SEG:BUNDLE_RECOGNIZE" {
memory = { 3 * tractograms.inject( 0.Gb ){ total, trk -> total + (trk.size() as MemoryUnit) } + 500.Mb }
memory = {
def trk_list = (tractograms instanceof List) ? tractograms : [tractograms]
def total = trk_list.inject(0.Gb) { total, trk ->
total + ((trk?.exists()) ? (trk.size() as MemoryUnit) : 0.B)
}
3 * total + 500.Mb
}
ext.run_qc = true
publishDir = [
path: { meta.session ? "${params.outdir}/${meta.id}/${meta.session}/dwi/bundles/" : "${params.outdir}/${meta.id}/dwi/bundles/" },
Expand Down
3 changes: 3 additions & 0 deletions conf/reproducible.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,7 @@
*/

process {
withName: '.*' {
Copy link
Copy Markdown
Collaborator

@levje levje Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this for? Some tasks will probably fail with a single CPU.

Copy link
Copy Markdown
Collaborator

@AlexVCaron AlexVCaron Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also not specific. Following docs, it falls off behind all other selectors. Report if it's cornering use-cases correctly. From my gist, this gets applied if no other config gets loaded

cpus = 1
}
}
5 changes: 4 additions & 1 deletion nextflow.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -415,6 +415,9 @@ profiles {
skip_preproc {
includeConfig 'conf/skip_preproc.config'
}
stubRun {
includeConfig 'conf/modules/stubrun.config'
}
test { includeConfig 'conf/test.config' }
test_full { includeConfig 'conf/test_full.config' }
}
Expand Down Expand Up @@ -519,7 +522,7 @@ manifest {
mainScript = 'main.nf'
defaultBranch = 'main'
nextflowVersion = '!>=25.04.6'
version = '1.0dev'
version = '0.1.0'
doi = ''
}

Expand Down
2 changes: 1 addition & 1 deletion nf-test.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ config {
ignore '.venv/**/*'

// run all test with defined profile(s) from the main nextflow.config
profile "test,full_pipeline"
profile "test,full_pipeline,stubRun"
Copy link
Copy Markdown
Collaborator

@AlexVCaron AlexVCaron Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Syntax opinion ? (Follow snake_case)


// list of filenames or patterns that should be trigger a full test run
triggers 'nextflow.config', 'nf-test.config', 'conf/test.config', 'tests/nextflow.config', 'tests/.nftignore'
Expand Down
15 changes: 10 additions & 5 deletions subworkflows/local/utils_nfcore_sf-tractomics_pipeline/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,11 +60,16 @@ workflow PIPELINE_INITIALISATION {
command = "nextflow run ${workflow.manifest.name} -profile <docker/singularity/.../institute> --input samplesheet.csv --outdir <OUTDIR>"
before_text = """
-\033[2m----------------------------------------------------------------------------------\033[0m-
\033[0;32m _.--'"'.\033[0m
\033[0;32m( ( ( )\033[0m
\033[0;33m(o)_ ) )\033[0m
\033[0;32m (o)_.'\033[0m
\033[0;32m )/\033[0m
\033[0;32m _---~~(~~-_.\033[0m
\033[0;32m _{ ) )\033[0m
\033[0;32m , ) -~~- ( ,-' )_\033[0m
\033[0;32m ( `-,_..`., )-- '_,)\033[0m
\033[0;32m( ` _) ( -~( -_ `, }\033[0m
\033[0;32m(_- _ ~_-~~~~`, ,' )\033[0m
\033[0;32m `~ -^( __;-,((()))\033[0m
\033[0;32m ~~~~ {_ -_(())\033[0m
\033[0;32m `\\ }\033[0m
\033[0;32m { }\033[0m

\033[0;34m __ ___ ___ __ __ __ ___ __ __ __ __ \033[0m
\033[0;34m (__ |__ ___ | |__) |__| | | | | |\\/| | / (__ \033[0m
Expand Down
4 changes: 2 additions & 2 deletions tests/default.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@
"scilpy": "2.2.2"
},
"Workflow": {
"scilus/sf-tractomics": "v1.0dev"
"scilus/sf-tractomics": "v0.1.0"
}
},
[
Expand Down Expand Up @@ -217,6 +217,6 @@
"nf-test": "0.9.3",
"nextflow": "25.10.2"
},
"timestamp": "2026-03-13T12:58:09.876806632"
"timestamp": "2026-03-26T16:15:03.763505689"
}
}
6 changes: 5 additions & 1 deletion workflows/sf-tractomics.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,10 @@ workflow SF_TRACTOMICS {
ch_bet_probability = channel.empty()
ch_synthstrip_weights = channel.empty()

if (workflow.profile.contains('reproducible') && workflow.profile.contains('gpu')) {
error "\033[0;31mERROR: Profiles 'reproducible' and 'gpu' are not compatible and cannot be used together. Please remove gpu if you want reproducible results.\033[0m"
}

/* Load topup config if provided */
if ( params.config_topup ) {
if ( file(params.config_topup).exists()) {
Expand Down Expand Up @@ -169,7 +173,7 @@ workflow SF_TRACTOMICS {
if ( params.run_bundle_seg ) {
BUNDLE_SEG(
TRACTOFLOW.out.dti_fa,
ch_input_tracking_qc.map{ meta, trk -> [meta, [trk]] },
ch_input_tracking_qc.map { meta, trk -> [meta, (trk instanceof List ? trk : [trk])] },
channel.empty(),
[
"run_easyreg": false, // BundleSeg does not support easyreg, so we set it to false to avoid confusion
Expand Down
Loading