Typed processes

Signed-off-by: Ben Sherman <[email protected]>

Author: bentsherman

docs/migrations/25-10.md

@@ -16,16 +16,16 @@ The `params` block is a new way to declare pipeline parameters in a Nextflow scr
 
 ```nextflow
 params {
-  // Path to input data.
-  input: Path
+    // Path to input data.
+    input: Path
 
-  // Whether to save intermediate files.
-  save_intermeds: Boolean = false
+    // Whether to save intermediate files.
+    save_intermeds: Boolean = false
 }
 
 workflow {
-  println "params.input = ${params.input}"
-  println "params.save_intermeds = ${params.save_intermeds}"
+    println "params.input = ${params.input}"
+    println "params.save_intermeds = ${params.save_intermeds}"
 }
 ```
 
@@ -39,33 +39,73 @@ Type annotations are a way to denote the *type* of a variable. They help documen
 
 ```nextflow
 workflow RNASEQ {
-  take:
-  reads: Channel<Path>
-  index: Value<Path>
+    take:
+    reads: Channel<Path>
+    index: Value<Path>
 
-  main:
-  samples_ch = QUANT( reads, index )
+    main:
+    samples_ch = QUANT( reads, index )
 
-  emit:
-  samples: Channel<Path> = samples_ch
+    emit:
+    samples: Channel<Path> = samples_ch
 }
 
 def isSraId(id: String) -> Boolean {
-  return id.startsWith('SRA')
+    return id.startsWith('SRA')
+}
+
+// feature flag required for typed processes
+nextflow.preview.types = true
+
+process fastqc {
+    input:
+    (id, fastq_1, fastq_2): Tuple<String,Path,Path>
+
+    output:
+    logs = tuple(id, file('fastqc_logs'))
+
+    script:
+    """
+    mkdir fastqc_logs
+    fastqc -o fastqc_logs -f fastq -q ${fastq_1} ${fastq_2}
+    """
 }
 ```
 
 The following declarations can be annotated with types:
 
 - Pipeline parameters (the `params` block)
 - Workflow takes and emits
+- Process inputs and outputs
 - Function parameters and returns
 - Local variables
 - Closure parameters
 - Workflow outputs (the `output` block)
 
 Type annotations can refer to any of the {ref}`standard types <stdlib-types>`.
 
+Some types have *generic type parameters*, which allow them to be reused in a type-safe manner with different types of data. For example:
+
+- The generic type `E` in `List<E>` and `Channel<E>` refers to the type of the elements in the list or channel
+
+- The generic types `K` and `V` in `Map<K,V>` refer to the types of keys and values in the map
+
+Here are some concrete examples of types that use type parameters:
+
+```nextflow
+// List<E> where E is String
+def sequences: List<String> = ['ATCG', 'GCTA', 'TTAG']
+
+// List<E> where E is Path  
+def fastqs: List<Path> = [file('sample1.fastq'), file('sample2.fastq')]
+
+// Map<K,V> where K is String and V is Integer
+def readCounts: Map<String,Integer> = [sample1: 1000, sample2: 1500]
+
+// Channel<E> where E is Path
+def ch_bams: Channel<Path> = channel.fromPath('*.bam')
+```
+
 Type annotations can be appended with `?` to denote that the value can be `null`:
 
 ```nextflow
@@ -78,6 +118,8 @@ In the type system, queue channels are represented as `Channel`, while value cha
 Nextflow supports Groovy-style type annotations using the `<type> <name>` syntax, but this approach is deprecated in {ref}`strict syntax <strict-syntax-page>`. While Groovy-style annotations remain valid for functions and local variables, the language server and `nextflow lint` automatically convert them to Nextflow-style annotations during code formatting.
 :::
 
+See {ref}`migrating-static-types` for details.
+
 ## Enhancements
 
 <h3>Nextflow plugin registry</h3>
@@ -113,7 +155,7 @@ workflow {
 
 This syntax is simpler and easier to use with the {ref}`strict syntax <strict-syntax-page>`. See {ref}`workflow-handlers` for details.
 
-<h3>Improved handling of dynamic directives</h3>
+<h3>Simpler syntax for dynamic directives</h3>
 
 The {ref}`strict syntax <strict-syntax-page>` allows dynamic process directives to be specified without a closure:
 
@@ -131,6 +173,8 @@ process hello {
 }
 ```
 
+Dynamic process settings in configuration files must still be specified with closures.
+
 See {ref}`dynamic-directives` for details.
 
 <h3>Configurable date formatting</h3>

docs/process-typed.md

@@ -0,0 +1,252 @@
+(process-typed-page)=
+
+# Processes (typed)
+
+:::{versionadded} 25.10.0
+:::
+
+:::{note}
+This feature requires the {ref}`strict syntax <strict-syntax-page>` to be enabled (`NXF_SYNTAX_PARSER=v2`).
+:::
+
+:::{note}
+Typed processes require the `nextflow.preview.types` feature flag to be enabled in every script that uses them.
+:::
+
+Typed processes use a new syntax for inputs and outputs that is based on static types:
+
+```nextflow
+process hello {
+    input:
+    message: String
+
+    output:
+    file('hello.txt')
+
+    script:
+    """
+    echo '${message}' > hello.txt
+    """
+}
+```
+
+See {ref}`syntax-process-typed` for a full description of the process syntax. See {ref}`migrating-static-types` for more information on migrating existing code to static types.
+
+## Inputs
+
+The `input:` section is used to declare the inputs of a process. An input declaration in a typed process consists of a name and a type:
+
+```nextflow
+process fastqc {
+    input:
+    (meta, fastq): Tuple<Map,Path>
+    extra_args: String
+
+    script:
+    """
+    echo 'meta: ${meta}`
+    echo 'fastq: ${fastq}'
+    echo 'extra_args: ${extra_args}'
+    """
+}
+```
+
+Any of the {ref}`standard types <stdlib-types>` can be used as type annotations, except for the dataflow types (`Channel` and `Value`) which can only be used in workflows.
+
+### File inputs
+
+Inputs of type `Path` or a collection of `Path` (e.g. `Set<Path>`) are automatically staged into the task directory.
+
+By default, the task will fail if any input receives a `null` value. You can mark an input as nullable by appending `?` to the type annotation:
+
+```nextflow
+process cat_opt {
+    input:
+    input: Path?
+
+    stage:
+    stageAs 'input.txt', input
+
+    output:
+    stdout()
+
+    script:
+    '''
+    [[ -f input.txt ]] && cat input.txt || echo 'empty input'
+    '''
+}
+```
+
+### Stage directives
+
+The `stage:` section can be specified after the `input:` section. You can use it to specify custom staging behavior using *stage directives*. These directives serve the same purpose as input qualifiers such as `env` and `stdin` in the legacy syntax.
+
+The `env` directive declares an environment variable in terms of task inputs:
+
+```nextflow
+process echo_env {
+    input:
+    hello: String
+
+    stage:
+    env 'HELLO', hello
+
+    script:
+    '''
+    echo "$HELLO world!"
+    '''
+}
+```
+
+The `stdin` directive defines the standard input of the task script:
+
+```nextflow
+process cat {
+    input:
+    message: String
+
+    stage:
+    stdin message
+
+    script:
+    """
+    cat -
+    """
+}
+```
+
+The `stageAs` directive stages an input file (or files) under a custom file pattern:
+
+```nextflow
+process blast {
+    input:
+    fasta: Path
+
+    stage:
+    stageAs 'query.fa', fasta
+
+    script:
+    """
+    blastp -query query.fa -db nr
+    """
+}
+```
+
+The file pattern can also reference task inputs:
+
+```nextflow
+process grep {
+    input:
+    id: String
+    fasta: Path
+
+    stage:
+    stageAs "${id}.fa", fasta
+
+    script:
+    """
+    cat ${id}.fa | grep '>'
+    """
+}
+```
+
+See {ref}`process-reference-typed` for the set of available stage directives.
+
+## Outputs
+
+The `output:` section is used to declare the outputs of a typed process. An output declaration in a typed process consists of a name, an optional type, and an output value:
+
+```nextflow
+process echo {
+    input:
+    message: String
+
+    output:
+    out_env: String = env('MESSAGE')
+    out_file: Path = file('message.txt')
+    out_std: String = stdout()
+
+    script:
+    """
+    export MESSAGE='${message}'
+
+    echo \$MESSAGE > message.txt
+
+    cat message.txt
+    """
+}
+```
+
+When there is only one output, the name can be omitted:
+
+```nextflow
+process echo {
+    input:
+    message: String
+
+    output:
+    stdout()
+
+    script:
+    """
+    echo '${message}'
+    """
+}
+```
+
+See {ref}`process-reference-typed` for the set of available output functions.
+
+### File outputs
+
+You can use the `file()` and `files()` functions in the `output:` section to get a single file or collection of files from the task directory.
+
+By default, the `file()` function will fail if the specified file is not present in the task directory. You can specify `optional: true` to allow the file to be missing, in which case the `file()` function will return `null`. For example:
+
+```nextflow
+process foo {
+    output:
+    file('output.txt', optional: true)
+
+    script:
+    """
+    exit 0
+    """
+}
+```
+
+## Topics
+
+The `topic:` section is used to emit values to a {ref}`topic channel <channel-topic>`. A topic emission consists of an output value and a topic name:
+
+```nextflow
+process cat {
+    input:
+    message: Path
+
+    output:
+    stdout()
+
+    topic:
+    tuple('bash', eval('bash --version')) >> 'versions'
+    tuple('cat', eval('cat --version')) >> 'versions'
+
+    script:
+    """
+    cat ${message}
+    """
+}
+```
+
+Topic emissions can use the same {ref}`output functions <process-reference-typed>` that are available in the `output:` section.
+
+## Script
+
+The `script:` and `exec:` sections behave the same way as {ref}`legacy processes <process-script>`.
+
+## Stub
+
+The `stub:` section behaves the same way as {ref}`legacy processes <process-stub>`.
+
+## Directives
+
+Directives behave the same way as {ref}`legacy processes <process-directives>`.

docs/reference/feature-flags.md

@@ -60,3 +60,9 @@ Feature flags are used to introduce experimental or other opt-in features. They
   This feature flag is no longer required to use topic channels.
   :::
 : When `true`, enables {ref}`topic channels <channel-topic>` feature.
+
+`nextflow.preview.types`
+: :::{versionadded} 25.10.0
+  :::
+: When `true`, enables the use of {ref}`typed processes <process-typed-page>`.
+: This feature flag must be enabled in every script that uses typed processes. Legacy processes can not be defined in scripts that enable this feature flag.