Merge with master@00a53b97 [ci fast]

Author: pditommaso

.github/workflows/build.yml

@@ -52,22 +52,7 @@ jobs:
                 echo "commit_message=$COMMIT_MESSAGE" | head -n 1 >> $GITHUB_OUTPUT
             fi
 
-      - name: Get changed files
-        id: changed-files
-        uses: tj-actions/changed-files@v43
-        with:
-          files_ignore: docs/**
-
-      - name: List all changed files
-        env:
-            ALL_CHANGED_FILES: ${{ steps.changed-files.outputs.all_changed_files }}
-        run: |
-            for file in ${ALL_CHANGED_FILES}; do
-                echo "$file was changed"
-            done
-
       - name: Setup env
-        if: steps.changed-files.outputs.any_changed == 'true'
         run: |
              rm -f $HOME/.gitconfig;
              mkdir -p "$HOME/.nextflow";
@@ -76,7 +61,6 @@ jobs:
           NXF_GITHUB_ACCESS_TOKEN: ${{ secrets.NXF_GITHUB_ACCESS_TOKEN }}
 
       - name: Setup Java ${{ matrix.java_version }}
-        if: steps.changed-files.outputs.any_changed == 'true'
         uses: actions/setup-java@v4
         with:
           java-version: ${{matrix.java_version}}
@@ -85,11 +69,9 @@ jobs:
           cache: gradle
 
       - name: Compile
-        if: steps.changed-files.outputs.any_changed == 'true'
         run: make assemble
 
       - name: Test
-        if: steps.changed-files.outputs.any_changed == 'true'
         run: |
             env | sort
             # configure test env
@@ -114,18 +96,16 @@ jobs:
 
       - name: Publish tests report
         uses: actions/upload-artifact@v4
-        if: steps.changed-files.outputs.any_changed == 'true' && always()
         with:
           name: report-unit-tests-jdk-${{ matrix.java_version }}
           path: |
             **/build/reports/tests/test
 
     outputs:
-      any_changed: ${{ steps.changed-files.outputs.any_changed }}
       commit_message: ${{ steps.get_commit_message.outputs.commit_message }}
 
   test:
-    if: ${{ !contains(needs.build.outputs.commit_message, '[ci fast]') && needs.build.outputs.any_changed == 'true' }}
+    if: ${{ !contains(needs.build.outputs.commit_message, '[ci fast]') }}
     needs: build
     runs-on: ubuntu-latest
     timeout-minutes: 90

changelog.txt

@@ -1,5 +1,15 @@
 NEXTFLOW CHANGE-LOG
 ===================
+
+24.10.5 - 4 Mar 2025
+- Improve Google Batch executor stability and error handling (#5690) [1a9e8bf5]
+- Fix Mount input file buckets in task arrays for Google Batch (#5739) [0d98442d]
+- Fix Google Batch task array causes process to fail (#5780) [d4c1ba12]
+- Fixing bug when execution with stub and no stub defined (#5473) [ba5ab069]
+- Fix Conda directive is not honoured by task array (#5704) [63124eb8]
+- Fix Use unbound queue for transfer thread pool (#5700) [d57a85d0]
+- Bump [email protected] [ee9c1fd4]
+
 25.01.0-edge - 12 Feb 2025
 - Add DeadlineExceededException to Google Batch retryable exceptions [944f48f9]
 - Add Fusion token validation (#5614) [1dcb18d6]
@@ -35,6 +45,11 @@ NEXTFLOW CHANGE-LOG
 - Bump net.minidev:json-smart:2.5.2 [b5c4faf4]
 - Bump netty-common:4.1.118.Final [8574e243]
 
+24.10.4 - 20 Jan 2025
+- Fix CLI params normalization (#5661) [42a9d8a0]
+- Fix azure repos when clone URL is used (#5667) [eb092c9f]
+- Fix Prevent the use existing work dir (#5682) [7ed233ea]
+
 24.12.0-edge - 20 Jan 2025
 - Add Warning when Cloudinfo fails in Google Batch machine type selection (#5649) [5aef532f]
 - Add rate limiter to wave requests (#5608) [ecf68294]

docs/_static/.DS_Store

@@ -80,33 +80,27 @@ See {ref}`channel-factory` for the full list of channel factories.
 
 ## Operators
 
-Operators are methods that consume and produce channels. Because channels are asynchronous, operators are necessary to manipulate the values in a channel, without using a process. As a result, operators are useful for implementing the "glue logic" between processes.
+Channel operators, or _operators_ for short, are functions that consume and produce channels. Because channels are asynchronous, operators are necessary to manipulate the values in a channel, aside from using a process. As a result, operators are useful for implementing the _glue logic_ between processes.
 
-See {ref}`operator-page` for the full list of operators. If you are new to Nextflow, here are some commonly-used operators to learn first:
+Commonly used operators include:
 
-Filtering:
+- {ref}`operator-combine`: emit the combinations of two channels
 
-- {ref}`operator-filter`: select all values in a channel that satisfy a condition
-- {ref}`operator-first`: select the first value in a channel
-- {ref}`operator-take`: select the first *n* values in a channel
-- {ref}`operator-unique`: select the unique values in a channel (i.e. remove duplicates)
+- {ref}`operator-collect`: collect the values from a channel into a list
 
-Transforming:
+- {ref}`operator-filter`: select the values in a channel that satisfy a condition
 
-- {ref}`operator-collect`: collect the values from a channel into a list
-- {ref}`operator-grouptuple`: group the values from a channel based on a grouping key
-- {ref}`operator-map`: transform each value from a channel with a mapping function
-- {ref}`operator-reduce`: accumulate each value from a channel into a single value
+- {ref}`operator-flatMap`: transform each value from a channel into a list and emit each list 
+element separately
 
-Combining multiple channels:
+- {ref}`operator-grouptuple`: group the values from a channel based on a grouping key
 
-- {ref}`operator-combine`: emit the combinations of two channels
-- {ref}`operator-concat`: emit the values from multiple channels (in the order in which the channels were given)
 - {ref}`operator-join`: join the values from two channels based on a matching key
-- {ref}`operator-mix`: emit the values from multiple channels (in the order in which items arrive)
 
-Miscellaneous:
+- {ref}`operator-map`: transform each value from a channel with a mapping function
+
+- {ref}`operator-mix`: emit the values from multiple channels
 
-- {ref}`operator-ifempty`: emit a channel, or a default value if the channel is empty
-- {ref}`operator-set`: assign a channel to a variable
 - {ref}`operator-view`: print each value in a channel to standard output
+
+See {ref}`operator-page` for the full list of operators.

docs/_static/report-resource-cpu-noheader.png

@@ -151,6 +151,38 @@ process foo {
 Refer to the source code of Nextflow's built-in executors to see how to implement the various components of an executor. You might be able to implement most of your executor by simply reusing existing code.
 :::
 
+### Filesystems
+
+Plugins can define custom filesystems that can be used by Nextflow to interact with external storage systems using a single interface. For more information about accessing remote files, see {ref}`remote-files`.
+
+To implement a custom filesystem, create a class in your plugin that extends [`FileSystemProvider`](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/nio/file/spi/FileSystemProvider.html). Implement the `getScheme()` method to define the URI scheme for your filesystem:
+
+```groovy
+import java.nio.file.spi.FileSystemProvider
+
+class MyFileSystemProvider extends FileSystemProvider {
+
+    @Override
+    String getScheme() {
+        return 'myfs'
+    }
+
+    // ...
+}
+```
+
+You can then use this filesystem in your pipeline:
+
+```nextflow
+input = file('myfs://path/to/input/file.txt')
+```
+
+See [Developing a Custom File System Provider](https://docs.oracle.com/javase/8/docs/technotes/guides/io/fsp/filesystemprovider.html) for more information. Refer to the `nf-https` module (`XFileSystemProvider`) or `nf-amazon` plugin (`S3FileSystemProvider`) in the Nextflow source code for examples of custom filesystems.
+
+:::{tip}
+Custom filesystems are an advanced type of plugin extension. Before creating a new filesystem, make sure that your use case can't already be supported through an existing filesystem such as HTTP or S3.
+:::
+
 ### Functions
 
 :::{versionadded} 22.09.0-edge
@@ -357,6 +389,10 @@ Here is an example meta file for a plugin release:
 }
 ```
 
+:::{note}
+The plugin version should be a valid [semantic version](https://semver.org/).
+:::
+
 (testing-plugins)=
 
 ## Testing plugins