S3 object tagging

Ask for help
1

I am trying to use object tagging to tag my S3 outputs so that I can implement tag-based S3 lifecycle policies. I am following the doc at https://www.nextflow.io/docs/latest/process.html where it says "*Allow the association of arbitrary tags with the published file e.g. tags: [FOO: 'Hello world'].
However, this led to the pipeline stalling, so my syntax is probably wrong. I am on Nextflow 23.10.1.

publishDir = [
            path: { "${params.output_dir}" },
            mode: "copy",
            overwrite: true,
            tags: ["file_type": "fastq"]
        ]

where the added line is the last line tags: ["file_type": "fastq"] .
Any help is appreciated. The pipeline stalled (previously working) and gives no error, so it’s hard to troubleshoot. Thanks!

2

Welcome, @helen !

Based on this test, I suggest you to remove the quotes on file_type and check if it works.

publishDir = [
            path: { "${params.output_dir}" },
            mode: "copy",
            overwrite: true,
            tags: [file_type: "fastq"]
        ]

Regarding Nextflow becoming unresponsive, could you share the log file?

1 Like
3

Thanks for the reply. It turned out that the pipeline stalled due to an entirely different issue. The syntax for tags works as is.

4

Thanks for reporting it back, Helen! Could you please share what was causing the issue?