Optional publishable output?

Ask for help
1

I have a process allowing the user to optionally generate a statistics file in addition to the main process output. I’d like to have it returned as a process output and published if it is generated, but to have nothing published and no error from a failure to generated expected process outputs if the user does not specify for it to be created. Is this possible in Nextflow?

2

Is it a problem if the output filenames are similar? If it’s OK, you can use the arity option. See the example below:

process FOO {
  input:
  path ifile

  output:
  // exactly one or two files are expected
  path('output_*.txt', arity: '1..2')


  script:
  """
  # ./do_something_with ${ifile} --output something.txt
  # ./do_something_else ${ifile} --output somethingelse.txt
  touch something.txt
  touch somethingelse.txt

  mv something.txt output_something.txt
  mv somethingelse.txt output_somethingelse.txt
  """
}

workflow {
  Channel
    .of(file('foo.txt'))
    | FOO
}

Output

tree work/c1/32c4bd0c9be8d84641010fb7475bac/
work/c1/32c4bd0c9be8d84641010fb7475bac/
├── foo.txt -> /Users/mribeirodantas/foo.txt
├── output_something.txt
└── output_somethingelse.txt

1 directory, 3 files

Commenting one of the touch (but it could be a conditional expression, etc):

process FOO {
  input:
  path ifile

  output:
  // exactly one or two files are expected
  path('output_*.txt', arity: '1..2')


  script:
  """
  # ./do_something_with ${ifile} --output something.txt
  # ./do_something_else ${ifile} --output somethingelse.txt
  touch something.txt
  # touch somethingelse.txt

  mv something.txt output_something.txt
  # mv somethingelse.txt output_somethingelse.txt
  """
}

workflow {
  Channel
    .of(file('foo.txt'))
    | FOO
}

tree work/e2/7dbaa857eb62ed98e3be3064402867/
work/e2/7dbaa857eb62ed98e3be3064402867/
├── foo.txt -> /Users/mribeirodantas/foo.txt
└── output_something.txt

1 directory, 2 files
3

That could possibly be made to work, but the challenge is that the files produced are of distinct filetypes, and the filename extensions are used downstream to process them differently.

4 
...
  path('output_*', arity: '1..2')
...

:smile:

5

Oh thanks, I’ll give that a try!

1 Like
6

I would not recommend using the arity to express this optional output because it’s not quite expressing what you mean. Instead I would specify a separate optional output channel and then publish that channel using the new workflow output definition.

That being said, if you aren’t willing to use this new syntax, the arity is probably the next best thing.

1 Like
7

There’s also the optional: true for files that are only sometimes created (e.g. with a flag).
https://www.nextflow.io/docs/latest/process.html#additional-options

1 Like
8

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.