Specifying task.cpus for modules?

Ask for help
1

I’d like to call a single module multiple times with different numbers of CPUs (e.g. varying the number of files that FASTQC parallelizes over). Is there a way to do this? I can’t figure out how to specify task.cpus outside the process definition.

2

The variation is based on what? If you want to set the number of CPUs based on the task number, check the snippet below:

process FOO {
  debug true
  cpus { (task.index == 1) ? "1" : "2" }

  input:
  val x

  output:
  stdout

  script:
  """
  echo $task.cpus
  """
}

workflow {
  Channel
    .of(1..10)
    | FOO
}

The snippet above will set cpus to 1 for the first process instance, and 2 for the remaining ones.

Output:

CleanShot 2024-06-12 at 13.21.08@2x
CleanShot 2024-06-12 at 13.21.08@2x1432×1026 89.8 KB

3

Thanks Marcel. The variation is based on the context in which the process is called, specifically which subworkflow is calling it.

Currently I’m doing this with params:

process FASTQC {
    label "FASTQC"
    cpus "${params.cpus}"
    memory "${params.mem}"
    input:
        path(reads)
    output:
        path("*.html"), emit: html
        path("*.zip"), emit: zip
    shell:
        '''
        fastqc -t !{params.cpus} !{reads}
        '''
}

And then specifying these with include:

 include { FASTQC } from "../modules/local/fastqc" addParams(cpus: "${params.fastqc_cpus}", mem: "${params.fastqc_mem}")

This works but is a little clunky, I’d be interested in alternative solutions if they exist.

4

Oh, I see. Thanks for the snippet, @willbradshaw. It’s clearer now what you’re looking for. addParams is indeed the Nextflow way of handling this. Right now I can’t think of another solution, sorry.