multiMap doesn't work in our pipeline with NXF 24.01.0-edge

Ask for help
,
1

Hi there!

On our nf-core/crisprseq pipeline tests’ are failing ever since a nextflow version change. It seems to be coming from the multiMap operator :

   Channel.fromSamplesheet("input")
    .multiMap { meta, fastq_1, fastq_2, reference, protospacer, template ->
        // meta.condition is part of the screening workflow and we need to remove it
        reads:   [ meta.id, meta - meta.subMap('condition') + [ single_end:fastq_2?false:true, self_reference:reference?false:true, template:template?true:false ], fastq_2?[ fastq_1, fastq_2 ]:[ fastq_1 ] ]
        reference:   [meta - meta.subMap('condition') + [ single_end:fastq_2?false:true, self_reference:reference?false:true, template:template?true:false ], reference]
        protospacer: [meta - meta.subMap('condition') + [ single_end:fastq_2?false:true, self_reference:reference?false:true, template:template?true:false ], protospacer]
        template:    [meta - meta.subMap('condition') + [ single_end:fastq_2?false:true, self_reference:reference?false:true, template:template?true:false ], template]
    }
    .set { ch_input }

as we use the operator in both workflows.
We now have this compilation error :

NXF_VER=24.01.0-edge nextflow run . -profile test_screening,docker --outdir ./results_screening                   
N E X T F L O W  ~  version 24.01.0-edge
Launching `./main.nf` [sick_gates] DSL2 - revision: 23593e18ec
ERROR ~ Module compilation error
- file : /Users/laurencekuhlburger/Documents/crisprseq/crisprseq/./workflows/crisprseq_targeted.nf
- cause: Unexpected input: '{' @ line 133, column 29.
   workflow CRISPRSEQ_TARGETED {
                               ^

1 error

NOTE: If this is the beginning of a process or workflow, there may be a syntax error in the body, such as a missing or extra comma, for which a more specific error message could not be produced.

All tests are failing in our PRs such as here : Template update 2.12 by mirpedrol · Pull Request #115 · nf-core/crisprseq · GitHub

Thank you! :slight_smile:
Laurence

1 Like
2

Trying to debug this issue further, it seems that the Elvis operator cannot be used with lists in the new version.

I could find a minimal example which fails:

a = "a"
test = a?[1]:[1, 2]

So, a workaround would be to use if/else instead of the Elvis operator when defining the channel reads:

if (fastq_2) {
    files = [ fastq_1, fastq_2 ]
} else {
    files = [ fastq_1 ]
}

I haven’t been able to find documentation regarding this change. It would be helpful to know what changed with Groovy 4 if anyone has more information.

3

Nobody expects the ‘{’!

There is a syntax error in workflow somewhere. Since it doesn’t affect 23.10.1 (latest) I might guess the newer version of Nextflow is more strict and catching an error.

It might be related to the switch to Groovy 4 that occurred with the latest edge release of Nextflow: Upgrade to Groovy 4 by pditommaso · Pull Request #4443 · nextflow-io/nextflow · GitHub

4

Adding some spaces might work? This works on my machine (famous last words…)

workflow {
    a = "a"
    test = a ? [1] : [1, 2]
    println test
}

edit: or were you trying to use the safe index operator? Groovy Goodness: Safe Index Based Access For Lists, Arrays and Maps - Messages from mrhaki