How to join branched channel with multi channel?

[complexgenome (S)]

Hi There,

I’d like to do a join on a branched channel and a multi channel.

I get output from a applybqsr process as:

output:
		tuple val(meta) ,path("${patient_id}.sorted.markdup.recal.bam"), path("${patient_id}.sorted.markdup.recal.bai"), emit: recal_bam_bai

I branch it out as:

applybqsr.out.recal_bam_bai.branch { meta, bam, bai ->
        normal: meta.tissue == "normal"
                    return [ meta.timepoint, meta, bam, bai ]
        tumor:  meta.tissue == "tumor"
                    return [ meta.timepoint, meta, bam, bai ]
        other: true
    }.set { ch_branched }

I’ve another process where consensus regions are generated where output is defined as:

output:

tuple val(meta), path("regions.txt"), emit: regions_txt
       tuple val(meta),  path("consensus.vcf"), emit: consensus_vcf

I’d like to send only tumor applybqsr and regions file to a process.
I try to do a join on these as the following:

consensus.out.regions_txt.map{meta,regions_file->[ "${meta.timepoint}", meta,regions_file ]}
.join(ch_branched.tumor.map{meta,bam_bai ->["${meta.timepoint}",meta,bam_bai]}, failOnMismatch:true, failOnDuplicate:true)
.multiMap{pid, meta1, regions_file, meta2, bam_bai ->
regions:[meta1, regions_file]
tumor_recal:[meta2, bam_bai]
}.set { ch_input_tumor_consensus }

I get error as:

ERROR ~ Invalid method invocation call with arguments: [MM-0486-T-01, [batch:SEMA-MM-001, timepoint:MM-0486-T-01, tissue:tumor, sequencing_type:wes], /mnt/data1/users/sanjeev/nextflow/meta_structure/work/0e/5efc4c5ad6c9776e3db00d2165ece2/MM-0486-T-01_T.sorted.markdup.recal.bam, /mnt/data1/users/sanjeev/nextflow/meta_structure/work/0e/5efc4c5ad6c9776e3db00d2165ece2/MM-0486-T-01_T.sorted.markdup.recal.bai] (java.util.ArrayList) on _closure21 type
– Check ‘.nextflow.log’ file for details

How do I resolve it?

[mribeirodantas (Marcel Ribeiro-Dantas)]

Invalid method invocation call with arguments usually means your channel looks differently from what you think. Try doing the join alone, before doing multiMap. Use view to see the channel content, confirm it looks like what you’re describing with the multiMap. There’s something you’re doing that makes the channel not look like what you want for the channel operators

[complexgenome (S)]

@mribeirodantas
I’m sorry that I really don’t understand how map, multiMap, join work in the nextflow.

[complexgenome (S)]

@mribeirodantas
When I only use join, I run into similar error:

consensus.out.regions_txt.map{meta,regions_file->[ "${meta.timepoint}", meta,regions_file  ]}
.join(ch_branched.tumor.map{meta,bambai ->["${meta.timepoint}",meta,bambai]}, failOnMismatch:true, failOnDuplicate:true)
.view()

Error:

ERROR ~ Invalid method invocation call with arguments: [MM-0486-T-01, [batch:SEMA-MM-001, timepoint:MM-0486-T-01, tissue:tumor, sequencing_type:wes], /mnt/data1/users/sanjeev/nextflow/meta_structure/work/0e/5efc4c5ad6c9776e3db00d2165ece2/MM-0486-T-01_T.sorted.markdup.recal.bam, /mnt/data1/users/sanjeev/nextflow/meta_structure/work/0e/5efc4c5ad6c9776e3db00d2165ece2/MM-0486-T-01_T.sorted.markdup.recal.bai] (java.util.ArrayList) on _closure21 type

How do I do what?

[mribeirodantas (Marcel Ribeiro-Dantas)]

Do you get this error without the join, just by doing the maps ?

[complexgenome (S)]

@mribeirodantas

Maps work with consensus’ output/channel

consensus.out.regions_txt.map{meta,regions_file->[ "${meta.timepoint}", meta,regions_file  ]}
.view()

But when I try with the branched channel it complains for the error I shared above.

ch_branched.tumor.map{meta,bam,bai ->["${meta.timepoint}",meta,bam,bai]}
.view()

Error:

ERROR ~ Invalid method invocation call with arguments: [MM-0486-T-01, [batch:SEMA-MM-001, timepoint:MM-0486-T-01, tissue:tumor, sequencing_type:wes], /mnt/data1/users/sanjeev/nextflow/meta_structure/work/0e/5efc4c5ad6c9776e3db00d2165ece2/MM-0486-T-01_T.sorted.markdup.recal.bam, /mnt/data1/users/sanjeev/nextflow/meta_structure/work/0e/5efc4c5ad6c9776e3db00d2165ece2/MM-0486-T-01_T.sorted.markdup.recal.bai] (java.util.ArrayList) on _closure20 type

[complexgenome (S)]

I’m able to join as:

consensus.out.regions_txt.map{meta,regions_file->[ "${meta.timepoint}", meta,regions_file  ]}
.join(ch_branched.tumor.map{tp,meta,bam,bai ->["${meta.timepoint}",meta,bam,bai]})
.view()

What’s the use of multiMap.
Further code could be:

.multiMap{pid, meta1, regions_file, meta2, bam,bai ->
regions:[meta1, regions_file]
tumor_recal:[meta2, bam,bai]
}.set { ch_input_tumor_consensus }

I’m unable to move forward.

[mribeirodantas (Marcel Ribeiro-Dantas)]

@complexgenome

But when I try with the branched channel it complains for the error I shared above.

Aha! We found where the error is coming from!

Let’s add some spacing to the channel element to see better how it’s structured.

[
  MM-0486-T-01,
  [
    batch:SEMA-MM-001, timepoint:MM-0486-T-01, tissue:tumor, sequencing_type:wes
  ],
  /mnt/data1/users/sanjeev/nextflow/meta_structure/work/0e/5efc4c5ad6c9776e3db00d2165ece2/MM-0486-T-01_T.sorted.markdup.recal.bam,
  /mnt/data1/users/sanjeev/nextflow/meta_structure/work/0e/5efc4c5ad6c9776e3db00d2165ece2/MM-0486-T-01_T.sorted.markdup.recal.bai
]

There are four items in this channel element. A value, a map, a path and another path. But your map call expects it to have three items (meta, bam and bai). Can you see the error now?

In your latest reply to this thread, you seem to have realized this (now your map expects 4 items: tp, meta, bam and bai).

@complexgenome

I’m unable to move forward.

Well, forward seems outside the scope of this post, right? You asked for joining with a branched channel. We’ve done it!

Please, make sure to select an answer and open a new post if you have a different question. Also, please, try to bring minimal reproducible examples for your questions, instead of real code that rely on files, multiple processes and so on. It’s much easier for us to help you with your issues this way

[complexgenome (S)]

@mribeirodantas
I used following code. It seems to work

consensus.out.regions_txt.map{meta,regions_file->[ "${meta.timepoint}", meta,regions_file ]}

.join(ch_branched.tumor.map{tp,meta,bam,bai ->["${meta.timepoint}",meta,bam,bai]})

.multiMap{pid, meta1, regions_file, meta2, bam,bai ->

region_txt:[meta1, regions_file]

tumor_recal:[meta2, bam,bai]

}.set { ch_input_tumor_consensus }

ch_input_tumor_consensus.region_txt.view()

ch_input_tumor_consensus.tumor_recal.view()

How do I make a dummy channel for the following data?

[
MM-0486-T-01,
[
batch:SEMA-MM-001, timepoint:MM-0486-T-01, tissue:tumor, sequencing_type:wes
],
/mnt/data1/users/sanjeev/nextflow/meta_structure/work/0e/5efc4c5ad6c9776e3db00d2165ece2/MM-0486-T-01_T.sorted.markdup.recal.bam,
/mnt/data1/users/sanjeev/nextflow/meta_structure/work/0e/5efc4c5ad6c9776e3db00d2165ece2/MM-0486-T-01_T.sorted.markdup.recal.bai
]

I see Channel.of(21..30) but the above example seems too complicated for that.

[mribeirodantas (Marcel Ribeiro-Dantas)]

Channel
  .of(['MM-0486-T-01',
      [batch:'SEMA-MM-001', timepoint:'MM-0486-T-01', tissue:'tumor', sequencing_type:'wes'],
      file('/mnt/data1/users/sanjeev/nextflow/meta_structure/work/0e/5efc4c5ad6c9776e3db00d2165ece2/MM-0486-T-01_T.sorted.markdup.recal.bam'),
      file('/mnt/data1/users/sanjeev/nextflow/meta_structure/work/0e/5efc4c5ad6c9776e3db00d2165ece2/MM-0486-T-01_T.sorted.markdup.recal.bai')
  ])