Create channel after collect from channel that has multiple files?

[complexgenome (S)]

Hello there,

I’ve following channel:

ch_msi=channel.of( [[batch:"SEMA-MM-003", timepoint:"MM-0256-T-03", tissue:"normal", sequencing_type:"wes"], [file("MM-0256-T-03_msi_output"), 
  file("MM-0256-T-03_msi_output_dis"),file("MM-0256-T-03_msi_output_germline"), file("MM-0256-T-03_msi_output_somatic")]],

[[batch:"SEMA-MM-002", timepoint:"MM-3530-T-01", tissue:"normal", sequencing_type:"wes"], [file("MM-3530-T-01_msi_output"), 
file("MM-3530-T-01_msi_output_dis"), file("MM-3530-T-01_msi_output_germline"), file("MM-3530-T-01_msi_output_somatic")]],

[[batch:"SEMA-MM-004", timepoint:"MM-0245-T-01", tissue:"normal", sequencing_type:"wes"], [file("MM-0245-T-01_msi_output"), 
file("MM-0245-T-01_msi_output_dis"), file("MM-0245-T-01_msi_output_germline"), file("MM-0245-T-01_msi_output_somatic")]]
)

I’d like to have a channel that has multiple files (specific from the above, say first file) after collect, and other data manipulation.

I’d like output as:

["path/MM-0256-T-03_msi_output", "path/MM-3530-T-01_msi_output","path/MM-0245-T-01_msi_output"]

That is only first file.

I tried the following but doesn’t seem to work

ch_msi | collect(flat: false) | transpose() | buffer(skip: 1)| map { it[0] } | view

How do I resolve or get desired result?

[mribeirodantas (Marcel Ribeiro-Dantas)]

Use this sequence of channel operators instead.

...
transpose_ch
  .buffer(skip: 1)
  .map { it[0] }
  .flatMap()
  .map { it[0] }
  .set { my_ch }
...

Output

[complexgenome (S)]

@mribeirodantas
Thank you for your response.

Sorry, the code shared doesn’t produce intended output.

ch_msi=channel.of( [[batch:"SEMA-MM-003", timepoint:"MM-0256-T-03", tissue:"normal", sequencing_type:"wes"], [file("MM-0256-T-03_msi_output"), 
  file("MM-0256-T-03_msi_output_dis"),file("MM-0256-T-03_msi_output_germline"), file("MM-0256-T-03_msi_output_somatic")]],

[[batch:"SEMA-MM-002", timepoint:"MM-3530-T-01", tissue:"normal", sequencing_type:"wes"], [file("MM-3530-T-01_msi_output"), 
file("MM-3530-T-01_msi_output_dis"), file("MM-3530-T-01_msi_output_germline"), file("MM-3530-T-01_msi_output_somatic")]],

[[batch:"SEMA-MM-004", timepoint:"MM-0245-T-01", tissue:"normal", sequencing_type:"wes"], [file("MM-0245-T-01_msi_output"), 
file("MM-0245-T-01_msi_output_dis"), file("MM-0245-T-01_msi_output_germline"), file("MM-0245-T-01_msi_output_somatic")]]
)

ch_msi.buffer(skip: 1)
  .map { it[0] }
  .flatMap().map { it[0] }  .set { my_ch_msi }
  my_ch_msi.view()

I get attached output.

Rather I’d like to see:

MM-3530-T-01_msi_output
MM-0245-T-01_msi_output
MM-0256-T-03_msi_output

Thanks

[mribeirodantas (Marcel Ribeiro-Dantas)]

@complexgenome

Sorry, the code shared doesn’t produce intended output.

It does work. But if you changed your original code removing collect(flat: false) | transpose(), it will not

Channel
  .of([
        [batch:"SEMA-MM-003", timepoint:"MM-0256-T-03", tissue:"normal", sequencing_type:"wes"],
        [file("MM-0256-T-03_msi_output"), file("MM-0256-T-03_msi_output_dis"),file("MM-0256-T-03_msi_output_germline"), file("MM-0256-T-03_msi_output_somatic")]
      ],
      [
        [batch:"SEMA-MM-002", timepoint:"MM-3530-T-01", tissue:"normal", sequencing_type:"wes"],
        [file("MM-3530-T-01_msi_output"), file("MM-3530-T-01_msi_output_dis"), file("MM-3530-T-01_msi_output_germline"), file("MM-3530-T-01_msi_output_somatic")]
      ],
      [
        [batch:"SEMA-MM-004", timepoint:"MM-0245-T-01", tissue:"normal", sequencing_type:"wes"],
        [file("MM-0245-T-01_msi_output"), file("MM-0245-T-01_msi_output_dis"), file("MM-0245-T-01_msi_output_germline"), file("MM-0245-T-01_msi_output_somatic")]
      ]
  )
  .set { ch_msi }

ch_msi| collect(flat: false)  | transpose() | set { ch_msi }

ch_msi
  .buffer(skip: 1)
  .map { it[0] }
  .flatMap()
  .map { it[0] }
  .set { my_ch_msi }
my_ch_msi.view()

[complexgenome (S)]

Thank you.

[complexgenome (S)]

@mribeirodantas

Sorry, this makes process runs 3 times.

How do I send these 3 files in one channel and thus running the process only once?

[
"MM-3530-T-01_msi_output",
"MM-0245-T-01_msi_output",
"MM-0256-T-03_msi_output"
]

Sorry for the confusion.
I hope my desired output is clearer?

[mribeirodantas (Marcel Ribeiro-Dantas)]

We’ve solved this type of problem a few times already, haven’t we?

Collect

[complexgenome (S)]

@mribeirodantas
Thank you.

    ch_msi 
    .buffer(skip: 1)
    .map { it[0] }
    .flatMap()
    .map { it[0] }
    .collect(flat: false) 
    .set { ch_msi_collected }