Hi
Thank you for setting up the Seqera platform.
I have a question which is about nf-core/nextflow behavior, and how Seqera fusion deals with it.
nf-core seems to have one task per command, so we have a samtools sort, samtools index, samtools stats task.
Each of these tasks needs the BAM file as input.
Does this mean that nextflow spins up multiple instance, and each instance downloads the 5GB BAM file from S3 each time? Looking at the timeline, it seems so, but that seems inefficient.
The nextflow docs (Working with files — Nextflow documentation) seem to indicate that nextflow can download the file once, and reusue it, but if each process is running in a different AWS EC2 instance, doesn’t that force re-downloading?
When I wrote pipelines in WDL, I usually had one task for samtools, that would run all the commands, and therefore download the BAM file once. Does the Fusion filesystem make the downloading only once?
Am I missing something here?
Thank you,
Uri David