Nf-core/pangenome aws-megatests fail - maybe because of fusion or wave?

Ask for help
, ,
1

Hi SEQERA community,

surprisingly the aws-megatests for the nf-core/pangenome pipeline fail: Seqera Cloud. This was already extensively discussed at Slack but it is unclear, why. Especially, because I can’t reproduce the error on my laptop or on our SLURM cluster.

Unique to these megatests are that they use Tower, docker (wave?), and the fusion system, if I understand correctly. The pipeline always fails at the moment it wants to sort a memory mapped data structure on disk.

Thanks for any help!

2

I’ll try to reproduce it on my side. Is setting the profile “test_full” and defining an “outdir” enough, or do I need some other configuration settings?

3

Hi @jordeu,

this would be awesome. You would also need to set two more parameters:

--n_haplotypes 8 --communities

The full profile is at pangenome/conf/test_full.config at 9740ffc4d0a00d6dc89e1c8f9579e628598d43a5 · nf-core/pangenome · GitHub.

4

I’ve been able to reproduce it. It is a Fusion bug. I’ve opened an internal ticket, and we are working on it.

Meanwhile, a non-ideal workaround is to set scratch = true only for that process.

process {
  withName:SMOOTHXG {
      scratch = true
  }
}
2 Likes
5

got it, I will forward this to the nf-core community
please ping me here, once this is fixed
thanks!

1 Like