# \#hpc **URL:** https://community.seqera.io/tag/hpc/39.md [Latest](https://community.seqera.io/latest.md) · [Categories](https://community.seqera.io/categories.md) · [Tags](https://community.seqera.io/tags.md) --- ## [Handling SLURM Node Evictions in Nextflow: A Simple Retry Strategy](https://community.seqera.io/t/handling-slurm-node-evictions-in-nextflow-a-simple-retry-strategy/2575)
**Author:** [@COASD](https://community.seqera.io/u/COASD)\ **Replies:** 1\ **Last updated:** [May 1, 2026, 11:35pm UTC](https://community.seqera.io/t/handling-slurm-node-evictions-in-nextflow-a-simple-retry-strategy/2575 "2026-05-01T23:35:11Z")
if, like me, you’re running Nextflow with SLURM on a cluster that experiences node evictions, you probably want a way to automatically rerun jobs when they fail due to eviction. According to the documentation on Spot In… --- ## [Handling large numbers of processes in an HPC](https://community.seqera.io/t/handling-large-numbers-of-processes-in-an-hpc/2206)
**Author:** [@ramirobarrantes](https://community.seqera.io/u/ramirobarrantes)\ **Replies:** 1\ **Last updated:** [June 19, 2025, 7:05am UTC](https://community.seqera.io/t/handling-large-numbers-of-processes-in-an-hpc/2206 "2025-06-19T07:05:52Z")
I need to run Sarek on a large number of genomes (more than 10,000). Our HPC is set up with apptainer and works pretty well. I think I can go two ways: Run Sarek as I usually do, and let it handle all the particulariti… --- ## [Jobs marked as failed by nextflow but slurm exit code 0](https://community.seqera.io/t/jobs-marked-as-failed-by-nextflow-but-slurm-exit-code-0/2179)
**Author:** [@asherps](https://community.seqera.io/u/asherps)\ **Replies:** 0\ **Last updated:** [June 9, 2025, 5:13pm UTC](https://community.seqera.io/t/jobs-marked-as-failed-by-nextflow-but-slurm-exit-code-0/2179 "2025-06-09T17:13:53Z")
Hi – we’ve been encountering an error with some of our nextflow pipelines (both nf-core and ones we are developing) in which we are getting an error that seems to suggest that one of the jobs failed and was terminated by… --- ## [How to store and share nextflow/nfcore results files](https://community.seqera.io/t/how-to-store-and-share-nextflow-nfcore-results-files/2176)
**Author:** [@ramirobarrantes](https://community.seqera.io/u/ramirobarrantes)\ **Replies:** 1\ **Last updated:** [June 9, 2025, 10:55am UTC](https://community.seqera.io/t/how-to-store-and-share-nextflow-nfcore-results-files/2176 "2025-06-09T10:55:21Z")
I have a very general question. During analyses we create results files that go into the output directory, which very frequently take a long time to generate. These results files can subsequently be used for graphs and r… --- ## [Cleanup and publishing performance](https://community.seqera.io/t/cleanup-and-publishing-performance/2162)
**Author:** [@Poshi](https://community.seqera.io/u/Poshi)\ **Replies:** 4\ **Last updated:** [May 27, 2025, 1:48pm UTC](https://community.seqera.io/t/cleanup-and-publishing-performance/2162 "2025-05-27T13:48:40Z")
I’m interested in assessing the performance of the publishing and cleanup tasks. Is there a way to know when the Nextflow process have finished the publishing of all the required outputs? I assume that it starts as soon… --- ## [Nf-core/rnaseq v3.18.0 – Minimum‑55‑CPU rule, account suspensions & debug‑queue limits](https://community.seqera.io/t/nf-core-rnaseq-v3-18-0-minimum-55-cpu-rule-account-suspensions-debug-queue-limits/2158)
**Author:** [@Dokmen](https://community.seqera.io/u/Dokmen)\ **Replies:** 3\ **Last updated:** [May 23, 2025, 5:35pm UTC](https://community.seqera.io/t/nf-core-rnaseq-v3-18-0-minimum-55-cpu-rule-account-suspensions-debug-queue-limits/2158 "2025-05-23T17:35:07Z")
Hello everyone, I am struggling to run nf‑core/rnaseq v3.18.0 (Nextflow v25.04.2) on the HPC system—specifically the ORF queue, which enforces 55 ≤ CPU ≤ 110 for every Slurm job. Light steps (FastQC, gunzip, Trim Galore… --- ## [Nextflow CPU limit does not take effect](https://community.seqera.io/t/nextflow-cpu-limit-does-not-take-effect/2011)
**Author:** [@Jason\_Ren](https://community.seqera.io/u/Jason_Ren)\ **Replies:** 0\ **Last updated:** [March 25, 2025, 1:58pm UTC](https://community.seqera.io/t/nextflow-cpu-limit-does-not-take-effect/2011 "2025-03-25T13:58:00Z")
Hi, I am running my program on Nextflow version 24.10.5. The content of my nextflow.config is as follows: params { // Workflow flags // TODO nf-core: Specify your pipeline's command line flags genome = "GR… --- ## [Nextflow 16sHiSeq workflow in a SGE cluster](https://community.seqera.io/t/nextflow-16shiseq-workflow-in-a-sge-cluster/1167)
**Author:** [@Johanndb](https://community.seqera.io/u/Johanndb)\ **Replies:** 4\ **Last updated:** [October 4, 2024, 1:38pm UTC](https://community.seqera.io/t/nextflow-16shiseq-workflow-in-a-sge-cluster/1167 "2024-10-04T13:38:57Z")
Hi All I am having a bit of trouble analyzing my PacBio Kinnex full length 16s rRNA data. The data I received from the sequencing center were already demultiplexed with primers removed. I tried pivoting to the BacBio Hi… --- ## [GPU selection with HPC scheduler and singularity containers](https://community.seqera.io/t/gpu-selection-with-hpc-scheduler-and-singularity-containers/1146)
**Author:** [@seanrjohnson](https://community.seqera.io/u/seanrjohnson)\ **Replies:** 1\ **Last updated:** [August 29, 2024, 5:26pm UTC](https://community.seqera.io/t/gpu-selection-with-hpc-scheduler-and-singularity-containers/1146 "2024-08-29T17:26:24Z")
I’m running Nextflow on an HPC system where each node has multiple GPUs and the job scheduler sets the environment variable $SGE\_CUDA\_sgr, based on which GPUs are assigned to the job. My Nextflow processes run from Appt… --- ## [Sbatch fatal error when mem-per-cpu is included and not when not included](https://community.seqera.io/t/sbatch-fatal-error-when-mem-per-cpu-is-included-and-not-when-not-included/1107)
**Author:** [@vmkalbskopf](https://community.seqera.io/u/vmkalbskopf)\ **Replies:** 1\ **Last updated:** [August 19, 2024, 9:20am UTC](https://community.seqera.io/t/sbatch-fatal-error-when-mem-per-cpu-is-included-and-not-when-not-included/1107 "2024-08-19T09:20:35Z")
I am trying to run the demo mode of this pipeline from Nanopore using the apptainer profile on an HPC that uses the slurm scheduler. If specify the --mem-per-cpu in the clusterOptions, I get the sbatch error: --mem --m… --- ## [Error on selecting a specific output from a process that outputs multiple files, and pass it to the next process?](https://community.seqera.io/t/error-on-selecting-a-specific-output-from-a-process-that-outputs-multiple-files-and-pass-it-to-the-next-process/1071)
**Author:** [@saif\_s](https://community.seqera.io/u/saif_s)\ **Replies:** 5\ **Last updated:** [August 11, 2024, 7:11pm UTC](https://community.seqera.io/t/error-on-selecting-a-specific-output-from-a-process-that-outputs-multiple-files-and-pass-it-to-the-next-process/1071 "2024-08-11T19:11:01Z")
I keep getting this error when I use my code below: Error: Not a valid path value type: groovyx.gpars.dataflow.DataflowBroadcast (DataflowBroadcast around DataflowStream\[?\]) Code: process QIME2DENOISE{ debug true l… --- ## [Singularity qiime2 container - qiime command not found](https://community.seqera.io/t/singularity-qiime2-container-qiime-command-not-found/1044)
**Author:** [@saif\_s](https://community.seqera.io/u/saif_s)\ **Replies:** 8\ **Last updated:** [August 3, 2024, 12:23am UTC](https://community.seqera.io/t/singularity-qiime2-container-qiime-command-not-found/1044 "2024-08-03T00:23:47Z")
Hello, I have tried testing multiple qiime2-core and qiime2-amplicon images using singularity. Everytime I try to test my nextflow script it ends up not recognizing qiime command (not finding qiime in my bash script). A… --- ## [Create jobs on Seqera Cloud and run on local server](https://community.seqera.io/t/create-jobs-on-seqera-cloud-and-run-on-local-server/1028)
**Author:** [@ckywong](https://community.seqera.io/u/ckywong)\ **Replies:** 4\ **Last updated:** [July 30, 2024, 4:32pm UTC](https://community.seqera.io/t/create-jobs-on-seqera-cloud-and-run-on-local-server/1028 "2024-07-30T16:32:11Z")
Hello, We currently have access to Seqera Cloud. And we are hoping to create jobs there and have the jobs running on our local servers, which are inside an institutional firewall blocking access from outside. We heard… --- ## [My first nextflow script, I'm getting alot of errors with every process - Is my approach wrong?](https://community.seqera.io/t/my-first-nextflow-script-im-getting-alot-of-errors-with-every-process-is-my-approach-wrong/988)
**Author:** [@saif\_s](https://community.seqera.io/u/saif_s)\ **Replies:** 1\ **Last updated:** [July 18, 2024, 8:03pm UTC](https://community.seqera.io/t/my-first-nextflow-script-im-getting-alot-of-errors-with-every-process-is-my-approach-wrong/988 "2024-07-18T20:03:10Z")
Hello, I’m trying to write a nextflow script that automates a workflow that will run on a high computing cloud and local computers. The script requires only Bash and Python to be used. What I’m trying to do is: Load … --- ## [Question about Nextflow and SLURM](https://community.seqera.io/t/question-about-nextflow-and-slurm/799)
**Author:** [@mpampuch](https://community.seqera.io/u/mpampuch)\ **Replies:** 1\ **Last updated:** [May 30, 2024, 11:52pm UTC](https://community.seqera.io/t/question-about-nextflow-and-slurm/799 "2024-05-30T23:52:36Z")
I’m new to Nextflow and HPC computing so I have a question about Nextflow and SLURM. Can SLURM syntax be slightly different between HPCs? If so, then if running Nextflow or an nf-core pipeline using SLURM, how does Nextf… --- ## [Platform launch folders and concurrent Nextflow runs](https://community.seqera.io/t/platform-launch-folders-and-concurrent-nextflow-runs/531)
**Author:** [@Federico](https://community.seqera.io/u/Federico)\ **Replies:** 1\ **Last updated:** [April 23, 2024, 9:43am UTC](https://community.seqera.io/t/platform-launch-folders-and-concurrent-nextflow-runs/531 "2024-04-23T09:43:23Z")
Hello, I’m trying to use the Seqera Platform to submit multiple instances of the same pipeline, with different inputs, over a SLURM cluster. So far, I’ve been able to successfully run the individual instances, but if I … --- ## [Slurm killing platform jobs](https://community.seqera.io/t/slurm-killing-platform-jobs/607)
**Author:** [@johnvusich](https://community.seqera.io/u/johnvusich)\ **Replies:** 3\ **Last updated:** [April 8, 2024, 9:31pm UTC](https://community.seqera.io/t/slurm-killing-platform-jobs/607 "2024-04-08T21:31:55Z")
I set up a compute environment to run jobs using our HPC. Our cluster uses Slurm. When I submit the head job, I can see that it is running (using squeue -u) and that eventually the cluster jobs start to run. However, aft… --- ## [Little discrepancies between results from normal and its optimized workflow](https://community.seqera.io/t/little-discrepancies-between-results-from-normal-and-its-optimized-workflow/444)
**Author:** [@animesh](https://community.seqera.io/u/animesh)\ **Replies:** 1\ **Last updated:** [January 30, 2024, 11:34am UTC](https://community.seqera.io/t/little-discrepancies-between-results-from-normal-and-its-optimized-workflow/444 "2024-01-30T11:34:17Z")
First i must say that it is truly amazing how https://tower.nf/ is able to orchestrate the slurm jobs :pray: It just took away a lot of pain from work :ok\_hand:and kind of magical to see that it could optimize it even mo… --- ## [Slurm node overhead](https://community.seqera.io/t/slurm-node-overhead/437)
**Author:** [@emiller](https://community.seqera.io/u/emiller)\ **Replies:** 1\ **Last updated:** [January 29, 2024, 5:00pm UTC](https://community.seqera.io/t/slurm-node-overhead/437 "2024-01-29T17:00:08Z")
Wondering if anyone has experience actually admining slurm clusters and could shead a little light on something that I’ve never found a definate answer on. When assigning processes to specific nodes and queues and reque… --- ## [Save as pipeline requirements](https://community.seqera.io/t/save-as-pipeline-requirements/375)
**Author:** [@arnaudceol](https://community.seqera.io/u/arnaudceol)\ **Replies:** 6\ **Last updated:** [January 23, 2024, 12:25pm UTC](https://community.seqera.io/t/save-as-pipeline-requirements/375 "2024-01-23T12:25:47Z")
Hi, I’m trying to save my jobs as pipeline without success. The jobs are simple nextflow pipelines launched fron our HPC. When I press “save as pipeline”, nothing happens. My guess is that I need some additional configur… --- ## [Slurm tower connection issue](https://community.seqera.io/t/slurm-tower-connection-issue/411)
**Author:** [@animesh](https://community.seqera.io/u/animesh)\ **Replies:** 3\ **Last updated:** [January 18, 2024, 10:54pm UTC](https://community.seqera.io/t/slurm-tower-connection-issue/411 "2024-01-18T22:54:14Z")
I tried submitting job to out HPC slurm via the tower but looks like account is not being relayed, error message below message: sbatch: error: Account specification required, but not provided sbatch: error: Batch job su… --- ## [Tearing down a orchestration before completion of the workflow](https://community.seqera.io/t/tearing-down-a-orchestration-before-completion-of-the-workflow/343)
**Author:** [@pelahi](https://community.seqera.io/u/pelahi)\ **Replies:** 3\ **Last updated:** [January 6, 2024, 2:34pm UTC](https://community.seqera.io/t/tearing-down-a-orchestration-before-completion-of-the-workflow/343 "2024-01-06T14:34:54Z")
Context It is not possible to run an orchestration indefinitely on HPC systems. Typical wall time limits means there are often limits of 24 hours and up to a few days. This might not be enough to complete an entire workf… --- ## [Handling lots of IO (large files or lots of small files) on HPC systems](https://community.seqera.io/t/handling-lots-of-io-large-files-or-lots-of-small-files-on-hpc-systems/342)
**Author:** [@pelahi](https://community.seqera.io/u/pelahi)\ **Replies:** 1\ **Last updated:** [December 18, 2023, 10:25am UTC](https://community.seqera.io/t/handling-lots-of-io-large-files-or-lots-of-small-files-on-hpc-systems/342 "2023-12-18T10:25:02Z")
Context Nextflow uses files to stitch a workflow together. However, the creation of large number of small files and directories is ill suited to the parallel filesystems used on HPC systems, which are shared resources wi… --- ## [Deploying tower -agent on HPC without root access](https://community.seqera.io/t/deploying-tower-agent-on-hpc-without-root-access/308)
**Author:** [@FloWuenne](https://community.seqera.io/u/FloWuenne)\ **Replies:** 6\ **Last updated:** [December 5, 2023, 2:00pm UTC](https://community.seqera.io/t/deploying-tower-agent-on-hpc-without-root-access/308 "2023-12-05T14:00:19Z")
Hi everyone, I recently set up our HPC environment on Seqera Platform. We are using a cluster with a SLURM scheduler provided by our University. Since we don’t have root access to the cluster, I was wondering how other … --- ## [Resume run with changed head job configuration](https://community.seqera.io/t/resume-run-with-changed-head-job-configuration/220)
**Author:** [@Johannes\_Alneberg](https://community.seqera.io/u/Johannes_Alneberg)\ **Replies:** 6\ **Last updated:** [October 26, 2023, 1:13pm UTC](https://community.seqera.io/t/resume-run-with-changed-head-job-configuration/220 "2023-10-26T13:13:37Z")
Hello! Background: Running on the enterprise version (23.2.0\_d157b0d) with SLURM on an offline HPC, using the GUI to launch the job. Issue: My head job unfortunately ran out of memory at a very late stage of the pipeli…