# Handling SLURM Node Evictions in Nextflow: A Simple Retry Strategy

**URL:** <https://community.seqera.io/t/handling-slurm-node-evictions-in-nextflow-a-simple-retry-strategy/2575>\
**Category:** Tips & Tricks\
**Tags:** nextflow, hpc, config, slurm\
**Created:** [February 18, 2026, 8:29am UTC](https://community.seqera.io/t/handling-slurm-node-evictions-in-nextflow-a-simple-retry-strategy/2575 "2026-02-18T08:29:00Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![COASD](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/coasd/32/2259_2.png) [@COASD](https://community.seqera.io/u/COASD)\
**Post date:** [February 18, 2026, 8:29am UTC](https://community.seqera.io/t/handling-slurm-node-evictions-in-nextflow-a-simple-retry-strategy/2575/1 "2026-02-18T08:29:00Z")

</div>

if, like me, you’re running Nextflow with SLURM on a cluster that experiences node evictions, you probably want a way to automatically rerun jobs when they fail due to eviction.

According to the documentation on [Spot Instance failures and retries](https://www.nextflow.io/docs/latest/guides/updating-spot-retries.html), this feature is implemented when using AWS Batch or Google Batch. However, I found a relatively simple solution to achieve similar behaviour on a SLURM‑based cluster.

When a job fails due to an eviction, SLURM reports a `NODE_FAIL`, and Nextflow treats this as the node being “terminated for an unknown reason.” We can use this to trigger automatic retries by adding the following to your config:

```groovy
errorStrategy = {
    if (task.previousException?.toString().contains("terminated for an unknown reason") && task.attempt <= 3) {
        return 'retry'
    }else{
        return 'ignore'
    }
}

```

This was my relatively simple “tick” that i found 🙂

---

<div class="post-metadata">

**Author:** ![steve](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/steve/32/125_2.png) [@steve](https://community.seqera.io/u/steve)\
**Post date:** [May 1, 2026, 11:35pm UTC](https://community.seqera.io/t/handling-slurm-node-evictions-in-nextflow-a-simple-retry-strategy/2575/2 "2026-05-01T23:35:11Z")

</div>

> [@COASD](#):
>
> `task.previousException`

wow never heard of this one before, good catch