# My jobs are being canceled and logs are unable to be recovered
**URL:** https://community.seqera.io/t/my-jobs-are-being-canceled-and-logs-are-unable-to-be-recovered/538
**Category:** Ask for help
**Created:** [March 1, 2024, 10:30pm UTC](https://community.seqera.io/t/my-jobs-are-being-canceled-and-logs-are-unable-to-be-recovered/538 "2024-03-01T22:30:26Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@Che](https://community.seqera.io/u/Che)
#### Post date: [March 1, 2024, 10:30pm UTC](https://community.seqera.io/t/my-jobs-are-being-canceled-and-logs-are-unable-to-be-recovered/538/1 "2024-03-01T22:30:26Z")
Hello, I have a SLURM workflow that fails randomly fails at pretty much any step.
The terminal reads with this error:
`terminated for an unknown reason -- Likely it has been terminated by the external system`
When i check the working directory the only files i found were `.command.run and .command.sh`
The nextflow logs I see errors in recovering any of the metadata about the run
```auto
Process ... terminated for an unknown reason -- Likely it has been terminated by the external system
[Task monitor] DEBUG nextflow.processor.TaskRun - Unable to dump output of process ... Cause: java.nio.file.NoSuchFileException:
[Task monitor] DEBUG nextflow.processor.TaskRun - Unable to dump error of process ... Cause: java.nio.file.NoSuchFileException:
```
Does anyone know what’s happening here, a random step in the workflow seems to fail one time and work the next? I am having a really hard time debugging with minimal explanation as to why. I can get the workflow to complete by restarting the run again and again through brute force but that defeats the purpose of nextflow workflows
---
### Author:  [@Adam\_Talbot](https://community.seqera.io/u/Adam_Talbot)
#### Post date: [March 5, 2024, 9:51am UTC](https://community.seqera.io/t/my-jobs-are-being-canceled-and-logs-are-unable-to-be-recovered/538/2 "2024-03-05T09:51:53Z")
What is happening is Nextflow is running the pipeline, it submits the process but the computer that runs the process is killing it. This implies something is wrong with your infrastructure, normally it’s an issue with permissions or configuration.
Given it is working sometimes but not others, I expect there is an element of randomness that is different between instances of the process. For example, do you have multiple machines in your Slurm cluster? Perhaps the permissions are different between them.
To start debugging, can you go to the working directory and run `bash .command.run`? This will mimic Nextflow running the process and you may see a log in the terminal that tells you what is happening.
---
### Author:  [@Che](https://community.seqera.io/u/Che)
#### Post date: [March 8, 2024, 9:09pm UTC](https://community.seqera.io/t/my-jobs-are-being-canceled-and-logs-are-unable-to-be-recovered/538/3 "2024-03-08T21:09:17Z")
Thanks for your help!
I have been able to run the jobs manually by submitting the `.command.run` script but any time i have a job with a queuesize above ~40 or so I run into this issue.
After talking with the sys admins they cant seem to find anything that indicates why these jobs are being killed. And theres no explanation for why the jobs aren’t able to print error logs or any intermediate files either.
The error seems to happen to a seemingly random job each attempt, so this def seems to be a issue agnostic of the sample or step in the workflow
---
### Author:  [@Adam\_Talbot](https://community.seqera.io/u/Adam_Talbot)
#### Post date: [March 13, 2024, 9:55am UTC](https://community.seqera.io/t/my-jobs-are-being-canceled-and-logs-are-unable-to-be-recovered/538/4 "2024-03-13T09:55:14Z")
Do you have a cap on how many jobs you can submit at once to your machine? You can control this with the [`executor.queueSize` config option.](https://www.nextflow.io/docs/latest/config.html?executor.queueSize=#scope-executor).
---
### Author:  [@johnvusich](https://community.seqera.io/u/johnvusich)
#### Post date: [March 27, 2024, 8:33pm UTC](https://community.seqera.io/t/my-jobs-are-being-canceled-and-logs-are-unable-to-be-recovered/538/5 "2024-03-27T20:33:26Z")
I am running into a similar issue, although it does not seem to be a cap on number of jobs because my Nextflow jobs are killed even if there are 8 or less jobs running. This only happens to jobs submitted using Seqera Platform. I haven’t run into problems running pipelines on the command line. Our cluster uses Slurm as well. Seqera tower submits the jobs using my user profile via ssh and the input data and output directory are in my home directory, so it shouldn’t be a permissions issue. Could it be the way the sys admins configure the cluster/slurm for security purposes?
---
### Author:  [@slebedeva](https://community.seqera.io/u/slebedeva)
#### Post date: [July 29, 2025, 10:33am UTC](https://community.seqera.io/t/my-jobs-are-being-canceled-and-logs-are-unable-to-be-recovered/538/6 "2025-07-29T10:33:33Z")
Hello, \>1 year later I have encountered a very similar problem. Were there in the end any settings or permissions that you found out to be the cause?