# Zombie runs, jobs has been killed in local but state is still running on Seqera **URL:** \ **Category:** Ask for help\ **Tags:** nextflow, tower, platform\ **Created:** [July 4, 2024, 2:58am UTC](https://community.seqera.io/t/zombie-runs-jobs-has-been-killed-in-local-but-state-is-still-running-on-seqera/921 "2024-07-04T02:58:30Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Phi\_Nguyen\_Truong](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/phi_nguyen_truong/32/826_2.png) [@Phi\_Nguyen\_Truong](https://community.seqera.io/u/Phi_Nguyen_Truong)\ **Post date:** [July 4, 2024, 2:58am UTC](https://community.seqera.io/t/zombie-runs-jobs-has-been-killed-in-local-but-state-is-still-running-on-seqera/921/1 "2024-07-04T02:58:30Z")
Hi, Seqera community. When running a [pipeline](https://github.com/PGScatalog/pgsc_calc) with CLI, runs are failed/canceled and I already checked the state with slurm. But the state is still running on Seqera. The success run’s state is success normally. ![image](https://europe1.discourse-cdn.com/flex013/uploads/seqera/original/1X/c417960b028e539859cd4de12565e0a0f2b91961.png) As I see, this phenomenon especially happens with this pipeline. **What are the possible causes in a pipeline that leads to the “zombie runs” problem in general? Also, how do I cancel a “zombie” job on Seqera?** Review developing history, I see: - [Zombie runs, redux: Jobs still shown as running · Issue #354 · seqeralabs/nf-tower · GitHub](https://github.com/seqeralabs/nf-tower/issues/354) - [Zombie run · Issue #313 · seqeralabs/nf-tower · GitHub](https://github.com/seqeralabs/nf-tower/issues/313) They also met the same problem. It seems they solved this by providing `workflow_id` and the Tower administrator has canceled for them. Is this way possible now? Thanks for the great platform. ---
**Author:** ![bpoli](https://avatars.discourse-cdn.com/v4/letter/b/22d042/32.png) [@bpoli](https://community.seqera.io/u/bpoli)\ **Post date:** [November 13, 2024, 1:50pm UTC](https://community.seqera.io/t/zombie-runs-jobs-has-been-killed-in-local-but-state-is-still-running-on-seqera/921/2 "2024-11-13T13:50:02Z")
I always get these “zombie runs” when I kill a nextflow run that I have submitted to the cluster with slurm using “scancel”. The PID gets killed as expected and the run interrupts, but the pipeline is shown as running in seqera cloud forever. ---
**Author:** ![alanwilter](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/alanwilter/32/1718_2.png) [@alanwilter](https://community.seqera.io/u/alanwilter)\ **Post date:** [March 28, 2025, 8:54am UTC](https://community.seqera.io/t/zombie-runs-jobs-has-been-killed-in-local-but-state-is-still-running-on-seqera/921/3 "2025-03-28T08:54:31Z")
I have the same problem, how to remove these zombies? ![Screenshot 2025-03-24 at 09.50.09](https://europe1.discourse-cdn.com/flex013/uploads/seqera/original/2X/4/4275f1e7ec8fd76cfefbde5cc94e452d379f20a7.png) ---
**Author:** ![robsyme](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/robsyme/32/342_2.png) [@robsyme](https://community.seqera.io/u/robsyme)\ **Post date:** [March 28, 2025, 4:15pm UTC](https://community.seqera.io/t/zombie-runs-jobs-has-been-killed-in-local-but-state-is-still-running-on-seqera/921/4 "2025-03-28T16:15:34Z")
If the Nextflow process is forcibly killed, it cannot report back to the Platform that the jobs have terminated. To force-kill the task from the Platform side, you can use the API directly and use the force=true parameter. You’ll need: The run id (which you can pull from the run url) The workspace ID (the numeric ID available on the org page for each workspace) A token (available to be minted in “User Tokens” in the top-right user menu. For example, to delete a single run: ```bash RUN_ID='2vEhA4EADamYMz' WORKSPACE_ID='64430228002560' curl -X POST "https://api.cloud.seqera.io/workflow/delete?workspaceId=$WORKSPACE_ID&force=true" \ -H "Authorization: Bearer $TOWER_ACCESS_TOKEN" \ -H "Content-Type: application/json" \ -d "{\"workflowIds\":\"$RUN_ID\"}" ``` To bulk delete runs, you can provide an array of run ids as the workflowIds value: ```bash curl -X POST "https://api.cloud.seqera.io/workflow/delete?workspaceId=$WORKSPACE_ID&force=true" \ -H "Authorization: Bearer $TOWER_ACCESS_TOKEN" \ -H "Content-Type: application/json" \ -d '{"workflowIds":["4GviYJvYgPzetw","1g40rRkH80oHAT"]}' ``` ---
**Author:** ![Phi\_Nguyen\_Truong](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/phi_nguyen_truong/32/826_2.png) [@Phi\_Nguyen\_Truong](https://community.seqera.io/u/Phi_Nguyen_Truong)\ **Post date:** [March 29, 2025, 2:44am UTC](https://community.seqera.io/t/zombie-runs-jobs-has-been-killed-in-local-but-state-is-still-running-on-seqera/921/5 "2025-03-29T02:44:54Z")
@robsyme I have tried your solution and got a response: ```auto {"failedWorkflowIds":[]} ``` However, the zombie run still hasn’t died yet. ---
**Author:** ![robsyme](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/robsyme/32/342_2.png) [@robsyme](https://community.seqera.io/u/robsyme)\ **Post date:** [March 30, 2025, 10:16pm UTC](https://community.seqera.io/t/zombie-runs-jobs-has-been-killed-in-local-but-state-is-still-running-on-seqera/921/6 "2025-03-30T22:16:17Z")
That message indicates that there were no failures in the cancellation. If you refresh the page is the run still present in the list of runs displayed by Platform? ---
**Author:** ![Phi\_Nguyen\_Truong](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/phi_nguyen_truong/32/826_2.png) [@Phi\_Nguyen\_Truong](https://community.seqera.io/u/Phi_Nguyen_Truong)\ **Post date:** [April 1, 2025, 7:00am UTC](https://community.seqera.io/t/zombie-runs-jobs-has-been-killed-in-local-but-state-is-still-running-on-seqera/921/7 "2025-04-01T07:00:29Z")
@robsyme Thanks you so much. When I check on Tower Runs history, the zombies have gone. However, the web link of zombies are still accessible, and I still see it running when access those links. There is any way to “force cancel” zombie jobs instead of delete them from the Runs History? ---
**Author:** ![system](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/system/32/2402_2.png) [@system](https://community.seqera.io/u/system)\ **Post date:** [April 8, 2025, 7:01am UTC](https://community.seqera.io/t/zombie-runs-jobs-has-been-killed-in-local-but-state-is-still-running-on-seqera/921/8 "2025-04-08T07:01:03Z")
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.