Hi everyone, I’m currently migrating my pipeline from Snakemake to Nextflow. One feature I’m struggling to replicate is partial workflow execution. my goal is resuming a pipeline where some upstream tasks failed or were skipped, without re-triggering the entire graph. The Scenario (simplified, see…

Hi @COASD , welcome to the community :slight_smile: I think that what you’re looking for is -resume. See the docs: Caching and resuming — Nextflow documentation No need for any special workflow logic, just run the same nextflow run command again and tack on -r…

Thanks for the quick reply, I am already using -resume but it does not quite solve the usecase i am looking at. For instance if B represents a long calculation with 50% success rate and C a large post processing step that has a script for processing as an inputs. I then make a change in my postpro…

Yes, Nextflow cares about tasks and work directories - once files are published via publishDir then they are effectively out of the DAG and no longer tracked (you can trick it if you’re careful, but that way madness lies). So to recap: It sounds like Nextflow is doing what I’d expect it to do (reus…

I think i managed to find a 90% solution for my usecase. Instead of creating a channel from the results in the publishDir i filter the channel based on the files that are there. In my workflow i have added param.base which i then check in my workflow. If then i want to run only from B i filter bas…

Great! Glad you got it working, thanks for posting your solution :slight_smile:

OK I managed it. Like Phil says, we’re in anti-pattern territory here, but it’s possible. I reallllly dislike this solution because it relies on filenames The strategy: Run the pipeline with ignore to go to completion Write results from B() to results/B directory Continue to C()… On run 2, check …

Handling Partial Pipeline Execution - How to skip successful upstream tasks without breaking cache

Ask for help

mahesh.binzerpanchal (Mahesh Binzer-Panchal) February 4, 2026, 10:03am 8

I wrote this a while back: How to skip specific/failed samples on next `-resume`. I think this is what is wanted.

Topic		Replies	Views
Caching and resuming to skip one of the parallel jobs Ask for help nextflow	7	89	October 22, 2025
How to skip specific/failed samples on next `-resume` Tips & Tricks nextflow	2	198	April 16, 2025
Caching doesn't work always \|\| already processed data fails Ask for help	1	256	February 14, 2024
Enabling resume for specific processes Tips & Tricks nextflow	4	492	September 15, 2025
Does -resume re-run tasks if input files are deleted from the work directory? Ask for help nextflow	3	48	February 20, 2026

Handling Partial Pipeline Execution - How to skip successful upstream tasks without breaking cache

Related topics