Hi Community,
I am working on a Nextflow workflow that involves processing a series of paired input files (e.g., [[bamfile1, bamfile2], [samfile1, samfile2]]). My workflow is running multiple processes, and as a final step, I want to cleanup the input files from the input path, but only after the entire workflow has successfully completed.
Here is my current implementation of the cleanup process:
groovy:
process cleanUpProcess {
cpus 2
memory "4 GB"
container params.cli
input:
val fileList // [[bamfile1,bamfile2],[samfile1,samfile2]]
output:
stdout
script:
def awsS3Commands = fileList.collect { files ->
def pairOne = files[0]
def pairTwo = files[1]
"""
aws s3 rm ${pairOne}
aws s3 rm ${pairTwo}
"""
}.join('\n')
"""
${awsS3Commands}
"""
}
workflow MethylDackel {
main:
// Multiple processes are being executed here
}
workflow {
def myParams = params
def myWorkflow = workflow
MethylDackel()
myWorkflow.onComplete {
if (myWorkflow.success) {
cleanUpProcess()
} else {
log.info "Failure!"
}
}
}
Challenge:
I understand that the onComplete block cannot directly invoke the cleanUpProcess. However, I want the cleanup process to only trigger after the workflow has successfully completed. This ensures that the input files are not removed prematurely or during an unsuccessful execution.
Question to the Community
Are there any workarounds or alternative approaches to trigger the cleanup process after successful workflow completion?
I would love to hear your suggestions or ideas on how to handle this use case efficiently while adhering to best practices. Thank you in advance for your guidance! 
@mribeirodantas @mahesh.binzerpanchal
