Are there any tools for the development, debugging, unit testing, and data validation in Nextflow?
I found execution report and trace report.
I’m wondering if there are anymore tools that facilitate the development cycle?
One thought I had was generating a Nextflow pipeline from JSON data. The scripting language does appear to support this. This would enable me to develop tools for my team.
There are some tools and it’s getting better all of the time.
Debugging: It’s still a bit experimental but you can enable this with a remote server and IntelliJ. Here is a demo of how to use it (note it’s now on stable release): https://www.youtube.com/watch?v=jtdlJQEOA1g&t=249s
Testing: nf-test is fantastic and can test at pipeline, workflow, process and function levels.
On of the best things about Nextflow is the portability, you can develop locally and migrate to an HPC or cloud provider by adding some configuration. I generally use a profile for local development to restrict compute resources before running the ‘full sized’ version elsewhere, the nf-core project uses this a lot so take a look at some of their examples.
Bigger picture, there are ongoing discussions on language improvements and a lot of these are focused on ease of development, stricter typing etc. Take a look at the big discussion on Github: Nextflow language improvements · nextflow-io/nextflow · Discussion #3107 · GitHub
One thought I had was generating a Nextflow pipeline from JSON data.
I’m not sure what this means?
1 Like
I’m going to check those resources out!
One thought I had was generating a Nextflow pipeline from JSON data.
I’m not sure what this means?
I created a python script that generates a pipeline from JSON instead of hard coding the pipeline itself. This made the pipeline more modular and could start and stop at any step and skip any steps.
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.