Nextflow Podcast Ep. 57: Pipeline chaining, meta pipelines and automation (Part 2 of 2)

Events
,
1

Hey everyone! :wave:

New episode alert! :nextflow: :studio_microphone:

Episode 57 is Part 2 of our pipeline chaining / meta pipelines discussion with @bentsherman and @emiller :chains:

Last episode we covered all the existing workarounds. This time we get into what’s changing in Nextflow itself to make pipeline composition easier.

Workflow Inputs:

  • New params typing in Nextflow 25.10: declare types directly in your script

  • Record types coming in 26.04: streamline sample sheet parsing so Nextflow handles CSV/JSON/YAML automatically

  • Params at the top level only: cleaner, truly modular sub-workflows

Workflow Outputs:

  • The new publish section replacing publishDir: finally a holistic view of what your pipeline actually produces

  • JSON output manifests with full metadata: “Nextflow pipeline as an API call”

  • No more guessing file paths or scanning directory trees :raised_hands:

Looking Ahead:

  • Piping pipelines on the command line: nextflow run A | nextflow run B (Ben has a working prototype!)

  • Lineage IDs in sample sheets: data-centric rather than pipeline-centric workflows

  • Schema-based validation for pipeline chaining in Seqera Platform

The dream: JSON params in, JSON manifest out, with all metadata preserved. Pipelines that can interface with each other using shared naming conventions and minimal glue code.

:books: Key resources:

:writing_hand: Show notes, transcript, links and more: Pipeline chaining, meta pipelines and automation (Part 2) | The Nextflow Podcast by Seqera

:television: The episode is available now on YouTube, Spotify, and all other podcast platforms! :headphones:

:input_latin_lowercase: As always, the podcast page has a detailed summary plus full transcript. Subtitles are manually curated!

What do you think? Are you excited about workflow outputs and record types? Already doing pipeline chaining? Let us know below!