How to deal with optional inputs in a workflow

Ask for help
1

Greetings all - I’m a bit perplexed as to have a bit of functionality working. I will provide a simplified example of my dilemma. Say I have a workflow that may have optional output:

process PRODUCER {
    input:
        val(meta)

    output:
        path('file1.txt'), emit: out1
        path('file2.txt'), optional: true, emit: out2



    script:
    """
    echo "foo" > file1.txt
    # Optionally create file2.txt
    # echo "foo" > file2.txt
    """
}

Now I want to have behavior in the next process tied to the outputs in the PRODUCER process:

process CONSUMER {
    input:
        path f1
        path f2

    debug true

    script:
    """
    cat ${f1}
    if [[ -f "${f2}" ]]; then
        cat ${f2}
    fi
    """
}

This is harder than it seems on the surface and I haven’t been able to find a solution, because the optional output of producer process can’t be optional in the input. I saw a bit of work revolving around having ‘default values’ but I’m unsure of where this ended up based on the git issues. Another approach I can think of is based on manipulating the outputs in a channel? Can someone help or point me in the right direction here? Thank you in advance and apologies if this is a question already posed - was unable to find duplicates.

2

This is not so much a problem of optional inputs, but how to keep two separate input channels synchronized with each other. You might be tempted to call CONSUMER like this:

CONSUMER(PRODUCER.out.out1, PRODUCER.out.out2)

But this is generally very dangerous, as you shouldn’t assume that the elements in channels are in a particular order. What you need to do is join the two output channels based on an unique key and using reminder:true to convert optional output into []. Now you can multiMap the joined channel into two synchronized channels for each input of CONSUMER.

3

I think the big picture idea is following

Leverage nextflow workflow design to conditional handle the 1 or 2 input for CONSUMER, rather than use script inside CONSUMER to conditional handle 1 or 2 inputs.

process CONSUMER {
    input:
        path f1
        path f2, optional: true

    debug true

    script:
    """
    script:
    if (f2) {
        """
        echo "Using f2: $genome_file"
        """
    } else {
        """
        echo "No f2 file provided, skipping"
        """
    }
    """
}

in your producer you can add val in the output tuple, this way your out2 will have a index than if needed can be merged with out1 using join by: [0]

output:
    val(meta), path('file1.txt'), emit: out1
    val(meta), path('file2.txt'), optional: true, emit: out2



script:
"""
echo "foo" > file1.txt
# Optionally create file2.txt
# echo "foo" > file2.txt
"""

}

4

When I use the syntax

path f2, optional: true

I keep getting the following error:

ERROR ~ No such variable: optional

I cannot figure out what I am doing wrong.

5

in current version nextflow it have not the option “optional“ for input block in process? unfortunately. I try too solve this problem

6

you are right.

I thought about it some more.

I think go with 2 processes.

using a conditional block to handle the process call. when 1 input, call A, when 2 input, call B

how about this demo script

#!/usr/bin/env nextflow

nextflow.enable.dsl=2

params.input1 = null
params.input2 = null   // optional second input

workflow {

    // Single input → processA
    if (params.input1 && !params.input2) {
        ch1 = Channel.fromPath(params.input1)
        processA(ch1)
    }

    // Two inputs → processB
    else if (params.input1 && params.input2) {
        ch1 = Channel.fromPath(params.input1)
        ch2 = Channel.fromPath(params.input2)
        processB(ch1, ch2)
    }

    else {
        exit 1, "❌ No valid inputs provided"
    }
}

// ------------------
// Process A
// ------------------
process processA {
    input:
    path file1

    output:
    path "output_A.txt"

    script:
    """
    echo "Process A running with input: ${file1}" > output_A.txt
    """
}

// ------------------
// Process B
// ------------------
process processB {
    input:
    path file1
    path file2

    output:
    path "output_B.txt"

    script:
    """
    echo "Process B running with inputs: ${file1} and ${file2}" > output_B.txt
    """
}