# Why nextflow overwrite my input?

**URL:** <https://community.seqera.io/t/why-nextflow-overwrite-my-input/1962>\
**Category:** Ask for help\
**Tags:** nextflow\
**Created:** [March 13, 2025, 4:59am UTC](https://community.seqera.io/t/why-nextflow-overwrite-my-input/1962 "2025-03-13T04:59:11Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![LChan](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/lchan/32/1659_2.png) [@LChan](https://community.seqera.io/u/LChan)\
**Post date:** [March 13, 2025, 4:59am UTC](https://community.seqera.io/t/why-nextflow-overwrite-my-input/1962/1 "2025-03-13T04:59:11Z")

</div>

[cleanfq.nf](https://community.seqera.io/uploads/short-url/sQlZ7bEdnEeoA8yI1JfextKMdKU.nf) (1.8 KB)  
[sub.fq.gz](https://community.seqera.io/uploads/short-url/tLKsSSfqvLT9LkDxlfQKopgK9ZT.gz) (46.7 KB)

Dear seqera community,

I am new to nextflow. Attached is my first nextfow script, I tried to clean some weird characters found in the sub.fq.gz headers attached. However, the nextflow I wrote did run the first two process with no error, but it seems overwrote the input. Also, the third step failed. I appreciate anyone can help me test and suggests me what I am doing run.

Thank you so much.  
Best,  
LC

---

<div class="post-metadata">

**Author:** ![GeraldineVdA](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/geraldinevda/32/178_2.png) [@GeraldineVdA](https://community.seqera.io/u/GeraldineVdA)\
**Post date:** [March 14, 2025, 4:58pm UTC](https://community.seqera.io/t/why-nextflow-overwrite-my-input/1962/2 "2025-03-14T16:58:27Z")

</div>

Hi @LChan, have you had a chance to go through our newcomer training course, [Hello Nextflow](https://training.nextflow.io/latest/hello_nextflow/)? It covers the basics of managing inputs and outputs so I’d recommend you check that out and see if it helps you understand what’s going wrong with your script.

---

<div class="post-metadata">

**Author:** ![Adam\_Talbot](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/adam_talbot/32/62_2.png) [@Adam\_Talbot](https://community.seqera.io/u/Adam_Talbot)\
**Post date:** [March 17, 2025, 5:25pm UTC](https://community.seqera.io/t/why-nextflow-overwrite-my-input/1962/3 "2025-03-17T17:25:49Z")

</div>

Hi @LChan,

In your workflow, you are creating an output file with the same name as an input file so it tries to write over the existing file, e.g.:

```groovy
process clean_fq {
    publishDir "${params.output_dir}", mode: 'copy'

    input:
    path input_file1
    val input_filename

    output:
    path "${input_filename}", emit: cleaned_file

    script:
    """
    zcat ${input_file1} | awk '{
        if (NR % 4 == 1) {
            gsub(/\\x00/, "")
        }
        if (\$0 != "") {
            print
        }
    }' | gzip > ${input_filename}

    """
}

workflow {
    // Resolve the full path to the input file
    input_file1 = file(params.input_file1)

    // Extract the filename (excluding dir path)
    input_filename = input_file1.name

    clean_fq(input_file1, input_filename)
}

```

becomes:

```bash
zcat sub.fq.gz | awk '{
    if (NR % 4 == 1) {
        gsub(/\\x00/, "")
    }
    if (\$0 != "") {
        print
    }
}' | gzip > sub.fq.gz

```

Instead, you should make sure to rename the output file created at runtime.

```groovy
#! /usr/bin/env nextflow

// Define the input parameters
params.input_files = "fastq/sub.fq.gz"
params.output_dir = "cleaned_fastq"

// Define the processes

// process to clean the fastq file
process clean_fq {
    publishDir "${params.output_dir}", mode: 'copy'

    input:
    path input_file

    output:
    path "${output_filename}", emit: cleaned_file

    script:
    output_filename = "${input_file.baseName}" + ".trim.fastq.gz"
    """
    zcat ${input_file} | awk '{
        if (NR % 4 == 1) {
            gsub(/\\x00/, "")
        }
        if (\$0 != "") {
            print
        }
    }' | gzip > ${output_filename}
    """
}

workflow {
    // Resolve the full path to the input file
    input_files = Channel.fromPath(params.input_files)

    clean_fq(input_files)
}

```

Differences:

- Replace [file](https://www.nextflow.io/docs/latest/working-with-files.html#opening-files) with [Channel.fromPath](https://www.nextflow.io/docs/latest/reference/channel.html#frompath) to handle as many inputs as you like (file will just do one)
- Determine the output filename within the process [using the file methods](https://www.nextflow.io/docs/latest/working-with-files.html)
- Use the output filename within the script instead of the value

Benefits:

- This will run as many samples as you like using a glob! (`--input_files "fastq/*.fq.gz"`)
- no file collisions within the process ever (i.e. overwriting the input with the output)
- Easier to pass to the next process

Hopefully this gets you started and helps you fix the other two.

---

<div class="post-metadata">

**Author:** ![LChan](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/lchan/32/1659_2.png) [@LChan](https://community.seqera.io/u/LChan)\
**Post date:** [March 19, 2025, 8:13pm UTC](https://community.seqera.io/t/why-nextflow-overwrite-my-input/1962/4 "2025-03-19T20:13:14Z")

</div>

Hi Adam,

Thank you so much for pointing out my issues. I thought the same file name with different input and output folder wouldn’t be an issue. My understanding was wrong. All my three processes works now.

In my terminal, the `--input_files "fastq/*.fq.gz"` must be in double-quotes.

Really appreciate your help.

Best,  
LC

---

<div class="post-metadata">

**Author:** ![LChan](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/lchan/32/1659_2.png) [@LChan](https://community.seqera.io/u/LChan)\
**Post date:** [March 19, 2025, 8:14pm UTC](https://community.seqera.io/t/why-nextflow-overwrite-my-input/1962/5 "2025-03-19T20:14:10Z")

</div>

Hi @GeraldineVdA,

Thanks for the suggestion. I watched the tutorial, it helps a lot.

Best,  
LC

---

<div class="post-metadata">

**Author:** ![Adam\_Talbot](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/adam_talbot/32/62_2.png) [@Adam\_Talbot](https://community.seqera.io/u/Adam_Talbot)\
**Post date:** [March 21, 2025, 1:48pm UTC](https://community.seqera.io/t/why-nextflow-overwrite-my-input/1962/6 "2025-03-21T13:48:18Z")

</div>

> [@LChan](#):
>
> In my terminal, the `--input_files "fastq/*.fq.gz"` must be in double-quotes.

Ah woops! that’s my mistake. Yes you want to pass the string to Nextflow and not let your shell expand the glob.

In your terminal, this:

```bash
--input_files fastq/*.fq.gz

```

becomes:

```bash
 --input_files fastq/1.fq.gz fastq/2.fq.gz fastq/3.fq.gz fastq/4.fq.gz ...

```

Which doesn’t make sense to Nextflow. Quoting it prevents the expansion.

---

<div class="post-metadata">

**Author:** ![system](https://dub1.discourse-cdn.com/flex013/user_avatar/community.seqera.io/system/32/2402_2.png) [@system](https://community.seqera.io/u/system)\
**Post date:** [March 28, 2025, 1:48pm UTC](https://community.seqera.io/t/why-nextflow-overwrite-my-input/1962/7 "2025-03-28T13:48:23Z")

</div>

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.