Singularity qiime2 container - qiime command not found

Ask for help
,
1

Hello,
I have tried testing multiple qiime2-core and qiime2-amplicon images using singularity. Everytime I try to test my nextflow script it ends up not recognizing qiime command (not finding qiime in my bash script). Am I doing something wrong?

https://hub.docker.com/r/qiime2/core

#!/usr/bin/env nextflow
process test {
	debug true
	label 'qiime'
        script:
        """
        qiime --info
        """

workflow {
	test()
}

config file

profiles{
	singularity{
    	singularity.enabled = true
    	process{
    	process.executor = 'slurm'
    	withLabel: qiime {
        	cpus = 10
        	memory = 100.GB
        	container = '/uufs/chpc.utah.edu/common/home/kapheim-group2/Sief/nextflow_script/singularity/amplicon_latest.sif'
    	}
    	}
	}
	conda{
    	conda.enabled = true
    	process{
        	process.executor = 'slurm'
        	withLabel: pandasenv{
            	cpus = 10
            	memory = 50.GB
            	conda = '/uufs/chpc.utah.edu/common/home/u6060645/software/pkg/miniconda3/envs/pandaenv'
        	}
    	}
	}
}
2

Hi @saif_s,

There’s a missing } in your Nextflow script. Can you please share the SIF image so that I can try to replicate it here on my side? You confirm that you set the -profile to singularity?

Next time, please make sure you share

  • the command line you used to run your pipeline e.g. nextflow run main.nf -profile singularity
  • the exact output error you got
  • The .nextflow.log file
3

Hi Marcel,

I will make sure to include a proper output profile next time. Regarding the command line here is exactly what I inputted
[u6060645@notch068:nextflow_script]$ nextflow run test.nf -profile singularity

the .command.log
....long directory removed/nextflow_script/work/4e/41bcb4aaf2f8ff510966c7fe489dc6/.command.sh: line 2: qiime: command not found
Command Exist Status: 127

the SIF image, please read below:

  1. I have tried using multiple images (the ones that I will link below are the ones that I have tried and they’ve given me exactly the same error!
  2. The way I got the SIF image is by using singularity pull dock://<image code or link>
  3. https://hub.docker.com/r/qiime2/core singularity pull dock://qiime2/core
  4. I also tried singularity pull docker://quay.io/qiime2/amplicon:2024.5
    as per this guide Installing QIIME 2 using Docker — QIIME 2 2024.5.0 documentation
4

It’s also worth mentioning that I got the same error even with the latest version of qiime2 as per the forum discussion here:

and there repo here

singularity build qiime2_2024.5 docker://quay.io/qiime2/amplicon:2024.5

5

Can’t reproduce. It works fine in Gitpod. You can try yourself here.

My main.nf:

#!/usr/bin/env nextflow
process test {
	debug true
	label 'qiime'
        script:
        """
        qiime --help
        """
}
workflow {
	test()
}

My nextflow.config

profiles{
	singularity{
    	singularity.enabled = true
    	process{
    	process.executor = 'slurm'
    	withLabel: qiime {
        	cpus = 10
        	memory = 1.GB
        	container = '/workspace/gitpod/nf-training/core_latest.sif'
    	}
    	}
	}
	conda{
    	conda.enabled = true
    	process{
        	process.executor = 'slurm'
        	withLabel: pandasenv{
            	cpus = 10
            	memory = 50.GB
            	conda = '/uufs/chpc.utah.edu/common/home/u6060645/software/pkg/miniconda3/envs/pandaenv'
        	}
    	}
	}
}

Built/pulled the singularity image with:

singularity pull docker://qiime2/core

Run the pipeline with

nextflow run main.nf -profile singularity

The issue is not with Nextflow, the pipeline code or Singularity. It’s probably related to something in our machine outside the scope of Nextflow.

6

Thanks for looking into it. I found a workaround for now. I built another conda sub-process for qiime2 using their provided yml. That fixed it for now. I will troubleshoot this problem further to see what the problem is.

1 Like
7

I just tested what you did and it worked. I believe (possibly) that the problem was with how I handled it with out HPCP. You need to first load singularity module before you can use it in nextflow. I made sure to load it first, and now it did work following the exact steps you provided.

Thank you!

8

I want to drop one more question if that’s okay.

My config file had process.container, between that and not loading module singularity, one or both was causing that error. Testing without module singularity loaded causes an error, and with loading it the error is gone (container = ‘’ is used), and with singularity loaded but using process.container I get the same error.

I can’t find in the document a clear difference between container vs process.container. Why was using process.container causing it not to identify the sif image?

9

After couple of tries and little reading. I think I understand. Process.config is not a process-specific option, it’s a global setting that goes into the main profile itself, so above singularity and conda. While conda=‘’ is a process-specific directive, that directs the processes to certain images.

That fixes the problem along with module loading singularity in home directory.

10

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.