Starting Nextflow in a singularity container and use it to run another singularity container

Ask for help
,
1

Hello,

I am trying to run nextflow in a singularity container and further use this for running other pipelines that are made for nextflow. This will help us bypass installation of nextflow in our servers. I am running this on a Linux server.

I am able to run nextflow on a singularity container

singularity exec --env NXF_HOME=/home/.nf-singularity \
-B /data/home/giab_NA12878/Data_LongRead_sequencing/:/input \
-B /data/home/my_tools/sifs/:/sifs \
-B /data/home/my_tools/nf_tools:/tools \
-B /data/home/databases/UCSC_download/:/db \
/data/home/my_tools/sifs/nextflow_latest.sif \
nextflow

I am also able to run the singularity container that contains the workflow on a separate server that has nextflow installed. I also made a config file for nextflow

profiles {
        foo {
                autoMounts = true
                docker.enabled = false
                singularity.enabled = true
                singularity.cacheDir = "$PWD"
                process.container = '/data/home/my_tools/sifs/wf_human_variantion_mr442.sif'
                report.overwrite = true
                timeline.overwrite = true
        }
}

I tried to launch the script like this:

#!/bin/bash
export NXF_VER=23.04.2;
export NXF_ANSI_LOG=false
singularity exec --env NXF_HOME=/home/.nf-singularity \
-B /data/home/giab_NA12878/Data_LongRead_sequencing/:/input \
-B /data/home/my_tools/sifs/:/sifs \
-B /data/home/my_tools/nf_tools:/tools \
-B /data/home/databases/UCSC_download/:/db \
/data/home/my_tools/sifs/nextflow_latest.sif \
nextflow run -c /input/wf_sing_nf.config /tools/wf-human-variation -profile foo \
-w /input/nf_test/workspace --snp --str --cnv \
--bam /input/giab_NA12878_only_chr_sort.bam --bam_min_coverage 0 \
--ref /db/human_selected_onlychr.fasta --sample_name giab_hg001_nf \
--basecaller_cfg dna_r10.4.1_e8.2_400bps_sup@v4.2.0 \
--out_dir /input/nf_test/HG001_test_nextflow \
-with-report /input/nf_test/run_report_hg001_nf.html \
-with-timeline /input/nf_test/run_timeline_hg001_nf.html

I get the following error

WARN: Inferring genetic sex of sample as params.sex was not provided.
Pulling Singularity image docker://ontresearch/wf-common:sha338caea0a2532dc0ea8f46638ccc322bb8f9af48 [cache /home/ontresearch-wf-common-sha338caea0a2532dc0ea8f46638ccc322bb8f9af48.img]
Pulling Singularity image docker://ontresearch/wf-human-variation-str:shaa2f49ce57886426516eadd4048b6fdf9c22c7437 [cache /home/ontresearch-wf-human-variation-str-shaa2f49ce57886426516eadd4048b6fdf9c22c7437.img]
Pulling Singularity image docker://ontresearch/spectre:sha5a2890023dc7a7899f47585103b4f5762fb9d1b3 [cache /home/ontresearch-spectre-sha5a2890023dc7a7899f47585103b4f5762fb9d1b3.img]
Pulling Singularity image docker://ontresearch/wf-human-variation-snp:sha17e686336bf6305f9c90b36bc52ff9dd1fa73ee9 [cache /home/ontresearch-wf-human-variation-snp-sha17e686336bf6305f9c90b36bc52ff9dd1fa73ee9.img]
ERROR ~ Error executing process > 'str:getVersions'

Caused by:
  Failed to pull singularity image
  command: singularity pull  --name ontresearch-wf-human-variation-str-shaa2f49ce57886426516eadd4048b6fdf9c22c7437.img.pulling.1720615544417 docker://ontresearch/wf-human-variation-str:shaa2f49ce57886426516eadd4048b6fdf9c22c7437 > /dev/null
  status : 127
  message:
    bash: line 1: singularity: command not found

I could add the .nextflow.log report as well if it is useful. I have tried multiple ways that the local singularity container is recognised but it isnt working. Is there something within my script I should add so that the other singularity image is also recognised? am I missing something?

Thanks for your help