Help setting apptainer cache dir

question on config variable setting~

I need to use nextflow v22.10.4 for a non-nf-core pipeline. I set my conda/singularity cache dirs in my config file, but am getting the following error indicating that apptainer is looking at the wrong directory (/$HOME/$USER/ instead of specified dir which is on a different node called /ceph/)

FATAL:   While making image from oci registry: error fetching image to cache: while building SIF from layers: while creating SIF: while unloading container: close /dicos_ui_home/ewissel/.apptainer/cache/oci-tmp/tmp_3980167945: disk quota exceeded

I tried to update the apptainer cache in my config file but must have done so unsuccessfully as I got the same error on a rerun. Can someone help me fix the apptainer cache directory? I used the singularity/conda cache dir commands as a template, so I don’t quite understand where I’m going wrong here.

## relevant partt of nextflow.config
profiles {
  standard {
    conda {
      useMamba = false
      conda.enabled = true
      // Allow longer conda creation timeout
      createTimeout = '2 h'
      cacheDir = "/ceph/work/IBMS-PHLab/emily/nf_conda/"
      }
      params.enable_conda = true
      singularity.enabled = false
      singularity.automounts = false
      docker.enabled = false
      podman.enabled = false
      shifter.enabled = false
      charliecloud.enabled = false
  }
  conda {
    conda {
    useMamba = false
    conda.enabled = true
    // Allow longer conda creation timeout
    createTimeout = '2 h'
    cacheDir = "/ceph/work/IBMS-PHLab/emily/nf_conda/"
    }
    params.enable_conda = true
    singularity.enabled = false
    singularity.automounts = false
    docker.enabled = false
    podman.enabled = false
    shifter.enabled = false
    charliecloud.enabled = false
  }
  singularity {
    singularity.enabled = true
    singularity.autoMounts = true
    singularity.runOptions = "--bind /ceph/sharedfs/work/IBMS-PHLab/emily/tmp:/tmp"
    singularity.cacheDir = "/ceph/work/IBMS-PHLab/emily/singularity-images/"
    params.enable_container=true
    docker.enabled = false
    podman.enabled = false
    shifter.enabled = false
    charliecloud.enabled = false
  }
  docker {
    singularity.enabled = false
    singularity.autoMounts = false
    docker.enabled = true
    params.enable_container=true
    podman.enabled = false
    shifter.enabled = false
    charliecloud.enabled = false
  }
  apptainer  {
    apptainer.cacheDir =  "/ceph/work/IBMS-PHLab/emily/singularity-images/"    ## what I added
 }
}

When I run nextflow, i specify -profile singularity, so I’m not sure if the apptainer cache needs to be connected to the singularity profile in this instead of how it is now? Thanks for the help!

Hey @ewissel.

What you’re actually doing is apptainer.apptainer.cacheDir = ".... You should do instead:

apptainer {
  cacheDir = "/ceph/work/IBMS-PHLab/emily/singularity-images/"
}

Or you can also set your cache directory for apptainer with an environment variable: NXF_APPTAINER_CACHEDIR.

Could you please try again and let me know if it worked?

this worked, and is a big “duh” in hindsight. Thank you!!

1 Like