I’m working on an nf-core pipeline, with the default CI set up. All release tests pass without issues, except one: pipeline download. The downloaded pipeline fails to run due to a Singularity issue. I get the following error:
Command error:
INFO: Converting SIF file to temporary sandbox...
FATAL: while extracting /home/runner/work/reportho/reportho/./community-cr-prod.seqera.io-docker-registry-v2-blobs-sha256-6b-6b2900901bc81cfb5d255a250ee196f4e2f8707ba6de704178eb40151fd849f8-data.img: root filesystem extraction failed: extract command failed: ERROR : Failed to create container process: Operation not permitted
: exit status 1
I’ve found information about this issue, but it was universally related to malformed containers. In this case, I am using a Seqera container, so no problems should arise. Has anyone experienced this issue, or do you have ideas on how to resolve it? The CI run can be found here.
This however, does not look like a proper singularity container, but rather a blob from docker.
In seqera containers you can, and should, create true singularity images that will look different from the one you have, they will be named like this, for example:
So, I would recommend before trying to debug more, to just building a true singularity image in the seqera containers service, and using that to see if it resolves. If not, then we can try to debug more.
This however, does not look like a proper singularity container, but rather a blob from docker.
I’m not sure that this is quite right @fmalmeida - both URLs are valid for Singularity containers from Seqera Containers. They just use different protocols: the first is https and the second is oras. Both should work with singularity pull. See this blog post for a bit more description. For Seqera Containers, the .sif flat image is a blob from the OCI registry. Singularity images just have a single layer with the one binary in it. Whether https or oras is better depends a little on use case, but for nf-core we recommend https (the first link) for reasons described in the blog post I linked above.
On the Seqera Containers web interface, you can choose https instead of oras with this check box:
Regarding what has gone wrong here, a search on Slack finds a fairly recent thread where the same error was being discussed. In that case, the problem was that the GitHub Actions runner was running out of space whilst pulling the Singularity image. It’d be my guess that this is the root cause here as well.
