Hi, I’m trying to add Nextflow to our GitLab CI/CD system and I’ve been trying to use Nextflow 23.10.1 from Docker unsuccessfully because when you call nextflow the following error is raised:
Error: Unable to access jarfile /.nextflow/framework/23.10.1/nextflow-23.10.1-one.jar Unable to initialize nextflow environment
It gives me the feeling these containers are not set up properly, but I must confess I’m not certain. So any help would be much appreciated.
Welcome to the forum, @JAlvarez. Can you provide more details about how you’re running Nextflow in your GitLab CI/CD system? Also, have you tried a different version?
Thank you @mribeirodantas ! I have tried to just do nextflow -version in the container from Docker and I was getting the exception above, even from my local machine (so no complicated GitLab CI/CD runners or anything like that, just a simple Ubuntu with Singularity CE).
The exact list of commands:
singularity pull nextflow.sif docker://nextflow/nextflow:23.10.1
singularity shell nextflow.sif
nextflow -version
Then I believe this is a version issue, because I did the following with nextflow:latest and it worked.
docker pull nextflow/nextflow:latest
docker run -ti nextflow/nextflow bash
nextflow run hello
Check the output:
Do you really need Nextflow 23.10.1?
UPDATE
I also tested 23.10.1 and it worked here 
It may be because you are using Docker? With Singularity the following happens:
$ singularity run docker://nextflow/nextflow:latest nextflow run hello
INFO: Converting OCI blobs to SIF format
INFO: Starting build...
INFO: Creating SIF file...
Error: Unable to access jarfile /.nextflow/framework/24.02.0-edge/nextflow-24.02.0-edge-one.jar
Unable to initialize nextflow environment
Based on my understanding, the problem is that the jar file is created by root user with no group or other reading permission (see below), and whilst Docker uses root as the default user, Singularity does not (main reason why it is the preferred technology in our cluster).
Singularity> ls -l /.nextflow/framework/24.02.0-edge/nextflow-24.02.0-edge-one.jar
-rw------- 1 root root 1706444 Mar 10 20:21 /.nextflow/framework/24.02.0-edge/nextflow-24.02.0-edge-one.jar
Using the default generated singularity image, there are two problems. One is that the NXF_HOME is at /.nextflow and is owned by the root, and the other is that it’s a read-only filesystem. You could solve the root problem by running singularity with --fake-root, but you will still hit the problem of a read-only filesystem.
One workaround is to change the NXF_HOME to a writable folder.
$ singularity run --env NXF_HOME=$(pwd)/.nf-singularity docker://nextflow/nextflow:latest nextflow run hello
INFO: Using cached SIF image
N E X T F L O W ~ version 24.02.0-edge
┃ Launching `https://github.com/nextflow-io/hello` [boring_sinoussi] DSL2 - revision: 7588c46ffe [master]
executor > local (4)
[9c/f086b8] process > sayHello (2) [100%] 4 of 4 ✔
Hello world!
Hola world!
Bonjour world!
Ciao world!
This will work, but you will always download all the Nextflow jar dependencies each time if that folder does not persist.
That is doable in our current infrastructure. Thank you very much!