Docker: how to run a script within nextflow?

Nextflow Help
1

Hi,
I’ve a docker container which runs fine on command line as:

docker exec vicc_test /bin/grandsonofvictor.py --output /full/path/ --sample_id M-0489 --type mutation --es_url elastic:name123@ec2-54-84-170-0.compute-1.amazonaws.com:9000 /path/annotated_variants.consensus.vcf

I get output at /full/path/

vicc_test is the docker tag.

Now, I’m interested in running this inside nextflow process. I’ve code below as:

 process test {
 
 container 'sinaidgt/vicc-api:bc55e0123405932bf2042641bc'
     maxForks 3
         debug true
         errorStrategy 'retry'
     maxRetries 2
 publishDir path: "path/full/learn_nextflow/", mode: 'copy' ,  overwrite: true
 
 output:
  path "versions.yml" , emit: versions
 stdout
     script:
     """
 echo "hello"
  /bin/grandsonofvictor.py
"""
 }
 
 workflow {
     test()
 }

It fails with error:

Command error:
hello
.command.sh: line 3: /bin/grandsonofvictor.py: No such file or directory

I have tried following as well:

./bin/grandsonofvictor.py
./grandsonofvictor.py
grandsonofvictor.py

But none of these have worked.

I am new to docker concept overall.

How do I get this working?

2

If I understood well, /bin/grandsonofvictor.py is not inside the container image, but at the host operating system. It should be in a folder named bin in the project folder of the Nextflow pipeline. You seem to be providing it from the root (/) as /bin/grandsonofvictor.py. Try putting it at ./bin/grandsonofvictor.py and giving it +x permission (chmod +x ./bin/grandsonofvictor.py. By doing these two things, Nextflow will automatically make sure grandsonofvictor.py is accessible from within the container image. You don’t need the ./bin/ when calling it in the process block.

3

Hi @mribeirodantas,

Thank you for looking into this.

I don’t think I understand and follow you. The grandsonofvictor is on docker.

I’ve a bin folder at the folder structure where I’m trying to run this nextflow script.
I do not see grandsonofvictor.py inside the bin folder.

How do I chmod the victor script? At docker? Can you please help me further?

If I put the victor script inside bin folder when how would I use the docker?

I only put grandsonofvictor.py in the process block and run it, I get error as:

.command.sh: line 4: grandsonofvictor.py: command not found

I also do not know how to copy scripts/files from docker to local.

I honestly do not understand docker and nextflow a layer on it is completely mysterious.

4

Ideally, you just let Nextflow take care of staging/moving files around. You shouldn’t have to worry about this.

I recommend you have a look at this section of the foundational training. It will help you understand what I meant in my previous post. Let me know if it’s not solved by then :wink:

5

@mribeirodantas

I know nextflow takes care of moving and staging.

Thanks for sharing the link, I’ve combed through it before posting on the forum. It doesn’t solve the issue I’m at. Instead of command line, I mention docker image inside the process interested as there are multiple processes with different docker images.

Sorry, if my post wasn’t clearer - the docker has a bin folder and then a script there as sonofwhatever.py

How do I check that nextflow is inside or activated the docker container? Is there a way to list ls contents there?

Finally, how do I have that script in bin folder at docker run?

Thanks for your replies.

6

If you’re sure the container image has a script named sonofwhatever.py at /bin/, and have configured Nextflow properly to use Docker with this container image for a specific process, I don’t see a reason for running into command not found. Can you share the container image so that I could try locally?

You can check the contents of the .command.run file within the task folder [of the process you’re interested in]. There should be docker command lines in it, such as in the nxf_launch function below:

nxf_launch() {
    docker run -i --cpu-shares 1024 -e "NXF_TASK_WORKDIR" -v /Users/mribeirodantas:/Users/mribeirodantas -w "$PWD" --name $NXF_BOXID quay.io/nextflow/rnaseq-nf:v1.2.1 /bin/bash -c "eval $(nxf_container_env); /bin/bash -ue /Users/mribeirodantas/work/f8/765aa40b350963bfda54fe6043b9eb/.command.sh"
}
7

Thank you for your reply.
The image I was using wasn’t latest. Solved after correcting for the image.

1 Like
8

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.