Using a container in a module

Ask for help
1

I created a docker container with python and several packages which I put in my module. But then the process can’t locate the files the way it’s supposed to:

This is my module:

process TRANSFER_DATA {
    label 'process_single'
    container 'docker://rbarrant/champlain:0.1'

    input:
    tuple val(meta), val(info)

    output:
    tuple val(meta), val(info), emit: info

    when:
    task.ext.when == null || task.ext.when

    script:
    def source_directory=meta.id

    """
    python transfer_data.py ${source_directory} 
    """
}

But then I get: >

Command error:
python: can’t open file ‘transfer_data.py’: [Errno 2] No such file or directory

And if for example I add

#!/usr/bin/env python

at the beginning of the script then it can’t find the local directories (I.e. ${source_directory}.

What am I missing?

2

python transfer_data.py ... doesn’t work because python will look for transfer_data.py in the current directory (where your process inputs are staged).

The ${source_directory} likely doesn’t work because it’s probably a reference to something not in the container image - when executing containers, nextflow only binds in what is necessary (and the nextflow way is to accept it as a path - then nextflow will stage it in the work directory).

The output seems odd.

3

transfer_data.py is in the bin directory, i have used that all the time and I can run this successfully in the conda environment. Definitely the ${source_directory} is not in the container image. All that said, you make some good points about the binding. Mine is a very particular use case so I will ask about that separately.

4

Container names should *not* be prefixed with docker:// or other (pseudo) protocol scheme

5

For a script to be found in the bin directory you can’t call it using the interpreter.

e.g.,
python transfer_data.py does not check the PATH variable for the script. Only the current directory.
But transfer_data.py without the interpreter before (python) will look in the PATH, which the bin/ folder is included in.

1 Like
6

Thank you @paolo , what should I put then? I did try various things but only this worked.

7

If your container was uploaded to Docker hub the canonical name is rbarrant/champlain:0.1

1 Like
8

Regarding the error

Command error:
python: can’t open file ‘transfer_data.py’: [Errno 2] No such file or directory

you need to change the process script to

transfer_data.py ${source_directory}

and transfer_data.py must have execute permissions. See here for more details

1 Like
9

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.