Custom module linked via symlink not found during import

Ask for help
1

I’m getting a ModuleNotFoundError when trying to import my custom module my_model

File "/path/to/predict_with_my_model/bin/predict.py", line 14, in <module>
      import my_model.data as d
  ModuleNotFoundError: No module named 'my_model'

In the bin directory there is a symlink to the local directory my_model, a symlink to the predict.py script and a few other symlinks and files.

$ ls bin
my_model  model.py  predict.py  predict.sh

I’m running the script inside a Nextflow pipeline using:

script:
    """
    predict.sh
    """

And predict.sh looks like this:

#!/bin/bash

python /path/to/predict_with_my_model/bin/predict.py

Even though the my_model directory is symlinked into bin, python can’t find the module when importing.

2

When you add a script to the bin/ directory, it is added to the $PATH within the process. This means those scripts become executable directly, as if they were available on the command line.

However, there’s no guarantee that the paths symlinked into bin/ will also be staged. In fact, I don’t think there’s any way for Nextflow to know it should symlink additional paths.

Then there’s the complication of Python import paths, which may or may not resolve correctly. It’s hard to tell from your example, but I can’t imagine it working reliably when using an absolute path to the Python file.

Additionally, your predict.sh script calls a hardcoded path, which is almost certainly going to fail. I’d recommend calling predict.py directly instead.

  • Move the files directly into the bin/ directory - no need to symlink them.
  • Add a Python shebang to predict.py if it doesn’t already have one (e.g., #!/usr/bin/env python3).
  • Replace predict.sh by calling predict.py directly:
script:
"""
predict.py
"""
1 Like