Parallelizing a perl script that is executed from a python script

Ask for help
1

Is it possible for a Perl script to run in parallel if it’s executed from a Python script that is executed by the Nextflow script?

I’m hoping that Nextflow will iterate over files in a path and parallelize the execution of the perl script even though it was called by a python script. I’m following the instructions here: Processes — Nextflow 23.10.0 documentation

I think I might have to pass the filepath to the python script which then passes it to the perl script, so then Nextflow knows to run the script in parallel?

2

Could you elaborate with some examples about how you see this working?

I’m a bit unclear by what you mean when you say the Perl script is executed by a Python script.

3

it sounds like you should just call the Perl script directly from Nextflow. No need for a Python intermediary. Typically in these kinds of situations we end up re-writing whatever custom behavior that was being performed in the Python wrapper directly into Nextflow instead.

1 Like
4

It seems like the Nextflow script that I got wasn’t set up well. It runs a Perl script that then runs a lot more Perl scripts and other CLI commands. I replaced the main Perl script with a Python one because the code quality was very poor. I now realize that the Perl scripts should not have been called by the main script in the first place for a number of reasons.

1 Like
5

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.