Parallelizing a perl script that is executed from a python script

Is it possible for a Perl script to run in parallel if it’s executed from a Python script that is executed by the Nextflow script?

I’m hoping that Nextflow will iterate over files in a path and parallelize the execution of the perl script even though it was called by a python script. I’m following the instructions here: Processes — Nextflow 23.10.0 documentation

I think I might have to pass the filepath to the python script which then passes it to the perl script, so then Nextflow knows to run the script in parallel?

Could you elaborate with some examples about how you see this working?

I’m a bit unclear by what you mean when you say the Perl script is executed by a Python script.

it sounds like you should just call the Perl script directly from Nextflow. No need for a Python intermediary. Typically in these kinds of situations we end up re-writing whatever custom behavior that was being performed in the Python wrapper directly into Nextflow instead.

It seems like the Nextflow script that I got wasn’t set up well. It runs a Perl script that then runs a lot more Perl scripts and other CLI commands. I replaced the main Perl script with a Python one because the code quality was very poor. I now realize that the Perl scripts should not have been called by the main script in the first place for a number of reasons.

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.