How to make errors from template scripts print and get added to .nextflow.log


I have a Nextflow module that runs a python script from bin folder to check sample input, it makes assertion errors if theres anything wrong with the input, but I can’t seem to get these errors to print to the logging output that appears when you run the pipeline or to print in the .nextflow.log - I checked another nf-core pipeline and they seem to use AssertionErrors so not sure what I need to do here?

Thanks for any direction!

Welcome to the community forum, @CharlotteAnne!

You can use, log.warn and log.error to print to the standard output and register the event/message to the Nextflow log file. Is this what you’re looking for? If you can share a minimal reproducible example, I can try to work out a solution for your specific case.

do you mean logger.error in the python script? or are you talking about having something in the nextflow module code itself?

could you possibly link me to somewhere this is done eg. in nfcore for example? I checked nfcore/rnaseq but I don’t understand how the input spreadsheet is checked now, theres no module that does this anymore, instead it appears to be a groovy function but unclear where this is defined: WorkflowRnaseq.validateInput(it)

I also checked nf-core/rnasplice - I am basically doing it exactly as they do with the SAMPLESHEET_CHECK module and has assertion errors in it.

Yes to both questions.

It’s a Nextflow function that should be placed in Nextflow script files, and nf-core pipelines do that with Groovy functions. You can also capture the stdout/stderr from the Python script and print it from the process (setting the debug process directive to true, for example).

Hrmm setting debug to true in my process still isn’t passing on the AssertionError.

Sorry I know a minimal example would be more helpful but that would take hours to put together because theres so many layers to this. → → → python script

I’d be grateful if you could point me to an nf-core pipeline that is doing this in the way you suggest - like I said I’ve implemented it like rnasplice/modules/local/ at f70bbce27cbade36f993a7e39fa6e574fc4c6d3a · nf-core/rnasplice · GitHub and its not working. I don’t know any groovy so would be a bit difficult for me to reimplement the python as groovy functions, but if you could link me to where in the rnaseq code this is happening I could try to emulate it :confused:

Thanks for your help!!

So I identified that actually the python script isn’t being run at all, but there are no errors output about that, the only error is that the next process doesn’t have its input. So now I need to figure out why the script isn’t being run :sweat_smile:

Just to clarify, if your Python script raises an error the traceback will be added to the .nextflow.log. If it runs successfully these logs will not surface in the Nextflow log file.

Thanks! btw my code is here → clipseq/modules/local/ at 03fe17f55c3d17faef94d7d542522e3c502066f7 · nf-core/clipseq · GitHub
can you see any obvious reason why the python script isn’t running/ any tips for how I can debug this?

Sorry this has been a total wild goose chase. My original error actually was elsewhere but now get this error trying to run the python script: /Users/capitac/repos/nfcore-clipseq/bin/ /usr/bin/env: bad interpreter: Permission denied

I have tried #!/usr/bin/env python3 and #!/usr/bin/env python at the start of my python script but neither work - any ideas?

It was an issue with the container - switched container everything is now happy - sorry to waste your time!!

So I guess to clarify for anyone reading, the answer to my original question is that AssertionErrors work totally fine :+1:

Thanks for sharing the outcome with us, @CharlotteAnne ! :wink:

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.