Config syntax errors with `includeConfig`

Ask for help
1

TLDR; Using includeConfig( { if/esle.. return file.config } )() pattern to include some dynamic configuration without causing syntax errors is not playing nice.

Not an issue with the language server per se, but while working on removing syntax errors from config files (see docs here), I have attempted to employ the advice in the Config Syntax documentation. The suggestion is to pass a closure to includeConfig with a conditional that will return the path to the correct config file to include for the condition. This pattern is nice and has cleaned up the configs nicely. I have run into two issues while doing this and wondering what I am missing.

  1. When I try to invoke includeConfig with the parentheses on the outside, as shown in the docs i.e., includeConfig({ ... })() nextflow flat-out refuses to parse the config file and reports Cannot invoke method call() on null object
  2. If I move the parens inside i.e., includeConfig({ ... }()) (note it’s a subtle difference) it works as far as the including the correct config, and nextflow config -flat confirms the expected parameters are set. Although it does warn WARN: Unknown config attribute 'includeConfig'

It seems as #2 is enough to stop the pipeline from launching.

I really want to have syntax error free configs so any new syntax errors will rise to the top as we develop. What might I be missing here?

2

I’ll wait for @bentsherman to weigh in on the docs example (thanks for reporting that!), but in the mean time it may be interesting to see the syntax that we used to work around this in the nf-core template:

// Load nf-core custom profiles from different Institutions
includeConfig variable_is_true ? "conf/one.conf" : "conf/two.confg"

So using a ternary expression. Seems to work well! (source)

1 Like
3

Thanks @ewels!
It would be good in my case to be able to use the full scope of the closure, similar to the example where the input to the conditional is generated. Maybe I can come-up with a work around. Always curious to know if missing something or when I could have a better approach.

4

Thanks for reporting. The parentheses is my mistake, with the existing config parser you need to put the parentheses around the entire closure invocation as you did in (2). I will correct that in the docs.

Here is a minimal example I made that works:

params {
    config = null
}

includeConfig ({
    if( params.config == 'foo' )
        return 'foo.config'

    if( params.config == 'bar' )
        return 'bar.config'

    return '/dev/null'
}())

For some reason, if params.config is undefined then the run will fail, so you have to provide a value for it and/or a default value in the config as I have done. I didn’t get the error about includeConfig.

5

Another way you can write it is:

includeConfig ({
  // ...
}.call())

Let me know if that works for you

6

Thanks @bentsherman, confirmation that the parentheses should be inside was all I needed to isolate the issue I was having. I can now use the full closure scope and get the expected return value.

1 Like
7

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.