Error on selecting a specific output from a process that outputs multiple files, and pass it to the next process?

Ask for help
,
1

I keep getting this error when I use my code below:

Error:

Not a valid path value type: groovyx.gpars.dataflow.DataflowBroadcast (DataflowBroadcast around DataflowStream[?])

Code:

process QIME2DENOISE{
	debug true
	label 'qiime2conda'
	input:
	path trimmed_file
	output:
	path 'rockRAPID_*'
	script:
	"""
	qiime dada2 denoise-paired --p-n-threads 8 --i-demultiplexed-seqs $trimmed_file --p-trim-left-f 0 --p-trim-left-r 0 --p-trunc-len-f 195 --p-trunc-len-r 219 --o-table ./rockRAPID_table.qza --o-representative-sequences ./rockRAPID_repseqs.qza --o-denoising-stats ./rockRAPID_denoising-stats.qza
	"""
}

process QIME2SUMMARIZE{
	debug true
	label 'qiime2conda'
	input:
	path 'rockRAPID_table.qza'
	output:
	path '*_visualization.qzv'
	script:
	"""
	qiime feature-table summarize --i-table rockRAPID_table.qza --o-visualization ./rockRAPID_table_visualization.qzv
	"""
}

workflow {
	qime2_denoise_ch = QIME2DENOISE(previous_process_ch)
	qime2_table_summary_ch = QIME2SUMMARIZE{qime2_denoise_ch}
}

qime2_denoise does output 3 files as they’re named in the script, the problem is I’m not sure if I’m retrieving the one I want properly in the next process.

Can you please help?

2

I believe that by

qime2_table_summary_ch = QIME2SUMMARIZE{qime2_denoise_ch}

you meant

qime2_table_summary_ch = QIME2SUMMARIZE(qime2_denoise_ch)
1 Like
3

I apologize for the typo, here is the full version, and yes you are correct that was a typo:

process QIME2DENOISE{
    debug true
    label 'qiime2conda'
    input:
    path trimmed_file

    output:
    path 'rockRAPID_*'

    script:
    """
    qiime dada2 denoise-paired --p-n-threads 8 --i-demultiplexed-seqs $trimmed_file --p-trim-left-f 0 --p-trim-left-r 0 --p-trunc-len-f 195 --p-trunc-len-r 219 --o-table ./rockRAPID_table.qza --o-representative-sequences ./rockRAPID_repseqs.qza --o-denoising-stats ./rockRAPID_denoising-stats.qza
    """
}

process QIME2SUMMARIZE{
    debug true

    label 'qiime2conda'

    input:
    path "rockRAPID_table.qza"

    output:
    path '*_visualization.qzv'

    script:
    """
    qiime feature-table summarize --i-table rockRAPID_table.qza --o-visualization ./rockRAPID_table_visualization.qzv
    """
}

workflow {
    qime2_denoise_ch = QIME2DENOISE(qime2_trim_ch)
    qime2_table_summary_ch = QIME2SUMMARIZE(qime2_denoise_ch)
}

The error that i’m getting in the log file is

                     There was a problem with the command:                     
   (1/1) Invalid value for '--i-table': rockRAPID_table.qza does not exist.
4

Can you please check why I’m getting error after trying this solution?

process QIME2DENOISE{
    debug true
    label 'qiime2conda'
    input:
    path trimmed_file

    output:
    path 'table.qza',   emit: table
    path 'repseqs.qza', emit: repseqs
    path 'denoising-stats.qza', emit: stats

    script:
    """
    qiime dada2 denoise-paired --p-n-threads 8 --i-demultiplexed-seqs $trimmed_file --p-trim-left-f 0 --p-trim-left-r 0 --p-trunc-len-f 195 --p-trunc-len-r 219 --o-table ./table.qza --o-representative-sequences ./repseqs.qza --o-denoising-stats ./denoising-stats.qza
    """
}

process QIME2SUMMARIZE{
    debug true

    label 'qiime2conda'

    input:
    path table

    output:
    path '*_visualization.qzv'

    script:
    """
    qiime feature-table summarize --i-table $table --o-visualization ./rockRAPID_table_visualization.qzv
    """
}


workflow {
    qime2_denoise_ch = QIME2DENOISE(qime2_trim_ch)
    qime2_summary_ch = QIME2SUMMARIZE(qime2_denoise_ch.out.table)
}

Removing qime2_summary_ch removes the error below:

ERROR ~ No such variable: Exception evaluating property 'out' for nextflow.script.ChannelOut, Reason: groovy.lang.MissingPropertyException: No such property: out for class: groovyx.gpars.dataflow.DataflowBroadcast

It is related to “out”. Here is the error from the log file:

Aug-09 09:54:13.739 [main] DEBUG nextflow.cli.Launcher - $> nextflow run main.nf -profile conda -resume
Aug-09 09:54:13.946 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 23.10.0
Aug-09 09:54:13.975 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/uufs/chpc.utah.edu/common/home/u6060645/.nextflow/plugins; core-plugins: nf-amazon@2.1.4,nf-azure@1.3.2,nf-cloudcache@0.3.0,nf-codecommit@0.1.5,nf-console@1.0.6,nf-ga4gh@1.1.0,nf-google@1.8.3,nf-tower@1.6.3,nf-wave@1.0.0
Aug-09 09:54:13.989 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Aug-09 09:54:13.990 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Aug-09 09:54:13.994 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Aug-09 09:54:14.008 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Aug-09 09:54:14.029 [main] DEBUG nextflow.config.ConfigBuilder - Found config local: /uufs/chpc.utah.edu/common/home/kapheim-group2/Sief/nextflow_script/nextflow.config
Aug-09 09:54:14.030 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /uufs/chpc.utah.edu/common/home/kapheim-group2/Sief/nextflow_script/nextflow.config
Aug-09 09:54:14.057 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `conda`
Aug-09 09:54:14.983 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [conda, singularity]
Aug-09 09:54:15.029 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 by global default
Aug-09 09:54:15.053 [main] INFO  nextflow.cli.CmdRun - Launching `main.nf` [shrivelled_rutherford] DSL2 - revision: 456e0ba77a
Aug-09 09:54:15.054 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Aug-09 09:54:15.055 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[]
Aug-09 09:54:15.067 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /uufs/chpc.utah.edu/common/home/u6060645/.nextflow/secrets/store.json
Aug-09 09:54:15.071 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@6e7c351d] - activable => nextflow.secret.LocalSecretsProvider@6e7c351d
Aug-09 09:54:15.156 [main] DEBUG nextflow.Session - Session UUID: 54de6efa-9df5-4c93-b88a-b59bdf46fed2
Aug-09 09:54:15.156 [main] DEBUG nextflow.Session - Run name: shrivelled_rutherford
Aug-09 09:54:15.157 [main] DEBUG nextflow.Session - Executor pool size: 32
Aug-09 09:54:15.167 [main] DEBUG nextflow.file.FilePorter - File porter settings maxRetries=3; maxTransfers=50; pollTimeout=null
Aug-09 09:54:15.172 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=96; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Aug-09 09:54:15.216 [main] DEBUG nextflow.cli.CmdRun - 
  Version: 23.10.0 build 5889
  Created: 15-10-2023 15:07 UTC (09:07 MDT)
  System: Linux 4.18.0-477.15.1.el8_8.x86_64
  Runtime: Groovy 3.0.19 on OpenJDK 64-Bit Server VM 17.0.1+12
  Encoding: UTF-8 (UTF-8)
  Process: 3721644@notch037 [10.242.75.37]
  CPUs: 32 - Mem: 100 GB (83 GB) - Swap: 15 GB (15 GB)
Aug-09 09:54:15.274 [main] DEBUG nextflow.Session - Work-dir: /uufs/chpc.utah.edu/common/home/kapheim-group2/Sief/nextflow_script/work [nfs]
Aug-09 09:54:15.275 [main] DEBUG nextflow.Session - Script base path does not exist or is not a directory: /uufs/chpc.utah.edu/common/home/kapheim-group2/Sief/nextflow_script/bin
Aug-09 09:54:15.287 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Aug-09 09:54:15.299 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Aug-09 09:54:15.324 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Aug-09 09:54:15.336 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 33; maxThreads: 1000
Aug-09 09:54:15.427 [main] DEBUG nextflow.Session - Session start
Aug-09 09:54:16.101 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Aug-09 09:54:16.231 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:pandasenv` matches labels `pandasenv` for process with name PYMANIFESTCREATOR
Aug-09 09:54:16.239 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Aug-09 09:54:16.239 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Aug-09 09:54:16.246 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Aug-09 09:54:16.253 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=32; memory=100 GB; capacity=32; pollInterval=100ms; dumpInterval=5m
Aug-09 09:54:16.256 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local)
Aug-09 09:54:16.357 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:qiime2conda` matches labels `qiime2conda` for process with name QIME2IMPORTMANIFEST
Aug-09 09:54:16.358 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Aug-09 09:54:16.358 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Aug-09 09:54:16.364 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:qiime2conda` matches labels `qiime2conda` for process with name QIME2TRIMM
Aug-09 09:54:16.365 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Aug-09 09:54:16.365 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Aug-09 09:54:16.373 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:qiime2conda` matches labels `qiime2conda` for process with name QIME2DENOISE
Aug-09 09:54:16.374 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Aug-09 09:54:16.374 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Aug-09 09:54:16.381 [main] DEBUG nextflow.script.ScriptRunner - Parsed script files:
  Script_33051c2cd25b493e: /uufs/chpc.utah.edu/common/home/kapheim-group2/Sief/nextflow_script/main.nf
Aug-09 09:54:16.381 [main] DEBUG nextflow.Session - Session aborted -- Cause: Exception evaluating property 'out' for nextflow.script.ChannelOut, Reason: groovy.lang.MissingPropertyException: No such property: out for class: groovyx.gpars.dataflow.DataflowBroadcast
Aug-09 09:54:16.410 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop
Aug-09 09:54:16.410 [main] ERROR nextflow.cli.Launcher - @unknown
groovy.lang.MissingPropertyException: Exception evaluating property 'out' for nextflow.script.ChannelOut, Reason: groovy.lang.MissingPropertyException: No such property: out for class: groovyx.gpars.dataflow.DataflowBroadcast
	at org.codehaus.groovy.runtime.DefaultGroovyMethods.getAtIterable(DefaultGroovyMethods.java:8763)
	at org.codehaus.groovy.runtime.DefaultGroovyMethods.getAt(DefaultGroovyMethods.java:8751)
	at groovy.lang.MetaClassImpl.getProperty(MetaClassImpl.java:1927)
	at groovy.lang.MetaClassImpl.getProperty(MetaClassImpl.java:3815)
	at groovy.lang.DelegatingMetaClass.getProperty(DelegatingMetaClass.java:126)
	at nextflow.script.ChannelOut.getProperty(ChannelOut.groovy:78)
	at org.codehaus.groovy.runtime.callsite.PogoGetPropertySite.getProperty(PogoGetPropertySite.java:49)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGetProperty(AbstractCallSite.java:329)
	at Script_33051c2cd25b493e$_runScript_closure6$_closure12.doCall(Script_33051c2cd25b493e:170)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
	at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:274)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1030)
	at groovy.lang.Closure.call(Closure.java:427)
	at groovy.lang.Closure.call(Closure.java:406)
	at nextflow.script.WorkflowDef.run0(WorkflowDef.groovy:204)
	at nextflow.script.WorkflowDef.run(WorkflowDef.groovy:188)
	at nextflow.script.BindableDef.invoke_a(BindableDef.groovy:51)
	at nextflow.script.IterableDef$invoke_a.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
	at nextflow.script.BaseScript.run0(BaseScript.groovy:183)
	at nextflow.script.BaseScript.run(BaseScript.groovy:192)
	at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:236)
	at nextflow.script.ScriptRunner.run(ScriptRunner.groovy:242)
	at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:137)
	at nextflow.cli.CmdRun.run(CmdRun.groovy:372)
	at nextflow.cli.Launcher.run(Launcher.groovy:499)
	at nextflow.cli.Launcher.main(Launcher.groovy:670)

Thank you.

6

When you assign a process output channel to a variable, you no longer need the out key. See below the two ways in which we do it.

FOO()
FOO.out.table.view()

or

my_ch = FOO()
my_ch.table.view()
7

That worked, thank you!!!

1 Like
8

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.