Hi all, I am running nf-core ampliseq v2.11.0 and encountering very slow recovery of cached processes. Inspection of .nextflow.log
suggests that for two of the processes, RENAME_RAW_DATA_FILES and CUTADAPT, the message appears saying something like DEBUG nextflow.splitter.AbstractSplitter - Creating gzip splitter for: /path/to/some.fastq.gz
. As I understand, this indicates that the gzipped file is being read from the disk. This makes sense when the process is being run, but I am also getting the same messages when I already have cached processes and run nextflow with -resume
flag use. Unfortunately, this becomes a bottleneck in re-running later stages of the workflow as our filesystem IO sometimes becomes quite slow.
Thus, my question is - is it possible to recover cached processes without reading the gzipped files from the disk?