Hi guys, I have a problem with Nextflow secrets.
My process looks like this:
process FOO {
time '24h'
secret 'USERNAME'
secret 'PASSWORD'
input:
[...]
output:
[...]
script:
"""
#!/usr/bin/env bash
## dummy echo for testing
echo \$USERNAME
echo \$PASSWORD
[...]
"""
}
USERNAME and PASSWORD exist in nextflow secrets. When I run the process, both echo commands print empty strings. If I change the name of the secrets variable, it fails, so Nextflow recognizes that USERNAME and PASSWORD exists. But of course this doesn’t help me. When I look into the .command.log, I see the following telling text:
grep: ./secrets/.nf-ec9a753b-9de3-466f-b0ec-c5e6b1a407d6.secrets: No such file or directory
So, Nextflow wants to look into the secrets directory in the current work dir, but the secrets directory is in my project root!
Looking into the .command.run, my suspicion is confirmed, Nextflow changes into the workdir before trying to read from the secrets file in root dir:
[[ "${NXF_CHDIR:-}" ]] && cd "$NXF_CHDIR"
NXF_SCRATCH=''
[[ $NXF_DEBUG > 0 ]] && nxf_env
touch [workdir.... ]/.command.begin
set +u
# conda environment
source $(conda info --json | awk '/conda_prefix/ { gsub(/"|,/, "", $2); print $2 }')/bin/activate foo
set -u
export PATH="$PATH:/foo/bar"
source /dev/stdin <<<"$(cat <(grep -w -e 'USERNAME=.*' -e 'PASSWORD=.*' ./secrets/.nf-ec9a753b-9de3-466f-b0ec-c5e6b1a407d6.secrets))"
Am I using secrets wrong? Do I have to give the secrets dir as input? Because linking it into the workdir would work, but would contradict the use of secrets imo. Or is it a bug?