Hi there,
I generate a pipeline trace using following code:
trace {
enabled = true
overwrite = true
file = 'pipeline_trace.txt'
fields='task_id,name,process,status,memory,vmem,realtime,%cpu,rss,%mem,peak_rss,scratch,wchar,rchar,exit,start,complete,workdir'
}
It is fine.
I’d like to make it more informative: input/sample names so I can trace it back for any error.
For instance please see below log:
task_id name process status memory vmem realtime %cpu rss %mem peak_rss scratch wchar rchar exit start complete workdir
6 wes:FASTP (6) wes:FASTP CACHED 20 GB 2.4 GB 26m 39s 553.8% 1.6 GB 0.1% 1.6 GB - 22.1 GB 24.5 GB 0 2024-02-15 12:08:01.210 2024-02-15 12:35:01.503 /mnt/data1/users/sanjeev/nextflow/batch_enhanced/work/10/6714ce13809956207ea812e52e91e9
7 wes:FASTP (7) wes:FASTP CACHED 20 GB 2.5 GB 24m 30s 589.1% 1.6 GB 0.1% 1.6 GB - 20.7 GB 22.1 GB 0 2024-02-15 11:43:16.821 2024-02-15 12:08:01.350 /mnt/data1/users/sanjeev/nextflow/batch_enhanced/work/73/8d4cbc0fae687207417d34a2a81857
3 wes:FASTP (3) wes:FASTP CACHED 20 GB 2.4 GB 26m 37s 564.4% 1.6 GB 0.1% 1.6 GB - 20.4 GB 22.5 GB 0 2024-02-15 11:43:16.918 2024-02-15 12:10:08.277 /mnt/data1/users/sanjeev/nextflow/batch_enhanced/work/ba/3b9787695e1191a054622565c5697b
2 wes:FASTP (2) wes:FASTP CACHED 20 GB 2.4 GB 25m 582.5% 1.5 GB 0.1% 1.5 GB - 20.8 GB 23.2 GB 0 2024-02-15 11:43:16.919 2024-02-15 12:08:31.751 /mnt/data1/users/sanjeev/nextflow/batch_enhanced/work/2a/efbebc4bc4a7115c6e0b242b6a68d5
5 wes:FASTP (5) wes:FASTP CACHED 20 GB 2.4 GB 24m 29s 580.0% 1.5 GB 0.1% 1.5 GB - 20 GB 22.3 GB 0 2024-02-15 11:43:16.917 2024-02-15 12:08:01.110 /mnt/data1/users/sanjeev/nextflow/batch_enhanced/work/27/f46a9a993880d7f49204cc893eeaa9
For failed:
task_id name process status memory vmem realtime %cpu rss %mem peak_rss scratch wchar rchar exit start complete workdir
33 wes:applybqsr (1) wes:applybqsr FAILED 60 GB - 2m 44s - - - - - - - 3 2024-02-16 09:13:24.924 2024-02-16 09:16:09.281 /mnt/data1/users/sanjeev/nextflow/batch_enhanced/work/75/d82724810ebfb36d68790a39294568
37 wes:applybqsr (2) wes:applybqsr FAILED 60 GB - 2m 50s - - - - - - - 3 2024-02-16 09:13:24.925 2024-02-16 09:16:15.081 /mnt/data1/users/sanjeev/nextflow/batch_enhanced/work/d8/31027411d6d163f042514029d2fae2
45 wes:applybqsr (4) wes:applybqsr FAILED 60 GB - 2m 41s - - - - - - - 3 2024-02-16 09:16:01.628 2024-02-16 09:18:42.639 /mnt/data1/users/sanjeev/nextflow/batch_enhanced/work/2e/60b825cf3e0c05aa94cb41a35995f6
47 wes:applybqsr (5) wes:applybqsr FAILED 60 GB - 2m 33s - - - - - - - 3 2024-02-16 09:16:15.181 2024-02-16 09:18:48.650 /mnt/data1/users/sanjeev/nextflow/batch_enhanced/work/46/f8e56a0aef3c63df0c8f89b4a22a36
48 wes:applybqsr (6) wes:applybqsr FAILED 60 GB - 2m 54s - - - - - - - 3 2024-02-16 09:16:09.380 2024-02-16 09:19:03.220 /mnt/data1/users/sanjeev/nextflow/batch_enhanced/work/6a/512a344cdd9c73984c1389e6fa7645
How are the numbers assigned/decided?
wes:applybqsr (6) wes:applybqsr (5) wes:FASTP (3) wes:FASTP (6)
It is not very informative what failed at the failed step.
Is it the order of input samples/data?
Is it possible to have their names instead of a number?