I want to enable reporting on a production pipeline, but I’d like to put the report files in a different folder to avoid cluttering the main folder with dozens of files that will be there only for reference purposes.
I see that I can configure the report.file variable, but I only want to change the folder, not the filename itself. Is there any way to modify that folder?
In case I have to inform a full path, including filename, then how could I get the default name? Docs say the default is report-<timestamp>.html , but if I write that into the code, I’m pretty sure I will get a literal report-<timestamp>.html instead of the expected report-$timestamp.html . How can I mimmick the default?
See my nextflow.config below:
report.enabled = true
report.file = "report-" + String.format('%tF-%<tH:%<tM', java.time.LocalDateTime.now()) + ".html"
After running nextflow run hello I got a report file named report-2024-10-09-21:40.html. You can change the String.format argument so that you have it the way you want. And, with that, you can easily attach a folder name so that your report will be stored somewhere else, for example:
report.file = "my_folder_name/report-" + String.format('%tF-%<tH:%<tM', java.time.LocalDateTime.now()) + ".html"
There’s an issue for it here: Separate the store directory from the filename in the trace, dag, report, and timeline scopes · Issue #5364 · nextflow-io/nextflow · GitHub
but also just to add the default name also includes the session id I think too.
1 Like
That sounds quite good, but now I can preview a small synchronization issue.
If I use that approach to set up the names for the report, the trace and the DAG, all of them will have slightly different timestamps, as the java.time.LocalDateTime.now() will be called multiple times.
Which is the proper place for me to put a variable that will store the result of a single call to now() so I can use in the other scopes?
The best solution would be to solve the issue created by @mahesh.binzerpanchal and only have to modify the folder name without having to re-generate the name manually.
There’s also the fact that we don’t know what will happen if the folder do not previously exist. Will it be created? Can the creation be automated in some way? Doesn’t seem user friendly to ask a user that uses the pipeline for the first time directly from the repository to have to create a set of folders prior to execution.
The folders are created automatically. There’s no need to create anything prior to running the pipeline.
1 Like
So there’s no location to put that definition to avoid calling it multiple times? Also, that definition will match with the other files that go to the default folder? Or will they have different names due to different timestamps? 
@mribeirodantas
You could provide it as a command line argument. Something like
nextflow run main.nf --name_id 2024_Nov_12
And then adding ${params.name_id} to your filenames, as shown in my first answer.
Well, with that idea I could just assign inside the pipeline a variable called params.name_id with the value of now(), and that could serve the purpose of “unified timestamp”. But it does not solve the issue of having the same timestamp as the default files.
In other words, if I want to change the folder for the report but leave the trace file by default, report and trace files will have different timestamps and won’t be matched. To have the same timestamp in all the files, I will be forced to re-create all the filenames for all the possible files (report, trace, dag…) just because I need to change a single folder in one of them.
Unfortunately, for now, yes. That’s how I would imagine myself doing that.
UPDATE: For clarity. If you want to change the folder for one of these files, you just need to change the file of one of these files, e.g.
report.file = 'my_folder/report.html'.
The reason you have to change the filenames of all of them is because you want to have different filenames for all of them.
1 Like
Errr… no @mribeirodantas , I want to keep the default filenames of all of them and only change the folder. I do not want different filenames, I want the same filename that they originally had by default.
In the case where I only want to change a single folder (e.g. for the report, like in your example), then I have no way to re-generate the same filename it had by default, so the timestamps that appear in the other files (dag, trace, timeline) will be different.
The only difference I want between the different files is the prefix that says which type of report do we have (report, dag, trace, timeline), but not the timestamp. As I said, I want to keep the default filenames.
If I understood well, the default filenames for those in the default directory may have different timestamps compared to the one you want to store in a different folder. You want all of them to have the same timestamp, which implies changing the timestamps to match that in the new directory (whose timestamp you have control over). So, yes, you want to change all four filenames, as this is the only way one can do now.
Not ideal, but the team is working on the issue Mahesh brought and we should soon have a solution decoupling the path from the filename 
Yes, that’s it. I cannot get the timestamp used by the default filenames, so I have to create my own and that forces me to change all the other files. But it is not that I want to do that, I’m forced to do that to keep them in sync.
Let’s see if the issue the team is working on is resolved soon, as that is precisely the functionality I’m lacking :-/
Thanks!
1 Like