Hi all, I had a nextflow process that used the move command to change the name of a folder inside the work directory, and had issues running it on AWS with fusion. Nextflow terminated with the Missing declared output files error, I guess because the mv command on the Fusion file system was taking too long and Nextflow did not wait for it to finish. Changing the mv command to cp solved the issue but of course in other infrastructure this makes the process slower. I was wondering if there is another possible solution to this.
1 Like
To move a “folder” on a block storage is an expensive operation because there is no concept of “folder” and no concept of “move”. So the implementation has to do a copy&remove of each single file inside the folder.
But this is supported by Fusion and the performance should be similar to the equivalent copy (that it’s a bit more performance but also has similar problems).
So, I guess that in your case there was some problem. Can you send me the .fusion.log file of that process that was doing the move?